SoylentNews

Mind-reading, prediction and intelligence - these three books promise three superpowers. But do they make Kate Douglas a better person? SELF help. Now try to keep an open mind. This is the world's bestselling genre, with around 45,000 titles in print. What's more, it has intellectual credibility. Today's cutting-edge titles are written by respected scientists and journalists, and include a sophisticated blend of neuroscience, psychology and philosophy. It's only human to want to better oneself, if just to outsmart the competition. So we should applaud the trend to bring the latest insights from the human sciences to a wider audience. But, having read a fair few of these titles, I can't help noticing a distinct lack of personal growth. Sure, I know more about my brain, mind and behaviour, but am I a better person?

http://www.newscientist.com/article/mg22129581.000-can-these-popsci-selfhelp-books-make-you-smarter.html

As I've stated several times, cheap IP cameras are a little flaky. So even if you calculate the appropriate number of pictures to take in a 24-hour period, midnight will inevitably arrive and the script will still have a handful or two of pictures left to take.

So first we need to kill off the previous day's job. Here's the way to do that from cron:

00 00 * * * /bin/ps -o pid,args -U fliptop | /bin/grep grab_pic_tv-ip551wi.pl | /bin/grep -v grep | /usr/bin/xargs /bin/kill -15

Substitute the username your scripts run under for the -U switch and the name of your perl script in the 1st grep. Basically, this command takes all process that run under a particular user, greps out just the one process your script runs as, throws out any instances of the grep statement itself that may make their way into the results, then kills it w/ signal 15.

Next, at one minute after midnight, we start the next instance of the script for the new day:

01 00 * * * /bin/nice -n 19 /usr/bin/perl /home/fliptop/tv-ip551wi/grab_pic_tv-ip551wi.pl

Again, substitute the appropriate script location and name. I usually run these things nicely so they don't interfere too badly w/ other processes that run from time to time.

All that's left is to create the movie from the previous day's images:

03 02 * * * /bin/nice -n 19 /usr/bin/perl /home/fliptop/tv-ip551wi/ffmpeg.pl

As before, substitute the proper location and script name. You can run this at any time after midnight.

As you can imagine, watching surveillance video can be pretty boring since most of the time there's nothing noteworthy happening. However, once in a while you can see some damn interesting weather. One day last May I was reviewing some footage and thought to myself, "that would probably look pretty cool if it was edited a little w/ some Vivaldi."

Coming next, cleaning up your images.

Creating a "movie" from a bunch of stills is pretty simple when you have ffmpeg handy. As before, code below is in chunks and the full script can be found here.

Since we're storing all a day's images in a particular directory, all we need to do is glob them up and pipe a concatenate to ffmpeg.

#!/usr/bin/perl -w

use strict;
use File::Glob ':glob';
use Date::Calc qw{ Today Add_Delta_Days };
use Data::Dumper;

use constant DEBUG => 1;
use constant BASE_DIR => '/home/fliptop/tv-ip551wi/';

We'll use File::Glob to gather all the image filenames into a list. We'll also need the Add_Delta_Days method to figure out yesterday's date so we know which files to glob (this code gets run sometime after midnight and operates on the previous day's pictures).

Please note that this script is designed to work on images from just one camera. It could easily be modified to handle multiple directories for more than one camera by setting the BASE_DIR constant to an array of directories then looping over them with the ffmpeg code below.

open LOG, '>>/tmp/ffmpeg_tv-ip551wi.log' or die "can't open log file: $!";

my @Yesterday = Add_Delta_Days(Today, -1);

# does the dir exist?
my ($year, $month, $day) = @Yesterday;
my $dir = sprintf "%s%s/%02d/%02d", BASE_DIR, @Yesterday;
my $movie_dir = sprintf "%s%s/%02d", BASE_DIR, $year, $month;

unless (-e $dir) {
  printf LOG "base directory %s does not exist!\n", $dir if DEBUG;
  die;
}

As always, open a log file for debugging purposes. After we figure out yesterday's date, make sure it exists and die if it doesn't.

my $files = sprintf "%s/*.jpg", $dir;
my @list = bsd_glob($files);

printf LOG "found %s images in directory %s\n", scalar(@list), $dir if DEBUG;
my $mpg = sprintf "%s/%s%02d%02d.mpg", $movie_dir, @Yesterday;

If the directory exists, we use the bsd_glob method to put all the filenames into the @list array. We'll give the final movie a name that corresponds to yesterday's date as well.

print LOG "creating mpg from images...\n" if DEBUG;
my $resp = qx{ /bin/cat $files |
               /usr/bin/ffmpeg -f image2pipe -sameq -vcodec mjpeg -i - -y $mpg };
print LOG "finished creating mpg\n" if DEBUG;

if ($resp) {
  printf LOG "unable to create mpeg: $resp\n" if DEBUG;
}

close LOG;

exit();

Here we use the cat command and pipe the output to ffmpeg, which will create a mjpeg movie. If you take one picture every 10 seconds over a 24-hour period, the final movie will be about 5 minutes long. See the ffmpeg man page for more information.

Coming next, invoking it all with cron.

Last night I cloned the git repository. Today I was perusing some static pages on the site and I noticed there's the term Slashdot in a few places, most notably on moderation page. But in the version I pulled down these have been changed to SN.

How long does it take to get changes merged down to the site's main trunk?

The iGuard IP250E requires cookies for authentication. Modifying Part 3's script is pretty simple. Below are just the additional code chunks. Whole script is available here.

use HTML::Form;
use HTTP::Cookies;

Add these to your includes.

my $ua = LWP::UserAgent->new;
$ua->cookie_jar(HTTP::Cookies->new(file => "/tmp/iguard-ip250e.txt", autosave => 1));
$ua->agent("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.13) Gecko/2009080317 Fedora/3.0.13-1.fc10 Firefox/3.0.13");

Here we tell our useragent object where to store any cookies.

my $uri = 'http://192.168.1.243/image.cgi';
my $res = $ua->request(HTTP::Request->new(GET => $uri));
my $status = $res->status_line;
printf LOG "unable to process initial cookie request, status code is: %s", $status
  unless $status eq '200 OK' || $status eq '302 Found';

my ($form) = HTML::Form->parse($res);
print LOG "form: ", Dumper($form) if DEBUG;

$form->value('$login_un' => 'username');
$form->value('$login_pw' => 'password');

On the iGuard IP250E, there's a simple login form returned if you request image.cgi We parse the form and assign the username/password params.

$res = $ua->request($form->click);
$status = $res->status_line;
printf LOG "unable to submit username/password, status code is: %s", $status
  unless $status eq '200 OK' || $status eq '302 Found';

$uri = 'http://192.168.1.243/showimg_pda.cgi?cam=1';

my $im = Image::Magick->new();

Submit the form by issuing click. The URI showimg_pda.cgi?cam=1 is the way to return just the image from the iGuard. The rest of the code is the same.

First off, a correction - in my last journal entry I said the Security Labs SLW-164 was around $220. I came across the invoice and it was $160.

So let's write some code!

I'll start with the TrendNet IP501P code. It's simpler than the iGuard because it uses basic authentication instead of cookies. The code to grab the pictures should do the following:

- Since it's not infrared, it should start requesting images about 45 minutes before sunrise and finish 2 hours after sunset (I came about these 2 time frames through much experimentation, YMMV).

- Each day's pictures should be stored in its own directory.

- Test the image request result and re-request it if it's not a jpeg.

- Add a timestamp to each image.

- While testing the code for a while, make it easy to log any issues with a debug hook.

Let's take a look at each chunk of code. The full script can be found here.

#!/usr/bin/perl -w

use strict;
use LWP::UserAgent;
use HTTP::Request;
use Date::Calc qw{ Today Today_and_Now Delta_YMDHMS Add_Delta_YMDHMS System_Clock };
use Astro::Sunrise;
use Image::Magick;
use Data::Dumper;

Pretty standard start to a perl script. Date::Calc is used to figure out the time and date for directory creation and also for the timestamp that's added to each image. In the case of a non-infrared camera, it'll also be used to determine if we're even going to request an image at all. Lastly, we'll use it to figure out if it's daylight saving time or not. Astro::Sunrise is used to calculate the sunrise and sunset times for my longitude and latitude. Image::Magick will annotate the timestamp to each image.

use constant DEBUG => 1;
use constant PAUSE => 10;
use constant BASE_DIR => '/home/fliptop/tv-ip501p/';

open LOG, '>>/tmp/grab_pic_tv-ip501p.log' or die "can't open log file: $!";
print LOG "\n------------------------------------------\n" if DEBUG;
printf LOG "Today is %s/%02d/%02d\n", Today if DEBUG;
print LOG "calculating 45 minutes before sunrise and 2 hours after sunset: " if DEBUG;

Pretty self explanatory, declare a few constants like the debug hook, how often we want to request an image, and where the images will be stored. Then we open the log file and give it some initial info.

my @time = System_Clock;        # dst is last value in list
my ($sunrise, $sunset) = sunrise(Today, xx.xx, yy.yy, 5, $time[8]);
my ($srh, $srm) = split /:/, $sunrise;
my ($ssh, $ssm) = split /:/, $sunset;

my @sr_adjusted = Add_Delta_YMDHMS(Today, $srh, $srm, 0, 0, 0, 0, 0, -45, 0);
my @ss_adjusted = Add_Delta_YMDHMS(Today, $ssh, $ssm, 0, 0, 0, 0, 2, 00, 0);

printf LOG "%s:%s %s:%s\n", $sr_adjusted[3], $sr_adjusted[4], $ss_adjusted[3], $ss_adjusted[4] if DEBUG;

my @start_time = ($sr_adjusted[3], $sr_adjusted[4], '00');
my @end_time = ($ss_adjusted[3], $ss_adjusted[4], '00');

Here we figure out the start/end times for requesting images. sunrise is a method in Astro::Sunrise that returns a two scalar times for sunrise/sunset when you pass in the date, your longitude, latitude, timezone, and a DST flag (which we get from the System_Clock method in Date::Calc). Make sure you substitute your own longitude/latitude values for xx.xx and yy.yy. The start/end times we'll use are stored in @start_time and @end_time.

my $uri = 'http://username:password@192.168.1.246/IMAGE.JPG';
my $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.13) Gecko/2009080317 Fedora/3.0.13-1.fc10 Firefox/3.0.13");

Here we set up the URL we'll use to request images. Note that with basic authentication the login credentials are part of the URL.

my $im = Image::Magick->new();

for (0..86400/PAUSE) {
  my @Today = Today;
  my @Today_and_Now = Today_and_Now;

  # create dir if it doesn't exist
  my $dir = sprintf "%s%s/%02d/%02d", BASE_DIR, @Today;

  unless (-e $dir) {
    printf LOG "creating base directory %s\n", $dir if DEBUG;

    my $resp = qx{ /bin/mkdir -p $dir };

    if ($resp) {
      printf LOG "unable to create image dir: %s\n", $resp;
      die;
    }
  }

After instantiating an Image::Magick object ref, we'll execute the next set of code for every image request. In this case, it'll be 86,400 seconds per day divided by the number of seconds between each request. First is to make sure the directory we'll be storing the images in exists. If it doesn't, create it. If we can't create it, something is wrong and there's no point in continuing.

  if ( &_take_pic(\@Today, \@Today_and_Now) ) {

    my $not_jpeg = 1;

    my $res = $ua->request(HTTP::Request->new(GET => $uri));
    my $status = $res->status_line;

    unless ($status eq '200 OK' || $status eq '302 Found') {
      printf LOG "unable to request image, status code is: %s\n", $status if DEBUG;
    }

&_take_pic is a method we'll show later that figures out if the current time is between the sunrise/sunset times we declared earlier. We set a $not_jpeg flag to TRUE so we can figure out (later) if we need to request the image again (I mentioned it before, the firmware on these cheap cameras if flaky and occasionally it doesn't return what you're looking for). Then we request the image, and if it doesn't return OK or FOUND we log the status code that is returned.

    while ($not_jpeg) {
      my $headers = $res->{_headers};
      my $ct = $headers->{'content-type'};

At first, $not_jpeg is TRUE, so we'll look at the content-type of the request result.

      if ($ct eq 'text/plain') {
        print LOG 'text/plain (no image) result: ', Dumper($res), "\n" if DEBUG;
        sleep 2;

        # request it again
        $res = $ua->request(HTTP::Request->new(GET => $uri));
        $status = $res->status_line;

        unless ($status eq '200 OK' || $status eq '302 Found') {
          printf LOG "unable to request image, status code is: %s", $status, "\n" if DEBUG;
        }
      }

If the content-type is plain text, we didn't get the result we were looking for, so we sleep for a couple of seconds and request it again.

      elsif ($ct eq 'image/jpeg') {
        print LOG 'image/jpge found: ', $headers->{'client-date'}, "\n" if DEBUG;
        $not_jpeg = 0;
      }

If it returns a jpeg image, all is well, and we can exit the loop.

      else {
        print LOG "unknown content-type: ", $ct, "\n" if DEBUG;
        sleep 2;

        $res = $ua->request(HTTP::Request->new(GET => $uri));
        $status = $res->status_line;

        unless ($status eq '200 OK' || $status eq '302 Found') {
          printf LOG "unable to request image, status code is: %s", $status, "\n" if DEBUG;
        }
      }
    }

And just in case something else is returned (besides plain text or an image), sleep a couple of seconds and try again. Now that I'm looking at all this again, it would probably be easier/simpler to do some kind of recursive call to a method for requesting the image instead of the if-then-else approach. Maybe I'll look into that later...

    my $content = $res->{_content};
    my $filename = sprintf "%s/%s.jpg", $dir, time;

    printf LOG "writing image %s\n", $filename if DEBUG;

    open BIN, ">$filename" or die "can't open binary output file: $!";
    binmode BIN;
    print BIN $content;
    close BIN;

The content should be a binary image file, so give it a unique filename and write it to disk.

    my $x = $im->Read($filename);
    print LOG "$x: $filename\n" if $x && DEBUG;

    $x = $im->Annotate(
      text       => sprintf("%s/%02d/%02d %02d:%02d:%02d", @Today_and_Now),
      font       => '/usr/share/fonts/dejavu/DejaVuSans-ExtraLight.ttf',
      pointsize  => 12,
      stroke     => 'white',
      undercolor => 'blue',
      fill       => 'red',
      x          => 510,
      y          => 465
    );
    print LOG "$x: $filename\n" if $x && DEBUG;

    $x = $im->Write($filename);
    print LOG "$x: $filename\n" if $x && DEBUG;

    @{$im} = ();        # clear the buffer
  }
  sleep PAUSE;
}

Now we use Image::Magick to annotate the timestamp. I found the red/white/blue combination to be the easiest to read. I configured my camera to return images that are 640x480, so I'm placing the timestamp in the lower right-hand corner (note that when working with Image::Magick (0,0) is the upper left-hand corner!). Then we pause and do it all again.

close LOG;

exit();

sub _take_pic {
  my ($todayref, $today_and_nowref) = @_;
  my @Today = @{ $todayref };
  my @Today_and_Now = @{ $today_and_nowref };

  my @after_start = Delta_YMDHMS(@Today, @start_time, @Today_and_Now);
  my @before_end = Delta_YMDHMS(@Today_and_Now, @Today, @end_time);

  # time is array elements 3, 4 and 5
  return 0 if $after_start[3] < 0 || $after_start[4] < 0 || $after_start[5] < 0;
  return 0 if $before_end[3] < 0 || $before_end[4] < 0 || $before_end[5] < 0;

  return 1;
}

The _take_pic subroutine uses the Delta_YMDHMS method of Date::Calc to see if the current time falls between what we declared earlier as the start/end time based on our sunrise/sunset calculation.

Coming up next, cookie-based authentication.

After posting my last journal entry, a reader informed me about cheap IP cameras available on Amazon, and had a few comments regarding using other methods to access the mjpeg stream. He also informed me about the openipcam.com website where one can find many of the commands that can be used to configure most IP cameras using nothing more than wget commands.

While his information was useful and I'll definitely look into it in more depth at a later date, at this time I'm intending to use the cameras to watch deer, therefore I don't need to access the mjpeg stream and only really need one picture every 5-10 seconds. Additionally, I want to add a timestamp to every picture so I know what time the deer show up and can plan my next bowhunt accordingly. Also, 1 of the IP cameras I happened to have lying around uses cookie-based authentication instead of basic authentication, and I'm not sure how I'd do that using wget (if there is a way, please let me know). For these reasons I'm going to continue writing about the perl code I wrote to grab the images.

When I'm done, I will certainly entertain suggestions and collaborate with any users who feel like my methods can be improved upon. If the Soylent community is anything like the /. user base, I'm quite certain there's a ton of readers out there who are more adept at this and can suggest better techniques to accomplish what I'm trying to do.

Now, onto the cameras I used. I had 2 old IP cameras lying around that I started with. Neither have WIFI or infrared. But, I wanted to use them to see if any of this was possible before I invested in additional cameras.

The first camera I tried was an old TrendNet IP501P. It's a POE (power-over-ethernet) model and has a low-profile design that would be perfect for an enclosure to use outside, but since it doesn't have infrared I decided to just put it in the window and point it at my barn. This camera is quite old (I don't remember when I bought it, perhaps 5 years ago?) and its firmware is not that great. Many times when the admin interface comes up it looks like it's having difficulty displaying all the images, links, etc. I made sure it had the latest firmware installed and it didn't fix the problem. No worries, though because I'll just be pulling a static image and that seems to work OK. The electrical tape covers the power and activity lights that cause a reflection when it's put near a window.

The 2nd old camera is an iGuard IP250e which is (was) touted to be the lighest and smallest IP camera on the market. The picture shows it mounted inside a Q-see outdoor enclosure. Unfortunately, the firmware of this camera is even worse than the IP501P. Many times when trying to grab an image it just returns "No image" instead of a jpeg. Additionally, it uses cookie-based authentication which makes accessing it from a script more cumbersome than basic authentication. Nevertheless, it was lying around and I used it.

After getting my code working and testing it for a couple of months, I invested in 2 WIFI infrared cameras. The TrendNet IP551WI cost about $60 from my wholesaler, and it uses basic authentication just like the IP501P. It has a wide-angle lens that can be focused, and the infrared illuminates a very dark area quite well. It also has an auto-sensor (at the top between the 2 IR lights) that toggles the IR off when the light is adequate and on when it's too dark.

I wanted to get an outdoor WIFI infrared camera next, and I couldn't find a cheap one at my wholesaler, so I bought a Security Labs SLW-164. I think it cost about $220. I don't recommend this camera for 2 reasons. First, it's infrared seems very weak. Second, the camera's lens is much more narrow angle than the TrendNet models. Compare these 2 pictures taken side-by-side in my pitch-black basement using the IP551WI and SLW-164. In the SLW pic you can barely make out the license plate of the car, but the IP551 brightly illuminates the entire half of my basement (the distance from the camera to the door in front of the car is about 40 feet). Notice the cat sitting on top of the car, eyes glowing. Not even visible in the SLW picture!

I called tech support at Security Labs and they told me there's no way to adjust the camera's settings to get a better low-light picture. They said I need to use it in conjunction with a regular light. BTW, both cameras are rated at 0 lux. When I eventually mount the SLW outside I'll put a motion detector on the spotlight where it'll be used to compensate for the poor IR illumination.

Coming up next, perl code.

A year ago I wondered if it was possible to use a cheap IP camera I had to record the wildlife activity (mainly deer) on my property. As you may know, most of the cheaper (<$100) IP cameras have shitty firmware and require Java or (even worse) ActiveX in order to view video. Additionally, the software that comes w/ these cameras requires Windows, and since I run Linux that's not going to work.

For what I wanted, though, I figured continuous footage was overkill and started to investigate if it were possible to just pull an image every 10 seconds or so, store it somewhere, then use ffmpeg to tie them all together into an mpeg once a day. All IP cameras have the ability to grab snapshots so I set about figuring out how to accomplish this.

Turns out it's pretty simple. Using a little perl and a few cron commands I was able to get everything working and, once it was put into motion, needed very little babysitting to work.

So over the next couple of weeks I'm going to write about what hardware I used (and what I don't recommend using), how I set my server up, show the code I used, and some results.

First off, some specifications:

• the cameras must be cheap, less than $100
• I wanted to use wireless cameras so I didn't have to run a bunch of CAT5 throughout my house
• the camera I had at the time did not include infrared, but the ones I wound up buying later did, and they were rated at 0 lux
• I chose to control everything w/ an old computer running CentOS 6 (basic server setup)

Coming up next, the cameras I used.

Local Motors is set to manufacture its first electric vehicle built through 3D-printing technologies.

The automotive firm says that AMT -- the Association For Manufacturing Technology -- is the first customer to line up for its 3D-printed car. The vehicle will be built and shown off at the International Manufacturing Technology Show 2014 in Chicago, Illinois, running 8 - 13 September this year.

Does it come in a flying version?