Webcam + Timelapse + OpenCV

Since the start of the year, I’ve made my laptop automatically take a photo of me every minute. A while ago I made a time lapse video out of it, but used a frame averaging technique, so it’s a bit blurry, but still interesting.

The main problem is that my head is not positioned exactly in the same spot every time the photo is taken, so the average frame becomes blurry. I looked into facial recognition and found OpenCV which is a free “computer vision” library written in C++. OpenCV comes with face detection, eye detection, and all sorts of different image and video processing tools. It also has wrappers for languages such as Python, Ruby and C#, so I decided to give the Python interface a go.

Compiling it took a fair amount of work on OS X with working Python bindings, and it was completely worth it. Once OpenCV was ready to go, detecting a face was trivial. It gives you a rectangle for each face that were detected in the image.

Usually it’s pretty accurate, but when I attempted to run this through all the webcam frames collected, it would sometimes think an inanimate object was a face:

There are ways to tweak how sensitive the detection is, but I thought it would be cool to also detect my eyes. Eye detection appears to be more inaccurate, so I made it count if there are 2 eyes detected within the detected face rectangle. The white squares are detected eyes (the label says “in”) and the blue squares (“out”) are eyes detected outside the face rectangle. The eye detected on the right does creepily look like an eye!

Once a correct face is detected, the position of the face is moved to the center of the image, so that my head is always in the center for the time lapse. I filtered out any images that didn’t detect a face or if the face was too small on the screen. I also set it to only pick 10 images per day, ending up with 1280 frames. It took me a weekend to do all of this, and here’s the video:

All the data is backed up to Amazon S3 just incase the laptop dies, so I plan on continuing this project, possibly for the rest of my life! I’ll be interested to hear what other ideas you have that I can do with these images. My next plan is to do more frame-average videos similar to the first video posted.

The code for this is fairly messy since it’s all mostly a hack, but you can access the script on github. If you have any questions, don’t hesitate to ask me on twitter: @gakman.

5 thoughts on “Webcam + Timelapse + OpenCV”

  1. Ok, seriously though: It’s noticeable that you’re not always looking at the camera, so it gets a bit jumpy sometimes. I was wondering if you thought about using some sort of relationship between eye size/mouth or eye position/face size size to detect if you’re looking to your monitor or to the laptop (not sure if it is easy to get that, looking at the sample picture.)

    Another use for that would be resize the pictures, so the face size won’t change much…

    1. We’re thinking alike :) I did originally plan on rescaling the image based on the detected face size, and also rotating the images so the eyes are lined up horizontally. I just ran out of time for the weekend!

      I’m not sure about the part about detecting when I’m facing the camera, I don’t remember seeing a thing in OpenCV to do it, but I’m sure it’s not too hard..

  2. I suggest you try to train some ML system to draw images procedurally from these samples. These images are all very similar, so there are a lot of patterns that the system could exploit.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>