Controlling my laptop with hand gesture
For months, I’ve been talking with my friend Canard about a project of interacting with a computer without touching it. We are convinced that it can be done with a simple webcam and that the “depth” information is not needed for simple interactions. (So no need to buy a kinect or two webcams).
Yesterday we started hacking on this. We are using OpenCV. We first discussed about the different steps of doing it. But as we explored OpenCV, we changed some of these previous decisions because we found some built-in functions.
After investigation, we decided that the first part of our system will be using a method named “Real-time Motion Template Gradients” by James Davis and Gary Bradski (you can read the paper here)
The very first step was to get the difference between two frames of the webcam video, what you can see in the next image is the movement : black means no movement, colored means there was a difference between the two frames:
And here we are, our very first demo:
The video analysis simply execute keyboard shortcuts to interact with the OS.
Canard worked on a Windows/Linux keystroke system while I created a naive gesture detection algorithm. We then just had to plug the two.
You can check out the code (and fork us) on our github repository.