I’m enrolled in the “Research Topics in HCI” course this semester. We’ve been working on some really cool assignments. For the last one we built a computer vision based multitouch trackpad. We had to first build a frame for the device which was the hard part (for my teammate Krishna and me at least).
Multitouch device
We took the easy way out and used foamcore for the frame. Wood was an option, but I never paid attention in carpentry class. The basic setup is pretty simple. It uses the rear diffused illumination approach. The multitouch surface is a sheet of acrylic backed by a layer of tracing paper. There is a lamp below this surface which illuminates it with IR light. When one more fingers touch the surface of the semi-transparent surface, IR light is reflected and captured by the camera (a Playstation 3 camera with a visible-light-cut filter). The camera is connected to a computer via a USB interface.
On the software side of things, there’s obviously a camera driver, a layer above it to track finger movements and the application which uses the data. Since we had about 20-25 hours to build this, we focused only on the application. To track finger movements we used Community Core Vision, which exports the touch data in the TUIO protocol.
We implemented a photo browsing application in Processing which allowed the user to flip through images in a directory, zoom and pan and finally annotate images. I’ve embedded a screen cast of the application in action. The first 10 seconds shows how CCV tracks the finger movements. When the application is running, the touch points on the external device are denoted by white dot(s) on the image.
Tags: hci, multitouch