3 posts from 2008

Nothing like a good vacation to clear the mind and the schedule.  Not happy with the current state of computer vision apps for multi-touch and doing a bit of research, I've decided to embark on building TactTool, another multi-touch framework.  There are quite a few out there, the most notable is Touchlib (soon to be renamed CommunityCore), BBTouch, and the soon to be released Lux.

So why you writing one instead of contributing to one of the above?

I've thought long and hard about this and decided to write my own because I want to do it in Processing.  (I wanted to use Openframeworks but after some evaluation, I don't think it is quite there yet). I also was evaluating Python because of the built-in OpenCV support. I am actually torn about Processing v. Python because I'd like to use PyAMF to get output directly into Flash.  We'll see how well I can integrate Merapi into Processing.  I don't know what I don't know yet so hopefully we'll see if this was a good bet.

Once I get everything in good shape, I'll open source it.  Watch this blog for up to the week progress! I have written semi-functional blob tracking in as3 but was never quite happy with the results because it was too slow. I know about Pixel Bender and have created some filters and even a basis of an Air app. to read pbj files and apply it to a webcam feed. However Flash10 (according to this blog post)  is _not_ going to leverage the GPU for now (I understand the reasoning for Flash but hopefully that won't be the case for Air).

Using OpenCV, I won't have to do the blob tracking math. I can focus on creating cool filters and integrating with light sensors from Phidgets for dynamic filter adjustments.  For now, these are the requirements :

- load and save configurations
- multiple camera support
- define filter region
- output options : TUIO//XML via AMF/socket server

I'm excited that I was able to get an account at Shapeways.  It is a 3D printing service that will allow me to create some funky cold fiducials (yes, cheesy 80's reference) for my DI table.

Below is a screeny of an OpenCV port to Processing.  Blob detection in under 100 LoC. The top two screens are raw camera output, the bottom left is the background subtraction image and the bottom right is the processed blob detection with x/y coordinates.  Looks like it won't take much to port output via TUIO and send directly to Flash.  I have some light sensors laying around and may try to hook 'em up and adjust brightness/contrast/tolerance based on outside light conditions.

OpenCV is very cool and Processing makes it easily accessible and joy to code.

There goes the AIR app. I was building that was going to use Pixel Bender to filter FTIR setup for blob detection  . . . don't even get me started on face recognition (eye tracking anyone)? Download for yourself and check it out.