computer vision stuff

kraft · 08-04-2007, 06:14 AM

Using the Intel's library, now face position recognition.
Now i'm searching to build model for first, recognize one face and only one, second build model of lips while speaking and make it able to recognize phonemes and by the way read lips. This plus voice recognition will kill the weak point of simple voice recognition.

So i will use the same tools or inspired from the Sphinx project for models of pronouciation, i will record some news on tv while people are speaking and try to get some vectors models from those records. In the end maybe it will help me to build a lips reading model.

ThunderStick · 08-04-2007, 07:25 AM

Good luck with that

kraft · 08-04-2007, 07:49 AM

It won't hurt trying

i found model of vectors for letters recognition using this library, so a particular mouth shape in a moment or a letter...still vectors

kraft · 08-04-2007, 09:37 AM

Video of the shape/face recognition

http://www.dailymotion.com/video/x2o...de-visage_auto

http://www.dailymotion.com/video/x2o...cond-test_tech

kraft · 08-04-2007, 12:09 PM

follow my eyes

http://www.dailymotion.com/video/x2o...w-my-eyes_tech

now read my lips...well not yet but with this sample code, i'll attempt to build a tool to first :
fill one array with points, second define a vector based on those points, in the end with a gentle and patient model bulding should give something that looks like a result

http://www.dailymotion.com/video/x2o...d-my-lips_tech

ReflexNPG · 08-04-2007, 01:47 PM

Nice idea!

What do you think about gesture recognition? Or about a "virtual console"? For example just slide the finger over the dash to change volume or seek.

Good luck with your project!

kraft · 08-04-2007, 02:01 PM

I thought about this but with a follow my eyes function but actually following an eye direction is much more difficult as it request a very sensitive camera or a high lighted environment, so a car is not a so good place for so tiny details....a finger to "touch" in the air some functions displayed on screen could be an elegant solution and a far "easier" to do

In addition of the follow me function, some morpho functions that could identify a particular face (opencv have them) could be amusing and a pretty good anti-theft system

kraft · 08-07-2007, 08:47 AM

Here is an attempt to make the thing follow my eyes and also detect in which direction i look.

To understand what is happening in the video, the circles appearing are here to materialize a region of interest the code detected. Inside of the circle you will see a segment starting from center and showing the heading of the move that happens.

After filtering i'm pretty sure that i can get it simplified and be usable as a mouse cursor or better as a virtual touch screen, still have work and probably it won't work but it's fun

Please note that the purpose of this video is not to be nice to watch, if i can do what i plan to do, the video shown actually won't even be visible. Actually it is just to show a feedback of what is happening inside the computer.

http://www.dailymotion.com/video/x2p...h-my-eyes_auto

http://www.dailymotion.com/video/x2p...ouvements_tech

kraft · 08-07-2007, 04:35 PM

This time a far better code very fluid with the real video layer, no delayed timings, looks like i was playing with some alive stuff or manipulating mercury but no directions detection, it is more suitable for gestures. The tracking abilities of this code and the direction detection of the other would probably make something really usable

http://www.dailymotion.com/video/x2p...e-video_webcam

ReflexNPG · 08-07-2007, 06:19 PM

your experiments are very interesting and i think an iphone like virtual interface is possible...

I've stopped my research, cause its very hard to use it in the car. There are so different light situations (even no light, when you drive @ night).
So it needs at least an infrared ilumination and when the sun is near the horizon - thats a great challenge

I have another experiment 4 you: I am trying to digitize the power line. All connected parts leave their traces in form of signals there. A neural network processes the signals and can find out if somethings "wrong". This could be an early detection of a possible malfunction. With enough data it would even be possible to identify the bad component. Second step would be the same with a microphone in the engine bay.

The system had 2 learn normal events like turning on lights or shifting gear etc

kraft · 08-07-2007, 08:43 PM

They are a lot of filtering options and ir camera with an ir projector can be used

Before i start something else i will try to go forward with this cv library, for lights etc it's already done, look inside this thread, there are some interesting pics with the motorhome

http://www.mp3car.com/vbulletin/off-...-attempts.html

KaZeR · 08-08-2007, 03:38 AM

This is really amazing.

Maybe you should try xvidcap or something like that, it would probably make better videos?

2k1Toaster · 08-08-2007, 03:51 AM

This is definately cool!

Good job so far!

shotgunefx · 08-08-2007, 04:03 AM

Interesting stuff, stop getting me thinking

As far as tracking the eyes, what about 3 (or more) IR leds around the screen, then a visible light filter over the camera, I would think you would be able to see them reflected in the iris if you were looking that way. Maybe you could use the change in the "cluster" position and size to determine eye position?