Calling Any Java Programmers - Computer Vision Project - Page 3

alti · 04-25-2006, 12:40 AM

here is a shot of what i get in jmstudio when i select capture and then a shot of what i get in your program.

sama · 04-25-2006, 02:10 AM

hey alti. black and white will not work with the algroithm out of the box as I'm currently using an HSI colour model. This is less sensative to light changes, since only changes to hue (colour) and saturation of that hue are considered, and intensity is ignored.

b&w is purely an intensity based colour model, so you have two choices. Either switch on the considerIntensity flag, or switch to an RGB colour model. I'll make this configurable when I get some time.

I'll also make it deal with B&W better since at the moment it'll do 3 cycles per pixel (R G and B) but it only needs to do one.

the code you need to change is in teh constructor of the motion detector I beleive. I'll have a look at the code tonight and externalise these settings, but lemme know how you go if you make the changes yourself.

PS Tidy your room!

alti · 04-25-2006, 07:26 PM

I changed the flag over and will check it out when i get home, thanks

sama · 04-26-2006, 01:48 AM

sorry, I got busy all last night!

try both hsi with intensity and the rgb options, to see what works best.

alti · 04-28-2006, 01:09 AM

alright i was able to get it to show the image from the camera in your app by changing tbe RGB flag to 1. have also played around with some of the other flags to see what happens, I changed the RGB elements flag from 3 to 1, thinkng that was what you were refering to above on the 1 vs 3 scans of the pixels.

I have messed with Hue, saturation, intesitity flag (both true and flase) and intesity

The best i get though is the image in the window, it does not seem to detec any motion. When it first starts up the image flickers light to dark and is grainy, it settles in to a better quality picture within 10 or 20 seconds, I image this is the code 'learning' the background.

I have to spend more time understanding yuor code, I think.

on a side note I have some color CCDs comming from Mark, should get to me next week.

sama · 04-28-2006, 02:23 AM

excellent stuff. i'll test the algorithm on a b&w setup (I'll just use 1 channel for now). I'm hoping to do some this weekend, but I'll have some time this monday for sure as it's a bank holiday in the UK.

the code should makes some sense if read in conjunction with computer vision primer.

it sounds like you're looking at the image learnt rather than the foreground. I'm not sure if you messed with what is displayed. when you move in front of the camera, do you appear at all? check these in ProcessorTest:

// image = motionDetector.getBackground();
image = motionDetector.getForeground();

if you view the background, that'll show you the learnt model, so you can see it's learning correctly. the foreground will be the current frame minus the background (+ noise and other light variations)

pyros · 04-28-2006, 08:57 AM

Wow, this is some good stuff. I'll take a look at the application and source this weekend.

I'm a java programmer but I haven't actively been doing standalone application development for about a year now (got suckered into web services development) but shouldn't be much of a learning curve to get back into. I'll try to check this out, seems pretty neat.

sama · 04-28-2006, 09:49 AM

I'm a web services sucker too. I do prefer standalone apps, but sadly the market demands otherwise.

welcome, and any questions, just ask.

sama · 05-01-2006, 07:15 AM

Alti, you were right. the RGB model didn't work. it does now. download the latest jar, which now includes the source code.

I've left it in RGB mode so it shoudl work for B&W cameras out of the box now.

my camera is very very bad, so bad, that it's leaving a trail of it's encoding artifcats, making it hell. when there is motion, the camera changes the blocks behind the moved object. probably in an attempt to save bandwidth and most definitely succeeding in making my life difficult... I need to buy a decent cam!

anyway, give this a shot and let me know how it goes.

alti · 05-01-2006, 08:51 AM

awesome, I will try it out tonight, I was able to work some more on it using a sony handicam at work, the nice part about that was that it could switch between b/w and color. I made some progress on understanding whats going on in the software as well. I saw the differences turning on and off the background and forground methods made, ect..

I will post more after i get it up and working tonight...

sama · 05-01-2006, 04:29 PM

hey...

I've found another bug which I'm trying to figure out. it seems the learning background part of the algorithm, it is acquiring new pixels, but it's not forgetting. I'm looking into this at the moment so I'm hoping to have it sussed soon.

I've also refactored the entire codebase to:
a) make the RGB much faster
B) make it more object oriented (I wrote the original code when I Was a final year

)
c) make it more other-coder-frielndly so you guys can join in quicker

alti · 05-01-2006, 04:57 PM

It works! I just had to turn on the considerIntensity flag!

btw the link at the top of the thread for the code download is busted, the one where you originally posted the code works.

sama · 05-01-2006, 07:57 PM

cool. Try this new version, it should work much faster and much more accurate. just let it learn the background for a good 30 seconds before you add anything.

Due to the background model being a running average, the frames learnt to begin with take a lot longer to "forget". So if there are objects that are in the frame to begin with and move after a few seconds, they linger in the model for a while before they're unlearnt.

I'd like to add a weighting method of some sort that gives newer frames higher value in the sample set.

I've discovered that for b&w, we only need intensity, so hue and saturation of colours can be ignored.

moreoever, the output can be colour (if the input was colour), even though the algorithm is working in b&w mode. the reason for this is that the pixel values of the foreground are copied in their full colour from the orignal image, but they are detected as intensity changes only from a b&w image. pretty cool eh?

anyhoos, have a mess around with this latest version when you get the chance. it's refactored and responsiblities of classes are much better seperated now, so it should be easier to grasp. I've still got more to do when I get the chance to.

some words of warning:
1) if the environmental lighting changes, the background model will be useless and will have to be learnt again.
2) the RGB mode sucks at the moment due to some bug. in general, HSI is a better model to use so looks like I'll focus on that for now, until RGB is required.

have a look at the constructor of ProcessorTest for some tweaking.

here's an image of the algorithm seperating my hand+water bottle from the rest of the room:

it's a bit choppy, but that's the purpose of the next part of the algorithm

coming soon:

1) smoothing prior to analysis - reduces the number of holes in the foreground
2) region analysis - grouping pixels that touch and placing a rectangle around them (with an id number)
3) region tracking - to make sure the id number stays associated with it's region
4) multimodal backgrounds - multiple background models based on different conditions. nighttime/day/overcast/sunny with autoswitching.

these will be very CPU intensive... but we'll see how it all pans out

all the above is still for stationary use. analysis whilst moving is completely new to me and I really need help there!

alti · 05-01-2006, 08:46 PM

sorry I should have been more specific, I got it working with the new code you posted above

great work, I will spend more time on understanding the new code, I had made some considerable headway on the old code and wish i had more time, but alas, this is finals week for me. I have also been reading up on optic flow, and as soon as i can get my head organized I will sit down and write up a good post.

sama · 05-02-2006, 06:20 PM

here's some small changes so that now it can clearly be seen how the algorithm is working:

for each pixel in the stream, a running mean is calculated. this is only for intensity and therefore is b&w as can be seen in the background image.

for each pixel in the stream, the difference between the background and the current value is caculated. That is:

SQRT[[meanPixelValue - currentPixelValue] * [meanPixelValue - currentPixelValue]] - a square root of a squared value ensures it's always positive

so now that we have the mean and variance, we say this (for each pixel):

Code:

            Mean
|------------|------------|
 < (-v * d) > < (+v * d) > 

where:
  v = variance
  d = deviation

if the new pixel value is within the mean and its variance, then it's a background pixel so ignore. if it's outside the mean and its variance, then it's a foreground pixel so extract it.

the variance is multiplied by the deviation, so the more deviation, the more variance is allowed.

not difficult at all!