Computer Vision I: Introduction - Stanford AI Lab

Computer Vision I: Introduction - Stanford AI Lab

Segmentation of Time Varying Images (and tracking) J. Kosecka, CS 223b Some slides From Computer Vision book D. Forsythe, J. Ponce Technique: Shot Boundary Detection Find the shots in a sequence of video shot boundaries usually result in big differences between

succeeding frames Strategy: compute interframe distances declare a boundary where these are big J. Kosecka, cs223b Possible distances frame differences histogram differences block comparisons edge differences Applications:

representation for movies, or video sequences find shot boundaries obtain most representative frame supports search Technique: Background Subtraction If we know what the background looks like, it is easy to identify interesting bits

Applications Person in an office Tracking cars on a road surveillance J. Kosecka, cs223b Approach: use a moving average to estimate background image subtract from current frame

large absolute values are interesting pixels trick: use morphological operations to clean up pixels Image Differencing J. Kosecka, cs223b Image Differencing: Results 1 frame difference

J. Kosecka, cs223b 5 frame difference Motion detection Background subtraction create an image of the stationary background by averaging a long sequence for any pixel, most measurements will be from the background computing the median measurements, for example, at each pixel, will with high probability assign that pixel the true background intensity - fixed threshold on differencing used to find

foreground pixels can also compute a distribution of background pixels by fitting a mixture of Gaussians to set of intensities and assuming large population is the background - adaptive thresholding to find foreground pixels J. Kosecka, cs223b A 300-Frame Sequence with a Busy Background click to start movie J. Kosecka, cs223b

Motion Detection Difference a frame from the known background frame even for interior points of homogeneous objects, likely to detect a difference this will also detect objects that are stationary but different from the background typical algorithm used in surveillance systems Motion detection algorithms such as these only work if the camera is stationary and objects are moving against a fixed background J. Kosecka, cs223b

Background Subtraction: Results Confidence corresponds to gray-level value. High confidence bright pixels, low confidence dark pixels. J. Kosecka, cs223b Background modeling: colorbased At each pixel model colors (r,g,b) or gray-level values g. The following equations are used to recursively estimate the mean and the variance at each pixel: = + (1 ) z t+1

t+1 t t2+1 = ( t2 + ( t+1 t ) 2 ) + (1 )( zt+1 t+1 ) 2 where zt+1 is the current measurement. The mean and the variance can both be time varying. The constant is set empirically to control the rate of adaptation (0< A pixel is marked as foreground if given red value r (or for any other say g) or b) we have | r measurement,

|> 3 max( , t J. Kosecka, cs223b r rcam Background model rcam is the variance of the camera noise, can be estimated from image differences of any two frames. If we compute differences for all channels, we can set a pixel as foreground if any of the

differences is above the preset threshold. Noise can be cleaned using connected component analysis and ignoring small components. Similarly we can model the chromaticity values rc, gc and use them for background subtraction: rc=r/(r+g+b), gc=g/(r+g+b) J. Kosecka, cs223b Background model: edge-based Model edges in the image. This can be done two different ways: Compute models for edges in a the average background image Subtract the background (model) image and

the new frame; compute edges in the subtraction image; mark all edges that are above a threshold. The threshold can be learned from examples The edges can be combined (color edges) or computed separately for all three color channels J. Kosecka, cs223b Foreground model Use either color histograms (4-bit per color), texture features, edge histograms to model the foreground Matching the foreground objects between

frames: tracking Can compare foreground regions directly: shift and subtract. SSD or correlation: M, N are two foreground regions. n C= n M(i, j N(i, j i j

= n n = n n [ M(i, j 2 N (i, j2 ]/ 2 J. Kosecka, cs223b

i = j= i = j= Histogram Matching Histogram Intersection min{h (i), h (i)} I (h , h ) = max{h (i), h (i)} c b c

b c b i i Chi Squared Formula ( hc (i ) hb (i )) ( hc , hb ) =2 hc (i ) + hb (i )

i 2 J. Kosecka, cs223b 2 Surveillance: J. Kosecka, cs223b Interacting people Background Subtraction

J. Kosecka, cs223b Background Subtraction J. Kosecka, cs223b Courtesy G. Hager Courtesy G. Hager J. Kosecka, cs223b Courtesy G. Hager J. Kosecka, cs223b

Adaptive Human-Motion Tracking Acquisition Decimation by factor 5 Motion detector Validation Skin color detector Grayscale convers. Motion Skin

colorpresence presence RGB to HSV convers. Image differencing Big contour presence Hue-saturat. Limiter Motion history im. Segmentation Skin color binary im.

Image closing Average travelled distance Segmentation Adaptation Continuous adaptation Motion initialization Tracking Distance scoring Contour to target

assignment Event creation J. Kosecka, cs223b Narrative-level output Adaptive Human-Motion Tracking J. Kosecka, cs223b QuickTime and a YUV420 codec decompressor are needed to see this picture.

J. Kosecka, cs223b Courtesy G. Hager

Recently Viewed Presentations

  • A rights-based, child-friendly school: Is a child-seeking school

    A rights-based, child-friendly school: Is a child-seeking school

    Is a child-seeking school actively identifying excluded children to get them enrolled in school and included in learning treating children as subjects with rights and States as duty-bearers with obligations to fulfil these rights demonstrating, promoting, and helping to monitor...
  • Questions and Answers

    Questions and Answers

    42053' N: 42 degrees 53 minutes Latitude North. 78053' W: 78 degrees 53 minutes Longitude West. 2 (2) Rectangular Coordinate Systems. Also referred to as Planar, Cartesian, and Grid coordinate system. It converts Earth's curved surface onto a flat map...
  • Data and AI OverviewChris MitchellRishi Arora

    Data and AI OverviewChris MitchellRishi Arora

    Bridge the gap between IT and the business, allowing everyone to contribute their insights, tags, and descriptions. Intuitive search and filtering to understand the data sources and their purpose. Let your data live where you want; connect using tools you...
  • The Nurses Role in the Use of Medical Aesthetics to Enhance ...

    The Nurses Role in the Use of Medical Aesthetics to Enhance ...

    The Nurses Role in the Use of Medical Aesthetics to Enhance Patient Satisfaction with Post Treatment Oncology Survivorship. Jennifer Cash, ARNP, MS. ... Small, R. & Hoang, D. A Practical Guide to Botulinum Toxin Procedures. Lippincott, Williams & Wilkins, Philadelphia...
  • Quizzes in general

    Quizzes in general

    As usual for any quiz or test, consider the directions closely. Directions: Each of the questions here relates to "The Promises." Based on your reading of the story, write an answer to each question. Make sure that your answers are...
  • 9-5 Inscribed Angles - teachers.henrico.k12.va.us

    9-5 Inscribed Angles - teachers.henrico.k12.va.us

    Exterior Angles An angle formed by two secants, two tangents, or a secant and a tangent drawn from a point outside the circle. Exterior Angles 1 j k 1 j k 1 j k Exterior Angle Theorem The measure of...
  • Practice Guides and Aids - Westlaw

    Practice Guides and Aids - Westlaw

    Practice Guides and Aids Practice Guides and Aids A wide variety of practice guides and materials are available on Westlaw. Some practice databases are state-specific, some are topical, and some are general in nature. Practice aids include texts, treatises, forms,...
  • 600 BC Joseph 1 Nephi Manasseh Lehi and

    600 BC Joseph 1 Nephi Manasseh Lehi and

    Moroni and Pahoran persuade 4,000 Lamanites to join the people of Ammon as they march to retake the city of Nephihah from the Lamanites. King Ammoron was slain by Teancum in the city of Nephihah. After many Lamanites were killed,...