Geo-Referenced Dynamic Pushbroom Stereo Mosaics for 3D and ...

Geo-Referenced Dynamic Pushbroom Stereo Mosaics for 3D and ...

C CVC L Sensor and Vision Research for Potential Transportation Applications Research Initiatives for Nano and Hi-tech Research January 18, 2006 Zhigang Zhu Visual Computing Laboratory Department of Computer Science City College of the City University New York C CVC L Multimodal Human/Vehicle Signatures Collaboration with AFRL Prof. Tom Huang at UIUC Prof. George Wolberg at CCNY IR c a m e r a H u m a n in c a r c o n tro l Noisy LDV signals Low-res, non-front faces Target detection?? Transportation???

c o n tro l V is ib le / I R v id e o H u m a n /F a c e D e te c tio n & T r a c k in g Remote sensing/hearing Multimodal signatures S e n s o rs , H um an & M ic r o p h o n e E n v ir o n m e n t P T Z cam Goal Challenges V ib r o m e t e r (L D V ) L D V T a r g e tin g a n d F o c u s in g V id e o V is u a l L ip R e a d in g

A c o u s t ic s ig n a ls M u ltim o d a l D e te c tio n (S e n s in g ) A u d io LD V V o ic e R e a d in g L ip m o t io n f e a tu r e s S p e e c h & fe a tu re s M u ltim o d a l R e c o g n itio n (A n a ly s is ) M u ltim o d a l F u s io n fo r E n h a n c e m e n t & R e c o g n itio n Face & fe a tu re s E nhanced speech (a n d te x t) M u ltim o d a l (A /V /T ) A lig n m e n t O u tp u t 1 T ext T a lk in g H e a d /F a c e S y n th e s is

M u ltim o d a l P r e s e n ta tio n (S y n th e s is ) O u tp u t 2 T a lk in g Face (A v a ta r) H um an U ser Figure 1. Multimodal remote hearing by human and machine: system diagram 2 C CVC L Major Sensor Acquisition Polytec Laser Doppler Vibrometer (LDV) OFV-505 Sensor Head, OFV-5000 Controller Tip-Tilt Stage VIB-A-P05 with Telescope FLIR ThermoVision Camera ThermoVision A40M Infrared Camera Canon Color/IR PTZ Camera Canon VC-C50i Low Light IR PTZ Camera Other video cameras Omnidirectional cameras, stereo head

Camcorders, webcam, CMOS sensors 3 C CVC L Polytec Laser Doppler Vibrometer Sensor Head OFV-505 HeNe (Helium-Neon) laser, =632.8 nm, W<1 mW OFV-SLR lens (f=30mm) 1.8 m 200+ m Any surfaces, Automatic focus Controller OFV-5000 Low pass (5, 20,100 kHz), high pass (100Hz) RS-232 interface for computer control Velocity Decoder VD-06 Ranges: 1, 2, 10 and 50 mm/s/V Resolution 0.02 m/s under 1mm/s/V range (2mv/20V) 350 kHz bandwidth analog output 24 bit, 96 kHz max. digital output on S/P-DIF interface 4 C CVC Dynamic Pushbroom Stereo Mosaics L Sponsored by: AFRL PIs: Zhigang Zhu, George Wolberg - CUNY City College; Robert Haralick CUNY Graduate Center Concept: Dynamic Pushbroom Stereo

GPS Accumulated motion Adaptive baseline Z F Results: Content-Based 3D Mosaics By F Height H dy By Sy dy Depth map Moving target Moving target Sy Accurate Fixed disparity depth video Sx

Right Mosaic Left Mosaic Description of Research Video Registration, representation and 3D static/dynamic content extraction from video sequences of dynamic urban scenes taken from aerial or ground vehicles. Stereo mosaics Innovative Claim (1) Pushbroom stereo mosaics for 3D dynamic scenes (2) Natural stereo matching primitives for 3D urban scenes and moving targets (2) Contentbased 3D mosaic rep for accurate 3D and motion Objectives Expected Contribution to Research Area (1) Rapid panoramic stereo mosaic construction (2) Accurate 3D reconstruction and parametric rep. (3) Robust moving target extraction and estimation An efficient and more accurate representation for large scale scenes; A new natural matching approach for both urban structures and moving targets 5 C CVC L

Key Results Accurate 3D reconstruction and motion detection on mosaics from real world video Portion of the depth map of ground and buildings; moving target are outliers (small white areas) Detected moving targets moving vehicles are identified by a 2D search in matching (from blue to red) Full Depth Map of mosaics from 1000+ frames. Sharp depth boundaries are obtained for further target recognition and motion detection 6 C CVC Under-Vehicle Inspection L Joint Work with UMass Computer Vision Lab Car drives over a 1D array of cameras Video Stereo Mosaics revealing occlusions

7 C CVC L Gamma-Ray Cargo Inspection Images from SAlC Mobile Vehicle and Cargo Inspection System (VACIS) 3D View 3D measurements from 2 pushbroom gamma-ray images 8 C CVC Summary L Novel Sensors Color, thermal, vibration Novel Algorithms Stereo mosaicing & 3D/ motion detection Emerging 3D Applications Security, surveillance & safety in transportation 9

Recently Viewed Presentations

  • Advances in Surgical Site Infection Prevention To Receive

    Advances in Surgical Site Infection Prevention To Receive

    The prevalence of SSI also varies widely among different types of surgical procedures. In one study, 90-day hospital readmission rates for SSI were calculated for amputation procedures; biliary, liver, and pancreas procedures; breast procedures, and hernia procedures, as in the...
  • Lamar Rose - African Methodist Episcopal Church

    Lamar Rose - African Methodist Episcopal Church

    RULES COMMITTEE Foster L. Chisholm, Chair AFRICAN METHODIST EPISCOPAL CHURCH FIFTH DISTRICT LAY ORGANIZATION June 29-July 1, 2017 CONVENTION RULES The Rules Committee met via email and compiled the following Convention Rules for the 60th Annual Session of the Fifth...
  • ACF-213/214 Finance I / Finance II

    ACF-213/214 Finance I / Finance II

    Double entry system. Accounting equation. Recording Process. Trial Balance. Describe the basic books of accounts and their function for recording of transactions for the purpose of control, including the use of the journal for the adjustment of prime entries.
  • Mitosis & Cell Cycle Cell Division Why? When?

    Mitosis & Cell Cycle Cell Division Why? When?

    Cells must reproduce else they die. The "life of a cell" is termed the cell cycle. The cell cycle has distinct phases, which are called G 1, S, G 2, and M. . Cells that have temporarily or reversibly stopped...
  • Kindergarten Art Review - St. Johns County School District

    Kindergarten Art Review - St. Johns County School District

    Remember, contour line helps define. Form, or make it appear. three-dimensional. ... Printmaking. Monoprint = 1 print. Linoleum Carving. Sculpture can be created by additive or subtractive technique. ... Kindergarten Art Review
  • PowerPoint Sunusu

    PowerPoint Sunusu

    WHAT IS CRIME? Munice and Mclaughlin (2001) define crime as ' an act or omission punishable by law ', and . Wikström(2006) views crime as ' an act of breaking a moral rule defined in criminal law
  • Theory Versus Practice in AI Planning

    Theory Versus Practice in AI Planning

    Automated Planning: Theory and Practice Chapter 3 ... The answer depends on which representation scheme we use Classical, set-theoretic, state-variable location 1 location 2 s0 Outline Background on complexity analysis Restrictions (and a few generalizations) of classical planning Decidability ...
  • Student Learning in the Supercomputing Challenge 2005-06:

    Student Learning in the Supercomputing Challenge 2005-06:

    Student Learning in the Supercomputing Challenge: Results of Kickoff and Expo Surveys (2005-06) Alexis Kaminsky, Ph.D. July 26, 2006 Kickoff and Expo Respondent Demographics SCC Respondents by Gender x Race What's Best about SCC?