Effects of Surface Characteristics on Alignment of Real

Effects of Surface Characteristics on Alignment of Real

Effects of Surface Characteristics on Alignment of Real and Graphic Objects in Stereoscopic Augmented Reality Environments Ming Hou Ergonomics in Teleoperation and Control (ETC) Laboratory Department of Mechanical and Industrial Engineering University of Toronto January 6th, 2003 3D Measurement in Unmodelled World Virtual Tape Measure Problem: Unknown relationship between real and virtual objects in Augmented

Reality displays Research on Real World Targets Virtual Pointer LINE Video Image Cylinder AREA VOLUME Hemisphere Pseudo-Transparency Phenomenon Real Object (in Video) Virtual pointer behind real surface Virtual pointer in front of surface

Transparency Effect Breakdown of Fusion + Fusion Conflict Conflict between Binocular Disparity and Occlusion Cues Virtual pointer behind real surface Virtual pointer in front of surface Fuse Real Surface Fuse Virtual Pointer Theory of Surface Interaction: Fusion Breakdown depends on Texture Density Virtual pointer behind real surface Virtual pointer in front of surface Low Texture Density High Texture Density

Research Motivation Is this conflict really significant? If yes, can it be used as an extra cue for detecting interactions between real and virtual objects, and thus locating the real objects in AR environment more easily and accurately? Main Hypotheses #1 More accurate to indicate position on (curved) surface with high texture density (HTD) than with low texture density (LTD) HTD better than LTD

Main Hypotheses #2 Orientation of observer relative to target surface will have influence Top View Top View Real Surface Stereo Camera Real Surface Stereo Camera Virtual Pointer Virtual Pointer Centre different from Off-centre Main Hypotheses #3 Form of Virtual Pointer (VP) will have impact on alignment

performance LINE AREA VOLUME Main Hypotheses #4 Binocular disparity (i.e. crossed vs 0 vs uncrossed) will affect alignment performance C ND UC No disparity (ND) > Crossed (C) or Uncrossed (UC) Experimental Investigation of AR Surface Effects

Stereo Camera Stereoscopic AR Display Virtual Pointer Spaceball Cylinder Indigo 2 Barrier Methodology Factorial Expt. # Design Independent Variables Dependent Variables

(Measurement) #1 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Orientation Placement Error between perceived target and its actual position on real surface #2 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Form

Placement Error + Confidence Rating + Preference Rating #3 5x5 Texture Density Surface Orientation Placement Error + Angular Error between estimated normal and real surface normal Subjective Comparisons Ease of Use Transparency Ease of Fusion 6 Images

(15 paired comparisons) Texture Density High Low Virtual Pointer Form LINE AREA VOLUME Methodology Factorial Expt. # Design Independent

Variables Dependent Variables (Measurement) Expt. #1 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Orientation Placement Error between perceived target and its actual position on real surface Expt. #2 Texture Density

Surface Orientation 2x2x3x3 Binocular Disparity VP Form Placement Error + Confidence Rating + Preference Rating Expt. #3 5x5 Texture Density Surface Orientation Placement Error + Angular Error between estimated normal and real surface normal Placement Error vs Texture Density - 1

M ean Placem en t Erro r (cm ) VP Placem ent Error vs Texture Density (Error Bar = 95% CI, F(1,11) = 11.14, P = 0.007) Farther 1.6 1.2 0.8 0.4 0 -0.4 \ Closer \ Cylinder Surface High

\ \ Low Texture Density Main Experimental Results - 1 M e a n P la c e m e n t E r r o r ( c m ) Placement Error vs Texture Density - 2 Placement Error along Surface Normal vs Texture Density (Error Bar = 95% CI, F(4,40) = 41.34, P < 0.001) 1.6 1.4 1.2 1 0.8 0.6

0.4 0.2 0 0 10 20 30 40 Texture Density (%) Main Experimental Results - 2 Placement Error vs Surface Orientation - 1 M e a n P la c e m en t E rro r (c m ) VP Placem e nt Error vs Targe t Pos ition (Error Bar = 95% CI, F(1,11) = 98.19, P < 0.001)

Farther 1.6 1.2 0.8 0.4 0 -0.4 Closer Ce ntre Right Targe t Pos ition (relative to obs e rve r) Main Experimental Results - 3 M e a n P la c e m e n t E rro r ( c m ) Placement Error vs Surface Orientation - 2

12 Place m e nt Error along Surface Norm al vs Targe t Pos ition (Error Bar = 95% CI, F(4,40) = 11.90, P < 0.001) 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 3 4 5

1 2 3 4 5 Targe t Pos ition on the Sphe re Main Experimental Results - 4 Angular Error vs Surface Orientation M e a n A lt it u d e E rro r ( d e g re e ) Altitude Error vs Targe t Pos ition on He m is phe re (Error Bar = 95% CI, F(4, 40) = 36.72, P < 0.001)

20 15 10 5 0 1 1 2 3 4 5 2 3 4 5 Targe t Pos ition

Main Experimental Results - 5 Subjective Comparison Results w.r.t. Ease of Use and Ease of Fusion Volume VP better than other VPs, regardless of texture density w.r.t. Transparency Volume VP + Highly textured least transparent combination Main Experimental Results - 6 Summary of Results - 1 Texture Density High better than Low Surface Orientation Centre

different from Off-Centre Summary of Results - 2 VP Form VOLUME > AREA > LINE (subjective comparisons) Binocular Disparity C ND UC Conclusions - 1 Perceptual conflict does exist when real

and virtual objects interact in 3D AR environments (a model proposed) Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects Optimal density value for Random Dot texture pattern was found as Engineering Solution for accurate 3D measurement R-V Interaction Interaction Process Process Model Model R-V Perception Cognition / Decision Making (Stereo Matching) (Stereo Matching + Cue Conflict Resolution)

behind Fusion difficulty VP behind, or at, real surface? at Fused Image? No fusion difficulty at (Transparency) No Stereo Matching :

VP in front of real surface? VP behind, or at, real surface? behind Yes User Localisation Achieved Adjustment (Breakdown/Conflict) External World Display of virtual pointer (VP)

superimposed upon real object surface AR Display VP Controller Conclusions - 1 Perceptual conflict does exist when real and virtual objects interact in 3D AR environments (a model proposed) Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects Optimal density value for Random Dot texture pattern found as Engineering Solution for accurate 3D measurement Conclusions - 2 Target position does affect alignment task: centrally located targets benefit performance, but have disadvantage when along the line of sight

Volumetric stereo graphic cursor (more fusable features along three dimensions) is subjectively the most favoured VP Pseudo-transparency contributes literature of depth perception cues (shapefrom-texture and stereo) Implications for for AR AR Interface Interface Design Design Implications Random Dot Stereogram enhances 3D alignment performance Perceptual conflicts can be used as extra depth cue to detect real object position 3D VP better than other VPs Perceptual errors always exist Limitations Implementation

Display Mode : stationary display without motion parallax and motion perspective Binocular Disparity : confounded with size cue and resolution Scope VP Design: line thickness of wire-frame VP Texture Pattern: square Random Dot pattern Future Work/Impact Work/Impact -- 11 Future Near Term Research Projected lighting : more practical Motion parallax with Video-HMD Computational vision may alleviate some error (being investigated) Projected lighting with random dot

texture pattern Simulated Projector Stereo Cameras Future Work/Impact - 2 Long Term Interests Integration of Computer Assisted Object Detection in AR Displays See-through HMD AR for Dismounted Soldiers in the Battlefield (e.g., Perceptual Conflicts, Navigational Aids, etc.) Other Human Computer Interaction (HCI) Topics (e.g., Integration of Electronic Information with

Human-Machine Systems, etc.) Acknowledgement Dr. Julius Grodski at Defence Research & Development Canada (DRDC) Toronto, Prof. Allison B. Sekuler and Prof. Paul Milgram at University of Toronto Dr. Stephen Ellis at NASA and Prof. Stanley Hamstra at University of Toronto Natural Sciences and Engineering Research Council (NSERC) Doctoral Scholarship Institute of Robotics and Intelligent Systems (IRIS), Canada Ontario Graduate Scholarship (OGS) Experimental #1 Main Result P la c e m e n t E r r o r (c m ) Farther 1 0

VP Placem e nt Error vs Surface Texture (Error Bar = 95% CI, F(1,9) = 619.71, P< 0.001) \ \ Cylinder Surface \ -1 -2 -3 -4 -5 -6 -7 -8 -9 Closer

High Low Texture Density \ VP Form and Orientation in 1st Experiment : Vertical Diagonal Observer Horizontal Interaction in Experiment # 1 VP Placement vs Texture and Target Position Placement Error (cm)

(Error Bar = +/- 1SD, F(1,9) = 246.33, P< 0.001) 1 0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -10 -11 High Low Centre Right Angular Displacement of Target Normal

Interaction between Surface Texture (High vs Low) and Target Position (Center vs Right) Paired Comparison Result Mean Z score 0 1.15 1.71 2.26 3.16 1 2 34

5 6 0 0.25 0.89 2.97 3.82 4.37 6 5 4 Ease of Use Image # Transparency Image #

Ease of Fusion Image # 2 0 1 1 3 0.69 0.73 2 3 2.08 2.14 5

4 2.26 6 Measurement of Placement Error along Surface Normal Estimated Target Surface Normal Real Surface Target Positive Error Positive error shows the estimated target is inside the sphere along surface normal M e a n P la c e m e n t E r r o r ( c m ) Placement Error vs Texture Density for Experiments 2 and 3

Placement Error along Surface Normal vs Texture Density (Error Bar = 95% CI, F(4,40) = 41.34, P < 0.001) 1.6 Experiment 2 Experiment 3 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20

30 Texture Density (%) 40 Definition of Angular Error Y Altitude Error : Angular distance between estimated normal TP and real surface normal TN N P ' Q

Azimuth Error : Angular Error between horizontal projection OR and OS ' T O S R ' '

Z X Angular Bias in Spherical Coordinate Y N E T Bias Area S Z X Angular bias tilted upwards from real surface normal (TN) Example of Distribution of Altitude and Azimuth

Altitude Error vs Azimuth Error at Target 5 60 Altitude Error (degree) 50 40 30 20 10 0 -180 -135 -90 -45 -10 0 45 90

135 180 -20 -30 Azimuth Error (degree) Angular bias between estimated and real surface normal Ground Truth Measurements in Real Scene Cylinder Stimulus Side View Stereo Cameras Top

View Iron plate Cylinder Calibration object Real World Origin (0,0,0) Real Distance in Depth (Z) Registration Verification: Measurement of a Pin and a Cube Stereo Cameras Calibration Target Calibration Cube

P e r c e n t a g e o f D is t a n c e ( D ) f o r 1 P ix e l C h a n g e in H o r iz o n t a l D is p a r it y VP Resolution for Experiment 2 VPVP Resolution and Tests Re s olution (Pe rceAccuracy ntage of Re al Dis tance vs One Pixe l Error of V P in Dis play) 8.00% 7.00% 6.00% 5.00% 4.00% 3.00% 2.00%

1.00% 0.00% 0 1 2 3 4 Dis tance of Obje ct from Cam e ras (m ) 5 Retinal Disparity in Stereoscopic Display for One Pixel Separation (exaggerated) ZD Stereoscopic Display Monitor

C b Apparent position of point C, due to 1 pixel horizontal disparity e 2e P ) a b 2arctg ( ) 2arctg ( ) d 2d P = 1 pixel separation XD Viewers Eye a 2e

b /2 d Psychophysical Standard for Texture Density Control Spatial scale (size) Homogeneity (spatial regularity, density is approximately constant over the surface) Isotropy (no orientation bias, equally to be oriented in all directions) compression Practical Augmented Reality Example Virtual Tape Measure for Minimally Invasive Surgery Distance between point 1 and point 2 is 7mm

Coordinate of Point 1 (2.3, 14.7, 96.2) Coordinate of Point 2 (1.8, 14.4, 95.8)

Recently Viewed Presentations

  • Dreaming with one eye open

    Dreaming with one eye open

    * These are rough averages of ranking in each category over last several years These are rough averages of ranking in each category over last several years These are rough averages of ranking in each category over last several years...
  • extend previous knowledge of sets and subsets using

    extend previous knowledge of sets and subsets using

    use an algebraic representation to explain the effect of a given positive rational scale factor applied to two-dimensional figures on a coordinate plane with the origin as the center of dilation.[8.3C] October 2014. Math 8th Grade
  • Developing an Awards Program for SW PA&#x27;s Businesses and ...

    Developing an Awards Program for SW PA's Businesses and ...

    The PNC Financial Services Group UPMC * Medium Business (50-999 employees) Del Monte Foods DMI Companies Eaton Corporation Massaro Corporation Michael Baker Corporation Oxford Development Company WESCO Distribution Inc. WESCO Distribution Inc. - Warrendale * Small Business (<50 employees) Backstage...
  • Pioneers of Financial Economics - SFU.ca

    Pioneers of Financial Economics - SFU.ca

    Epistemology of Hume's Attack on Christian Miracles. An integral part of a much larger philosophical project, Hume's attack is about the use of inductive empiricism to infer causes from effects, a problem that inspired Bayes(1763) and still generates intellectual debate....
  • PROJECT TITLE - California State University, Northridge

    PROJECT TITLE - California State University, Northridge

    Shahnam Mirzaei, Ph.D. ECE Department. CECS. May 18, 2018. Introduction & Guiding Questions. The MPT (Math Placement Test) is used to determine eligibility for enrollment in Math 102, 103, 105, 150A, and 255A.
  • SHAREHOLDERS RIGHTS OBTAINING RELIEF FROM THE ENGLISH COURTS

    SHAREHOLDERS RIGHTS OBTAINING RELIEF FROM THE ENGLISH COURTS

    Rupert D'Cruz (Barrister, Littleton Chambers) [email protected] 12 April 2011 - Ekaterinburg. SHAREHOLDERS' RIGHTS - OBTAINING RELIEF FROM THE ENGLISH COURTS. Categories of Shareholder Disputes. Breach of a shareholders' agreement and articles of association.
  • Title

    Title

    [8] Chun-Yang Chen and P. P. Vaidyanathan, "Precoded V-BLAST for ISI MIMO channels," IEEE International Symposium on Circuit and System Kos, Greece, May 2006, [9] Chun-Yang Chen and P. P. Vaidyanathan, "IIR Ultra-Wideband Pulse Shaper Design," Proc. 39th Asilomar Conference...
  • Soil Formation and Composition - Atlanta Public Schools

    Soil Formation and Composition - Atlanta Public Schools

    Soil Formation and Composition Chapter 6 Section 2 ... solid layer of rock beneath the soil, soil is always being formed whenever it is exposed Soil Composition and Texture Soil is a mixture of rock particles, minerals, decayed organic material,...