Photogrammetry Laboratory - Real time Stereo

Real time high resolution stereophotogrammetry

Project team members

John Morris
Georgy Gimel'farb
Patrice Delmas
   Students
Khurram Jawed
Tariq Khan
Waqar Khan
Ratheesh Kalarot
...

Real-time Stereo Project

Using two cameras directly connected to an FPGA implementing Gimelfarb's Symmetric Dynamic Programming Stereo (SDPS) correspondence algorithm, we are able to obtain high resolution disparity maps (1024x768) in real time (30 fps). These short videos were captured in our laboratory using two Sentech CameraLink cameras (1024x768 pixels) directly connected to an Altera EP3S2 150F1152C3 FPGA on a Gidel ProcStar III card on an 8-lane PCIexpress bus.
TitleDisparity mapOcclusion mapNotes
Traintrain_disp.wmv (3.0 Mb)train_occ.wmv (4.2 Mb) 'Thomas' is running on an oval track: you see him change colour as he moves from the rear of the track towards the camera.
Khurramkhurram_disp.wmv (2.8 Mb)khurram_occ.wmv (5.7 Mb) Note that the occlusion map shows Khurram's outline clearly even when he is in the background!
Crowd
(the largest crowd that we can fit in our lab)		dphw_twq.wmv (12.2 Mb)Appended to disparity map Here you see a pair of models moving in the scene. The occlusion map video for the same 'act' follows the disparity map.
Ball
A little basketball practice in the lab - luckily the student out of sight can catch!		 DPHW_ball.wmv (4.2 Mb) Definitely real-time! We can capture the ball in flight! Next time we'll try a serious game like cricket! In this video, the rectified left and right images, the disparity map and the occlusion map play simultaneously.
Contour Maps
Ball (again!) ContourMap.wmv (3.6 Mb) Again, we capture the ball in flight! Main panel - foreground contours. Background contours have been removed for clarity.
Left panels - original left and right images; disparity map and occlusion map.
As the ball comes closer, it separates from the thrower (the depth resolution is lower in the distance - a natural consuquence of stereo geometry) and becomes clearly outlined. There is a small amount of motion blur and de-focussing as the ball reaches the catcher.
Credits
SDPS algorithmGeorgy Gimel'farb
Hardware implementation of SDPSJohn Morris
Real time rectification and overall systemKhurram Jawed
Calibration, contour mapsTariq Khan

Publications
ReferenceDownloadBibTex
Real time stereo
John Morris, Georgy Gimel’farb,
Real time Stereo Image Matching System, NZ Patent Application 567986, May 2, 2008 		   
Real time stereo
Khurram Jawed, John Morris, Tariq Khan, Georgy Gimel’farb,
Real Time Rectiifcation for Stereo Correspondence 		EUC'2009, Vancouver
Concurrent Stereo Matching
John Morris, Georgy Gimel’farb, Jiang Liu, Patrice Delmas,
Concurrent Stereo Matching: An Image Noise-Driven Model, Lecture Notes in Computer Science, Volume 3757, Oct 2005, Pages 46 - 59 		LNCS bibtex
Patrice Delmas, Georgy L. Gimel'farb, Jiang Liu and John Morris, A Noise-Driven Paradigm for Solving the Stereo Correspondence Problem, Proc MICAI 2005: Advances in Artificial Intelligence, 4th Mexican International Conference on Artificial Intelligence, Monterrey, Mexico, November 14-18, 2005, pages 307-317 (LNCS 3789)   bibtex
Georgy Gimel'farb, Jiang Liu, John Morris and Patrice Delmas, Concurrent Stereo under Photometric Image Distortions, ICPR, Hong Kong, Aug 2006 ps (4.5Mb)bibtex
Jiang Liu, Noise-driven Concurrent Stereo Matching, PhD thesis, The University of Auckland, 2006 Web site  
Working Papers None currently
Resources Bibliographies
 Stereo papers - Main
 Stereo papers - Aux D