## 3D localization of a labeled target by means of a stereo vision configuration with subvoxel resolution |

Optics Express, Vol. 18, Issue 23, pp. 24152-24162 (2010)

http://dx.doi.org/10.1364/OE.18.024152

Acrobat PDF (1471 KB)

### Abstract

We present a method for the visual measurement of the 3D position and orientation of a moving target. Three dimensional sensing is based on stereo vision while high resolution results from a pseudo-periodic pattern (PPP) fixed onto the target. The PPP is suited for optimizing image processing that is based on phase computations. We describe experimental setup, image processing and system calibration. Resolutions reported are in the micrometer range for target position (*x, y, z*) and of 5.3 × 10^{−4}*rad*. for target orientation (θ* _{x}*, θ

*,*

_{y}*θ*). These performances have to be appreciated with respect to the vision system used. The latter makes that every image pixel corresponds to an actual distance of 0.3 × 0.3

_{z}*mm*

^{2}on the target while the PPP is made of elementary dots of 1

*mm*with a period of 2

*mm*. Target tilts as large as

*π/*4 are allowed with respect to the

*Z*axis of the system.

© 2010 Optical Society of America

## 1. Introduction

1. T. Kanade and C. Zitnick, “A cooperative algorithm for stereo matching and occlusion detection,” IEEE Trans. Pattern Anal. Mach. Intell. **22**(7), 675–684 (2000). [CrossRef]

2. D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Comput. Vis. **47**(1), 7–42 (2002). [CrossRef]

3. A. Saxena, S. H. Chung, and A. Y. Ng, “3-D depth reconstruction from a single still image,” Int. J. Comput. Vis. **76**(1), 53–69 (2008). [CrossRef]

6. P. Sandoz, J. C. Ravassard, S. Dembélé, and A. Janex, “Phase-sensitive vision technique for high accuracy position measurement of moving targets,” IEEE Transactions on Instrumentation and Measurement **49**(44), 867–872 (2000). [CrossRef]

^{−4}

*rad*. angular resolution.

## 2. Measurement Principle

### 2.1. Basics of phase computations for in-plane position measurement

6. P. Sandoz, J. C. Ravassard, S. Dembélé, and A. Janex, “Phase-sensitive vision technique for high accuracy position measurement of moving targets,” IEEE Transactions on Instrumentation and Measurement **49**(44), 867–872 (2000). [CrossRef]

*u*

_{1},

*v*

_{1}) and (

*u*

_{2},

*v*

_{2}) respectively. After inverse Fourier transform, we obtain the magnitude image of Fig. 1.c and the wrapped phase map of Fig. 1.d; both corresponding to the lobe (

*u*

_{1},

*v*

_{1}). The magnitude image determines clearly the coarse position of the pattern while the wrapped phase encodes its fine position versus the image pixel frame. Since the dot pattern is known to be regular, the phase map can be unwrapped (cf Fig. 1.e) and fitted by a least square plane. The same filtering process is applied to the second spectral lobe (

*u*

_{2},

*v*

_{2}) and thus a second phase plane, perpendicular to the first one is obtained as represented in Fig. 1.f. Since the PPP is known to be made of

*N*periods of dots, the unwrapped phase planes correspond to phase excursions of (−

*Nπ*, +

*Nπ*) in both directions, with a phase equal to zero for the central dot. The unwrapped phase plane equations are thus adjusted with the appropriate 2

*kπ*constants and the PPP center is obtained as the (

*x,y*) position for which both phase plane equations are equal to zero. This analytic determination of the center position provides subpixel resolution thanks to least square fitting and to noise rejection by spectral filtering. The complete mathematical description of the technique can be found elsewhere [9

9. P. Sandoz, J. M. Friedt, and E. Carry, “In-plane rigid-body vibration mode characterization with a nanometer resolution by stroboscopic imaging of a microstructured pattern,” Rev. Sci. Instrum. **78**, 023706 (2007). [CrossRef] [PubMed]

^{−3}pixel using a standard CCD camera (8bits) while in-plane orientation is also derived from phase map equations with a resolution better than 10

^{−3}rad. In the case of Fig. 1.a however, the pattern symmetry induces a

*π/*2 ambiguity in in-plane orientation; this point will be dealt with later. Next this subpixel method is applied to 3D measurements by using of a stereo vision configuration.

### 2.2. Stereo vision configuration and 3D work space calibration

*μ*eye

*UI*−2210

*SE*-

*M*, 640× 480 pixels) equipped with identical 12

*mm*focal length lenses (

*Computar*1 : 1.2,1/2”). The target is placed at about 50

*cm*from the camera plane and is made of the PPP stuck on a rotation stage (

*Thorlabs CR*1 –

*Z*7) that is placed on an

*XYZ*combination of three linear displacement motors (

*Physik Instrumente M*403.1

*DG*).

10. J. Y. Bouguet, Camera calibration toolbox for matlab (2008). *http://www.vision.caltech.edu/bouguetj/calibdoc.*

*XYZ*) position of the pattern center from its set of subpixel positions in the left and right images.

## 3. Experiments and Results

### 3.1. Angular measurements

*, Θ*

_{X}*, Θ*

_{Y}*) to complement the (X,Y,Z) coordinates of the target centre. Angles are retrieved through the determination the 3D position of four complementary points on the pattern as illustrated in Fig. 3.b. Figure 3 presents images of the pattern as recorded simultaneously by the left and right cameras respectively. We first notice that one white dot is missing on the right-bottom diagonal of the pattern. This missing point was removed deliberately in order to break the pattern symmetry and thus to avoid*

_{Z}*π/*2 ambiguities in in-plane orientation. In Fig. 3.b, the dots symmetrical to the missing one with respect to the pattern corners are marked with red arrows. This set of four points; i.e. the missing one and its three counterparts, corresponds to unwrapped phase values equal to: (–10

*π*, −10

*π*), (−10

*π*, 10

*π*), (10

*π*, −10

*π*) and (10

*π*, 10

*π*). Subpixel coordinates corresponding to these phase values are derived from phase plane least square equations in both images. These coordinates are then fed into the geometrical model issued from the system calibration to obtain the 3D position of these four additional points. The target orientation (Θ

*, Θ*

_{X}*, Θ*

_{Y}*) is thus derived from their spatial distribution with respect to the target center position. However at this stage, Θ*

_{Z}*is obtained with a*

_{Z}*π/*2 ambiguity. The latter is easily removed by identifying the position of the missing dot. For that purpose, the gray level intensity is checked in the recorded images for the four complementary points marked in Fig. 3.b. The missing dot is then simply determined by the position with the lowest intensity observed. In this way, the target position is finally reconstructed unambiguously.

*π*/4 are allowed around axis

*X*and

*Y*while the full 2

*π*range is resolved around

*Z*. The actual PPP visibility can be appreciated in the videos of Fig. 5. The latter presents images recorded by the left and right cameras during the target rotation as well as the reconstructed positions. The proper target position reconstruction is achieved while the recorded shape of the PPP is altered as a function of the tilt angle. This capability is verified even in the case of the rotation around the

*X*axis that produces increased and dissymmetrical pattern distortions.

### 3.2. Method repeatability

*μm*from each other along the

*Z*direction, for a total excursion length of 1

*mm*. Reconstructed position and orientation dispersions were evaluated at each target position and statistical data presented in Tab.2 were obtained. In the table,

*Worst*and

*Best*lines are based on a single set of 100 measures while

*Mean*lines result from 5100 measures. In the first three columns, we can see that the

*best*and

*worst*position values are in good agreement with the 1

*mm*displacement applied to the target. In statistical position parameters, a factor of about four is observed between worst and best cases. One reason of this is the sensitivity of the setup to external disturbances. However, we are unable to distinguish between contributions of actual method capabilities and that of environmental disturbances. We then consider the average over those 51 measurements to be an unperfect but representative estimation of the method repeatability. The latter is evaluated to be 0.53, 0.52 and 2.06

*μm*along the

*X*,

*Y*and

*Z*axis respectively, as given by the standard deviations observed. The resulting

*standard deviation volume*is thus as small as 0.57

*μm*

^{3}. The dispersion of the reconstructed positions is represented in Fig. 6 for two cases. Fig. 6.a corresponds to a position whose statistics are close to the average ones. In this figure, the full scales are 2.5

*μm*in both

*X*and

*Y*and 10

*μm*in

*Z*. They have to be compared with the optical resolution of the vision system equal to 60

*μm*in the object plane. Fig. 6.b corresponds to the worst case observed among the 51 sets of data. It appears clearly that a few points move away from the main cloud of points. The dotted lines between points show that these points result from images recorded consecutively and this continuous perturbation can thus be attributed to environmental disturbances occurring during data recording. In spite of this external noise, the full scales are only 7

*μm*in both

*X*and

*Y*and 30

*μm*in

*Z*and the standard deviation on (

*x*,

*y*) coordinates corresponds to only 2 · 10

^{−2}image pixel. These results demonstrate that stereo-vision benefits from the subpixel capabilities of phase measurements for achieving subvoxel resolution in 3D target position reconstruction.

^{−4}

*rad.*over the 51 target positions and for the three rotation axis. Peak-valley fluctuations are twice lower than for position while standard deviation remains very stable over the 51 data sets and lower than 2 · 10

^{−4}

*rad.*Orientation seems thus to be less sensitive than position to external disturbances while providing also an excellent level of performances.

### 3.3. Displacement reconstruction and method specifications

#### Measurement volume

*mm*period. The PPP extension is thus 29

*mm*in both directions and the observation field is evaluated to be 161 × 116

*mm*

^{2}. To perform 3D position measurements, the PPP has to remain present in the recorded images. The allowed

*XY*target displacements are thus reduced to 128 × 83

*mm*

^{2}; i.e. 106

*cm*

^{2}(with a 2

*mm*margin kept around the image). The depth of focus was measured to be larger than 200

*mm*. Furthermore the subpixel method has been found to accommodate with defocus and allows measurements over extended depths [11]. The measurement volume is thus of the order of 10

^{3}

*cm*

^{3}or even larger.

#### Method resolution

^{−4}

*rad.*Position resolution is 1.6

*μm*in

*X*and

*Y*directions and 6.2

*μm*in

*Z*. The vision system magnification makes that a single image pixel corresponds to an actual size of about 300

*μm*on the target. The subvoxel capabilities of the method are thus clearly demonstrated. The lower performance along the

*Z*direction is due to the geometrical configuration. The proportion between lateral and vertical resolutions depends on the actual angle formed by the optical axis of the two cameras. These method capabilities are valid for the configuration used for experiments. In fact, resolution adjustments can be made through actual system parameters, especially camera signal to noise ratio and optical magnification. In the latter case, the measurement volume is also affected.

*σ*values.

## 4. Conclusions

*mm*focal length lens, we make this micrometer range of resolution compatible with an extended resolved space, about 10

*cm*along each spatial direction. In the present configuration, the PPP is assumed to be present in each recorded image. This restriction can be avoided by using a position encryption technique in the pattern design for accommodating with any field of observation as it was demonstrated in other works [13

13. P. Sandoz, R. Zeggari, L. Froelhy, J. L. Prétet, and C. Mougin, “Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns,” J. Microsc. **255**(3), 293–303 (2007). [CrossRef]

14. J. A. Galeano-Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomechatronics **4**(1), 65–82 (2010). [CrossRef]

15. J. Batlle, J. Marti, P. Ridao, and J. Amat, “A new FPGA/DSP-based parallel architecture for real-time image processing,” Real-Time Imaging **8**(5), 345–356 (2002). [CrossRef]

## Acknowledgments

## References and links

1. | T. Kanade and C. Zitnick, “A cooperative algorithm for stereo matching and occlusion detection,” IEEE Trans. Pattern Anal. Mach. Intell. |

2. | D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Comput. Vis. |

3. | A. Saxena, S. H. Chung, and A. Y. Ng, “3-D depth reconstruction from a single still image,” Int. J. Comput. Vis. |

4. | W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan, “Image-based visual hulls,” in |

5. | C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” |

6. | P. Sandoz, J. C. Ravassard, S. Dembélé, and A. Janex, “Phase-sensitive vision technique for high accuracy position measurement of moving targets,” IEEE Transactions on Instrumentation and Measurement |

7. | P. Sandoz, V. Bonnans, and T. Gharbi, “High-accuracy position and orientation measurement of extended two-dimensional surfaces by a phase-sensitive vision method,” Appl. Opt. |

8. | P. Sandoz, B. Trolard, D. Marsaut, and T. Gharbi, “Microstructured surface element for high-accuracy position measurement by vision and phase measurement,” Ibero-American Conf. RIAO-OPTILAS, Proc. SPIE |

9. | P. Sandoz, J. M. Friedt, and E. Carry, “In-plane rigid-body vibration mode characterization with a nanometer resolution by stroboscopic imaging of a microstructured pattern,” Rev. Sci. Instrum. |

10. | J. Y. Bouguet, Camera calibration toolbox for matlab (2008). |

11. | J. A. Galeano-Zea, P. Sandoz, and L. Robert, “Position encryption of extended surfaces for subpixel localization of small-sized fields of observation,” in |

12. | R. J. Hansman, “Characteristics of instrumentation,” in |

13. | P. Sandoz, R. Zeggari, L. Froelhy, J. L. Prétet, and C. Mougin, “Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns,” J. Microsc. |

14. | J. A. Galeano-Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomechatronics |

15. | J. Batlle, J. Marti, P. Ridao, and J. Amat, “A new FPGA/DSP-based parallel architecture for real-time image processing,” Real-Time Imaging |

**OCIS Codes**

(120.4640) Instrumentation, measurement, and metrology : Optical instruments

(150.6910) Machine vision : Three-dimensional sensing

**ToC Category:**

Instrumentation, Measurement, and Metrology

**History**

Original Manuscript: September 7, 2010

Revised Manuscript: October 8, 2010

Manuscript Accepted: October 9, 2010

Published: November 3, 2010

**Virtual Issues**

Vol. 6, Iss. 1 *Virtual Journal for Biomedical Optics*

**Citation**

Néstor A. Arias H., Patrick Sandoz, Jaime E. Meneses, Miguel A. Suarez, and Tijani Gharbi, "3D localization of a labeled target by
means of a stereo vision configuration
with subvoxel resolution," Opt. Express **18**, 24152-24162 (2010)

http://www.opticsinfobase.org/vjbo/abstract.cfm?URI=oe-18-23-24152

Sort: Year | Journal | Reset

### References

- T. Kanade, and C. Zitnick, “A cooperative algorithm for stereo matching and occlusion detection,” IEEE Trans. Pattern Anal. Mach. Intell. 22(7), 675–684 (2000). [CrossRef]
- D. Scharstein, and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Comput. Vis. 47(1), 7–42 (2002). [CrossRef]
- A. Saxena, S. H. Chung, and A. Y. Ng, “3-D depth reconstruction from a single still image,” Int. J. Comput. Vis. 76(1), 53–69 (2008). [CrossRef]
- W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan, “Image-based visual hulls,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press/Addison-Wesley Publishing Co., 2000), pp. 369–374.
- C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation,” ACM SIGGRAPH 2004 Papers, (ACM, 2004), pp. 600–608.
- P. Sandoz, J. C. Ravassard, S. Dembélé, and A. Janex, “Phase-sensitive vision technique for high accuracy position measurement of moving targets,” IEEE Trans. Instrum. Meas. 49(44), 867–872 (2000). [CrossRef]
- P. Sandoz, V. Bonnans, and T. Gharbi, “High-accuracy position and orientation measurement of extended twodimensional surfaces by a phase-sensitive vision method,” Appl. Opt. 41(26), 5503–5511 (2002). [CrossRef] [PubMed]
- P. Sandoz, B. Trolard, D. Marsaut, and T. Gharbi, ““Microstructured surface element for high-accuracy position measurement by vision and phase measurement,” Ibero-American Conf. RIAO-OPTILAS,” Proc. SPIE 5622, 606–611 (2004).
- P. Sandoz, J. M. Friedt, and E. Carry, “In-plane rigid-body vibration mode characterization with a nanometer resolution by stroboscopic imaging of a microstructured pattern,” Rev. Sci. Instrum. 78, 023706 (2007). [CrossRef] [PubMed]
- J. Y. Bouguet, Camera calibration toolbox for matlab (2008). http://www.vision.caltech.edu/bouguetj/calib_doc/
- J. A. G. Zea, P. Sandoz, and L. Robert, “Position encryption of extended surfaces for subpixel localization of small-sized fields of observation,” in Proc. IEEE on International Symposium on Optomechatronic Technologies, (IEEE, 2009), pp. 21–27.
- R. J. Hansman, “Characteristics of instrumentation,” in The Measurement, Instrumentation, and Sensors Handbook, J. G. Webster, ed. (Springer-Verlag, 1999).
- P. Sandoz, R. Zeggari, L. Froelhy, J. L. Prétet, and C. Mougin, “Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns,” J. Microsc. 255(3), 293–303 (2007). [CrossRef]
- J. A. G. Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomechatronics 4(1), 65–82 (2010). [CrossRef]
- J. Batlle, J. Marti, P. Ridao, and J. Amat, “A new FPGA/DSP-based parallel architecture for real-time image processing,” Real-Time Imaging 8(5), 345–356 (2002). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Supplementary Material

» Media 1: AVI (2735 KB)

» Media 2: AVI (2288 KB)

» Media 3: AVI (2718 KB)

» Media 4: AVI (1472 KB)

« Previous Article | Next Article »

OSA is a member of CrossRef.