OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A

| OPTICS, IMAGE SCIENCE, AND VISION

  • Editor: Franco Gori
  • Vol. 29, Iss. 11 — Nov. 1, 2012
  • pp: 2451–2458
« Show journal navigation

Toward stereovisual monitoring of three-dimensional translations with submicrometer resolution

Patrick Sandoz, Imen Elhechmi, and Tijani Gharbi  »View Author Affiliations


JOSA A, Vol. 29, Issue 11, pp. 2451-2458 (2012)
http://dx.doi.org/10.1364/JOSAA.29.002451


View Full Text Article

Acrobat PDF (1558 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

We reported on stereovisual localization of a labeled target versus three-dimensional (3D) position and orientation with a resolution of a few micrometers [Opt. Express 18, 24152 (2010)]. A pseudo-periodic pattern (PPP) is fixed on the target, whose center is identified with subpixel accuracy in both stereo images. This subpixel position definition is fed into the geometrical model of the stereovision system and, thus, leads to subvoxel resolution in the 3D target positioning. This paper reports on improvements and specialization of the method for addressing the measurement of 3D translations: (a) The use of an encrypted PPP wider than the field of observation of the cameras has two beneficial effects. First, the allowed lateral target displacements are wider than the field of view of each camera, thus extending the workspace volume. Second, the 3D position is always derived from the same zone located at the center of the camera sensor chip. A simplified geometrical model is thus sufficient, and the effects of the lens distortions lead to a different kind of calibration issues. (b) By considering only translations, the pattern directions remain stationary in the recorded images. Two-dimensional Fourier transforms are then replaced by single dimension ones, thus reducing the computation time. (c) The choice of a higher magnification lens allows the achievement of submicrometer resolution in target position determination. This level of performance makes the method attractive in various automated applications requiring microstage position control and sensing. This approach may, for instance, fulfill the requirements for the coarse positioning of specimens in front of nanotechnology instruments that are equipped with their own high-accuracy but short-excursion-range translation stages.

© 2012 Optical Society of America

1. INTRODUCTION

This paper presents further capabilities of this approach for the highly accurate positioning of a labeled target. This work is specialized in the monitoring of 3D translations, as typically performed by microstages supporting specimens in various microtechnology and nanotechnology instruments. A practical application might be the straightforward localization of tiny areas of interest on specimens after their transfer from one instrument to another. Since only translations are tracked, the orientation of the pattern with respect to the pixel frames of the cameras is a priori known. This allows the design of a simplified PPP as well as faster processing procedures by replacing 2D Fourier transforms with one-dimensional (1D) ones. We also use a PPP whose dimensions are much larger than the field of observation of the cameras used in the stereovision system. This choice has different implications that improve the overall capabilities of the method:
  • The allowed displacement range and the image definition are no longer linked with each other through the imaging lens magnification. Large displacements are reconstructed from images of nonoverlapping areas of the PPP. The absolute distance separating nonoverlapping zones of the pattern is retrieved from the systematic positioning of the current view with respect to the whole PPP.
  • The PPP provides a size reference that can be used for the calibration of the displacements observed as well as of the magnification of the imaging system.
  • Because of the large size of the pattern, the current view of each camera is always covering some zone of the pattern. Any area of the recorded images can thus be used for position reconstruction. A good choice is then to consider only the central pixel of each camera and to determine its corresponding position on the PPP. A complete description of the geometrical model [3

    3. J. Y. Bouguet, “Camera calibration toolbox for MATLAB” (2008), http://www.vision.caltech.edu/bouguetj/calib_doc/.

    ] of the setup is no longer necessary, since we always work on the optical axis of the two cameras. The effects of image distortions are thus reduced, and calibration can be thought of in a different way.

For the validation of the method, we chose to work with a relatively high-magnification lens in order to demonstrate submicrometer resolutions that cannot usually be achieved by means of stereovision. However, a lower magnification could also be used to allow an extended range of displacement, especially along the Z direction.

2. PSEUDO-PERIODIC PATTERN FOR 3D TRANSLATION MEASUREMENT

Fig. 1. 3 bits position encryption sequence. (a) Principle and (b) 2D pattern obtained.

Figure 1(b) presents the pattern suited for 2D position encoding, as obtained by multiplying the stripe pattern by itself after a rotation of π/2. The resulting pattern presents π/2 ambiguities that are not problematic, since only translations are considered in our case. In the general case of target displacements versus the six degrees of freedom, the pattern designs discussed in [5

5. J. A. Galeano-Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomech. 4, 65–82 (2010). [CrossRef]

] would provide solutions for unambiguous measurements. In practice, we used the pattern concept of Fig. 1 with words of 6 bits. The elementary stripe period was chosen to be 150 μm. The final size of the pattern is thus 31mm×31mm; i.e., 207 lines and columns, while the smallest detail to be resolved by the imaging system is 75μm×75μm. The minimum area required for the reading of the 6 consecutive bits necessary for a proper position identification is thus 2.7mm×2.7mm, i.e., 18 lines and columns. Experimentally, the pattern is fixed on the translation stage in such a way that the pattern directions are parallel to those of the camera pixel frame.

This kind of pattern has also been used for the 2D positioning of a patterned plate translated in the plane [8

8. P. Sandoz and M. Jacquot, “Lensless vision system for in-plane positioning of a patterned plate with subpixel resolution,” J. Opt Soc. Am. A 28, 2494–2500 (2011). [CrossRef]

]. In this case, the vision system is based on digital holography, which is advantageous for compactness and wide depth of field. The present work can be seen as a 3D counterpart of this approach using a pseudorandomly encoded target.

3. 2D IMAGE PROCESSING

The procedure of image processing applied to each stereo image recorded is summarized in Fig. 2. Figure 2(a) presents the central band of a recorded image in the horizontal direction. The pattern features appear with a high contrast, whereas some noise due to pattern imperfections is present. The image distortions that can be observed will be dealt with later. Since the horizontal stripes of the pattern provide the same information, the signal summation along the columns allows a significant improvement of the SNR as represented in Fig. 2(b). This signal is representative of the zone of the pattern currently observed by the camera. The signal processing is based on the Morlet wavelet transform [9

9. R. Kronland-Martinet, J. Morlet, and A. Grossmann, “Analysis of sound patterns through wavelet transforms,” Int. J. Pattern Recogn. Artif. Intell. 1, 273–302 (1987). [CrossRef]

] of the intensity distribution of Fig. 2(b). Figures 2(c) and 2(d) present, respectively, the wrapped phase and the modulus of the component of the wavelet transform that corresponds to the spatial frequency of the stripes. The phase signal is representative of the fine position of the stripes with respect to the camera pixel frame, while the modulus minimums are representative of the missing stripes. In practice, the Morlet wavelet transform is only computed for the spatial frequency of interest, but with two different resolution trade-offs. A wide wavelet—allowing a high spectral resolution—is used for phase determination, whereas a narrow one—leading to high spatial resolution—is used for modulus computation, and thus for an easy discrimination of the missing stripes. Figure 2(e) presents an intermediate step used to convert data relative to the pixel indices into data relative to the stripe indices and vice versa. The binary signal of Fig. 2(e) actually results from the thresholding of the phase of Fig. 2(c) following the condition |ϕ|<π/2. This condition is fulfilled for pixels that are closer to a maximum than to a minimum of the sinelike signal. Figure 2(e) can thus be used as a clock signal of the stripes suited for converting pixel indices into stripe indices and vice versa. Thanks to this conversion table, the modulus variations of Fig. 2(d) can be plotted as a function of the stripe index, as represented in Fig. 2(f). At this stage, the absent stripes are easily detected by thresholding. The definition of the threshold value with respect to the average wavelet transform modulus observed on a rolling window of five stripes makes this thresholding step very robust. The resulting positions of the missing stripes indicate which stripe is representative of a bit value within every set of three stripes. The pseudorandom sequence under observation is finally derived from Fig. 2(f). It provides a coarse determination of the zone of the pattern under view. This coarse measurement is adjusted with the phase data of Fig. 2(c) to obtain an absolute and high-resolution determination of the zone under view. The result is always expressed in phase as a function of the central pixel of the image frame as
Φc=2kcπ+ϕc,
(1)
where Φc is the absolute phase of the central pixel, kc is the stripe index at the central pixel, and ϕc is the wrapped phase at the central pixel. The latter is not directly taken in the data given by Fig. 2(c) but is derived from the least square fitting of the unwrapped phase for the benefit of data averaging and of SNR enhancement.

Fig. 2. Image processing steps: (a) horizontal band of the recorded image, (b) intensity profile obtained by summing along image columns, (c) spectral phase of (b), (d) modulus of (b), (e) binary signal locating the intensity extrema, (f) analog signal for the discrimination of absent and present stripes.

This sequence of image processing is applied four times in the same way, i.e., to the images of the left and right cameras and along the horizontal and vertical directions. At the end we obtain the absolute and high-accuracy pattern position versus the X and Y directions of each camera.

4. 3D TRANSLATION RECONSTRUCTION

Fig. 3. Effects of target translation on the measured position for each camera. A and B stand for the cameras; x and z stand for the unit vectors along the target displacement axes; θa and θb are the angles of the optical axes with respect to the z axis; a and b represent the vectors on which the target displacements are projected for cameras A and B, respectively; O and O represent the initial and final target positions, respectively; xa and xb are the pattern abscissa at its intersections with the optical axes of the two cameras; Δx and Δz represent the target displacement from O to O.
Fig. 4. Variations in the measured positions xa and xb for cameras A and B, respectively, while the target was translated along the (a) Z and (b) X directions by 340 μm.

The Y direction is not represented in Fig. 3. For a perfectly aligned setup, the Y direction would be independent of the X and Z directions, and both cameras should give the same Y data:
Δya=Δyb=Δy.
(6)
In practice, alignment is not perfect and lenses produce distortions, especially at high magnification, as in our case. Some coupling between the X, Y, and Z components results from these actual conditions. Figure 5 presents the variations observed along the Y direction for the target displacements corresponding to Fig. 4. Such coupling appears indeed, following two different aspects: linear and nonlinear. The linear contribution is due to the alignment mismatch between the (X,Y,Z) axes of the PZT, the PPP plane, and the camera optical axis. For instance, the pseudo-periodic plane appears to be rotated about 1.15° around the Z axis of the PZT in Fig. 5(b). The nonlinear contribution presents a periodic behavior with an amplitude of about 0.25 μm and a period corresponding to that of the PPP [(150/sinθA)μm in Fig. 5(a) and 150 μm in Fig. 5(b)]. This periodic modulation is due to the distortions introduced by the lenses and that appear slightly in Fig. 2(a). As we have mainly a black and white pattern image, a transition of half a period with respect to any given analysis window is analogous to a shift of the analysis window itself, since the black parts of the image does not contribute to the phase computation. This apparent shift of the region of interest that modulates the effects of the lens distortions is responsible for the periodic modulation appearing in Fig. 5. We simply notice that this nonlinear modulation tends to be maximum at the highest magnification as in our demonstration setup; it would be relaxed in a lower resolution configuration. Similar comments could be made from Fig. 6, which presents the variations of the computed phase while the PZT is translated along the Y direction, and in which appear both linear and nonlinear coupling also. Different strategies can be thought of for minimizing or compensating for the nonlinear modulation. They will be presented in a later work dedicated to an in-depth discussion of the calibration issues linked to this approach, which is outside this proof-of-principle paper.

Fig. 5. Variations in the measured positions ya and yb for cameras A and B, respectively, while the target was translated along the (a) Z and (b) X directions by 340 μm.
Fig. 6. Variations in the measured positions (a) ya and yb and (b) xa and xb for cameras A and B, respectively, while the target was translated along the Y direction by 240 μm.

5. METHOD DEMONSTRATION AND PERFORMANCE

Various experiments were carried out for the demonstration of the method and for the evaluation of the performance obtained. Figures 79 present the resulting data while an axial displacement of 10 μm was applied to the servo-controlled PZT along the X, Y, and Z directions, respectively. In each case, we present the reconstructed translation as well as the deviation from a straight line. We observe the ability of the method to detect and reconstruct target translations along the three directions. The standard deviations observed are 37, 33, and 45 nm for the X, Y, and Z directions, respectively. We may notice that these data are not affected by the systematic errors due to the misalignment and that the nonlinearities discussed previously have a low impact because of the short excursion of 10 μm relative to the pattern period of 150 μm. These deviations from a straight line are nevertheless representative of the ultimate capabilities of the method with respect to the SNR allowed by the devices used experimentally. By applying the usual statistical rule [10

10. R. J. Hansman, “Characteristics of instrumentation,” in The Measurement, Instrumentation, and Sensors Handbook, J. G. Webster, ed. (Springer-Verlag, 1999).

], a good evaluation of the method resolution corresponds to three times the standard deviation observed, i.e., better than 0.15 μm for the results of Figs. 79. This estimation is comparable to the theoretical limit of P/N of grid methods based on Fourier transforms [11

11. B. Zhao and A. Asundi, “Microscopic grid methods—resolution and sensitivity,” Opt. Laser Eng. 36, 437–450 (2001). [CrossRef]

], where P is the pattern period and N is the pixel number (1280×1024 in our case).

Fig. 7. Reconstructed target translation along X versus (a) the position given by the PZT capacitive sensor and (b) deviation from a straight line (standard deviation: 0.037 μm).
Fig. 8. Reconstructed target translation along Y versus (a) the position given by the PZT capacitive sensor and (b) deviation from a straight line (standard deviation: 0.045 μm).
Fig. 9. Reconstructed target translation along Z (a) versus the position given by the PZT capacitive sensor and (b) deviation from a straight line (standard deviation: 0.033 μm).

Figures 10 and 11 demonstrate the capabilities of the method to reconstruct 3D translations. For that purpose, a spiral-like displacement was applied to the PZT transducer supporting the PPP. We observe that the 3D motions are properly reconstructed even for an excursion larger than the period of the PPP. The mismatch observed between the reconstructed data (black) and the PZT positions returned by its capacitive sensor (red) are due to the setup misalignment discussed previously. The compensation for this mismatch can be obtained by applying a system coordinate transformation after setup calibration.

Fig. 10. Reconstruction of a 3D spiral-like translation applied to the PZT tranducer. Black: reconstructed position; red: position returned by the PZT capacitive sensor. (a) 3D view, (b) front view, (c) side view.
Fig. 11. Same as Fig. 10 with a displacement range larger than the period of the PPP.

Fig. 12. Reconstruction of an extended 3D translation (Δx=7mm, Δy=10mm, Δz=7mm).

6. CONCLUSION AND PROSPECTS

The stereovisual measurement of the 3D translations of a PPP is demonstrated in this paper with submicrometer resolution. This level of performance results from the high accuracy allowed by the phase processing of the PPP images. The approach based on an extended PPP has two main advantages: first, the measurement range is only limited by the actual size of the PPP and does not depend on the vision system magnification. The common trade-off “resolution versus measurement range” is thus partially released. Second, the 3D position is retrieved from two pairs of (X,Y) coordinates on the PPP. The latter correspond to the two particular PPP’s points that cross the optical axes of the left and right cameras respectively. The image processing is thus always based on the same areas of the recorded images, i.e., horizontal and vertical bands that avoid image corners where distortions are maximal. These image bands serve only to decrypt the sequence of pseudo-periodic code necessary for the identification of the intersection points with the optical axis. The effects of image distortions are thus significantly reduced in comparison with the usual case of 3D scene reconstruction from stereo images. Furthermore, these effects are kept constant during in-plane displacements and are only slightly affected by out-of-plane displacements of the target. Thanks to this property, the complete geometrical model of the stereovision configuration is not required. While the demonstration results of this proof of principle were obtained by means of an experimental evaluation of the camera orientations, further work has still to be carried out for the complete discussion of the calibration issues linked to this measurement approach.

Since in this approach the image processing is mainly based on 1D signals [cf. Fig. 2(b)], the stereovisual recording could be based on three linear cameras instead of on two 2D ones. In such a configuration, two linear cameras would correspond to the Xa and Xb directions considered in this paper, while the third one would correspond to the Y direction. Then the amount of data to be transferred to the processing unit would be drastically reduced and the allowed image rate increased (up to 60 kHz with commercial devices). Provided that the image processing is fast enough (for instance, by using dedicated hardware such as digital signal processing or field-programmable gate array devices), a 3D positioning bandwidth of several tens of kilohertz could be expected. Another prospect would be to combine this stereovisual technique with the digital holography approach described elsewhere [8

8. P. Sandoz and M. Jacquot, “Lensless vision system for in-plane positioning of a patterned plate with subpixel resolution,” J. Opt Soc. Am. A 28, 2494–2500 (2011). [CrossRef]

], with the advantage of having the almost unlimited depth of focus (several centimeters) allowed by digital holography while maintaining a submicrometer resolution.

REFERENCES

1.

H. S. Cho, Optomechatronics: Fusion of Optical and Mechatronic Engineering (CRC Press, 2006).

2.

N. A. Arias H., P. Sandoz, J. E. Meneses, M. A. Suarez, and T. Gharbi, “3D localization of a labeled target by means of a stereo vision configuration with subvoxel resolution,” Opt. Express 18, 24152–24162 (2010). [CrossRef]

3.

J. Y. Bouguet, “Camera calibration toolbox for MATLAB” (2008), http://www.vision.caltech.edu/bouguetj/calib_doc/.

4.

P. Sandoz, R. Zeggari, L. Froelhy, J. L. Prétet, and C. Mougin, “Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns,” J. Microsc. 225, 293–303 (2007). [CrossRef]

5.

J. A. Galeano-Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomech. 4, 65–82 (2010). [CrossRef]

6.

Z. Galeano, A. July, P. Sandoz, E. Gaiffe, S. Launay, L. Robert, M. Jacquot, F. Hirchaud, J. L. Prétet, and C. Mougin, “Position-referenced microscopy for live cell culture monitoring,” Biomed. Opt. Express 2, 1307–1318 (2011). [CrossRef]

7.

S. W. Golomb, Shift Register Sequences (Holden-Day, 1967).

8.

P. Sandoz and M. Jacquot, “Lensless vision system for in-plane positioning of a patterned plate with subpixel resolution,” J. Opt Soc. Am. A 28, 2494–2500 (2011). [CrossRef]

9.

R. Kronland-Martinet, J. Morlet, and A. Grossmann, “Analysis of sound patterns through wavelet transforms,” Int. J. Pattern Recogn. Artif. Intell. 1, 273–302 (1987). [CrossRef]

10.

R. J. Hansman, “Characteristics of instrumentation,” in The Measurement, Instrumentation, and Sensors Handbook, J. G. Webster, ed. (Springer-Verlag, 1999).

11.

B. Zhao and A. Asundi, “Microscopic grid methods—resolution and sensitivity,” Opt. Laser Eng. 36, 437–450 (2001). [CrossRef]

OCIS Codes
(100.0100) Image processing : Image processing
(120.0120) Instrumentation, measurement, and metrology : Instrumentation, measurement, and metrology
(150.5670) Machine vision : Range finding
(150.6910) Machine vision : Three-dimensional sensing

ToC Category:
Image Processing

History
Original Manuscript: June 26, 2012
Manuscript Accepted: September 22, 2012
Published: October 22, 2012

Citation
Patrick Sandoz, Imen Elhechmi, and Tijani Gharbi, "Toward stereovisual monitoring of three-dimensional translations with submicrometer resolution," J. Opt. Soc. Am. A 29, 2451-2458 (2012)
http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-29-11-2451


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. H. S. Cho, Optomechatronics: Fusion of Optical and Mechatronic Engineering (CRC Press, 2006).
  2. N. A. Arias H., P. Sandoz, J. E. Meneses, M. A. Suarez, and T. Gharbi, “3D localization of a labeled target by means of a stereo vision configuration with subvoxel resolution,” Opt. Express 18, 24152–24162 (2010). [CrossRef]
  3. J. Y. Bouguet, “Camera calibration toolbox for MATLAB” (2008), http://www.vision.caltech.edu/bouguetj/calib_doc/ .
  4. P. Sandoz, R. Zeggari, L. Froelhy, J. L. Prétet, and C. Mougin, “Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns,” J. Microsc. 225, 293–303 (2007). [CrossRef]
  5. J. A. Galeano-Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomech. 4, 65–82 (2010). [CrossRef]
  6. Z. Galeano, A. July, P. Sandoz, E. Gaiffe, S. Launay, L. Robert, M. Jacquot, F. Hirchaud, J. L. Prétet, and C. Mougin, “Position-referenced microscopy for live cell culture monitoring,” Biomed. Opt. Express 2, 1307–1318 (2011). [CrossRef]
  7. S. W. Golomb, Shift Register Sequences (Holden-Day, 1967).
  8. P. Sandoz and M. Jacquot, “Lensless vision system for in-plane positioning of a patterned plate with subpixel resolution,” J. Opt Soc. Am. A 28, 2494–2500 (2011). [CrossRef]
  9. R. Kronland-Martinet, J. Morlet, and A. Grossmann, “Analysis of sound patterns through wavelet transforms,” Int. J. Pattern Recogn. Artif. Intell. 1, 273–302 (1987). [CrossRef]
  10. R. J. Hansman, “Characteristics of instrumentation,” in The Measurement, Instrumentation, and Sensors Handbook, J. G. Webster, ed. (Springer-Verlag, 1999).
  11. B. Zhao and A. Asundi, “Microscopic grid methods—resolution and sensitivity,” Opt. Laser Eng. 36, 437–450 (2001). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited