A generalized reference-plane-based calibration method in optical triangular profilometry

doi:10.1364/OE.17.020735

« Show journal navigation

A generalized reference-plane-based calibration method in optical triangular profilometry

Suochao Cui and Xiao Zhu »View Author Affiliations

Optics Express, Vol. 17, Issue 23, pp. 20735-20746 (2009)
http://dx.doi.org/10.1364/OE.17.020735

View Full Text Article

Acrobat PDF (252 KB)

Journal Search
Article Lookup

Article Tools

Citations

Alert me when this article is cited
Export Citation/Save

Article Navigation

▸ Article Outline
– Abstract
1. Introduction
2. Methods
3. Experiments
4. Conclusion
– References and links

Article Tools
Full Text
Article Info
References
Cited By
Figures
Metrics
Download PDF

Viewing Equations in the
HTML Article

Optimal Viewing Recommendations

Full Text
Article Info
References (19)
Cited By
Figures (5)
Metrics

Abstract

In this paper, a generalized reference-plane-based calibration method is proposed in optical triangular profilometry by exploring projection ray tracing method and image ray tracing method. The pin-hole camera model is used to model the camera and the projector, and parallel planes model is used to model the reference and test planes. The camera, projector, and planes can be in arbitrary positions and arbitrary directions. The reciprocal of the height and the reciprocal of the phase shift (or pixel position vertical distance) are in linear relationship. Experiments are conducted to verify the proposed method.

1. Introduction

Recently used calibration methods in optical triangular profilometry of diffuse surface can be classified into two categories based on whether a reference plane is used or not: the reference-plane approach and the no-reference-plane approach. One of the key features of the reference-plane approaches is the deduction of calibration equations from the geometry of the system. For example, Zhou et al. proposed a phase to height method based on the geometrics of the optical triangular profilometry in 1994 [1

W. S. Zhou and X. Y. Su, “A direct mapping algorithm for phase-measuring profilometry,” J. Mod. Opt. 41(1), 89–94 ( 1994). [CrossRef]

], where the heights (distances) between the test and reference planes are used in calibration procedure. They solve the problem of the co-plane for the camera axis and the projector axis and give the expression of the phase to height in two dimensions:

1 / h (x, y) = p (x, y) / a (x, y) + b (x, y)

, parameters

a (x, y), b (x, y)

are the coefficients at the object position

(x, y)

. In their system, the camera must be in normal view of the reference plane. Chen et al. used a least-squares approach to evaluate the carrier phases [2

L. Chen and C. Quan, “Fringe projection profilometry with nonparallel illumination: a least-squares approach,” Opt. Lett. 30(16), 2101–2103 ( 2005). [CrossRef] [PubMed]

, 3

L. Chen and C. J. Tay, “Carrier phase component removal: a generalized least-square approach,” J. Opt. Soc. Am. A 23(2), 435–443 ( 2006). [CrossRef]

], and Guo et al. used similar approach in their calibration methods [4

H. Guo, H. He, Y. Yu, and M. Chen, “Least-squares calibration method for fringe projection profilometry,” Opt. Eng. 44(3), 033603 ( 2005). [CrossRef]

, 5

H. Guo, M. Chen, and P. Zheng, “Least-squares fitting of carrier phase distribution by using a rational function in fringe projection profilometry,” Opt. Lett. 31(24), 3588–3590 ( 2006). [CrossRef] [PubMed]

], the cameras in their system setup are in normal view too, which is a rigorous condition difficult to satisfy in practice. Rajoub et al. proposed a method supporting the rotation of the camera [6

B. A. Rajoub, M. J. Lalor, D. R. Burton, and S. A. Karout, “A new model for measuring object shape using non-collimated fringe-pattern projections,” J. Opt. A, Pure Appl. Opt. 9(6), S66–S75 ( 2007). [CrossRef]

], but they did not solve the problems of the normal view of the camera, either. Wang et al. proposed a method in which the camera can be in any view of the reference plane [7

Z. Wang, H. Du, and H. Bi, “Out-of-plane shape determination in generalized fringe projection profilometry,” Opt. Express 14(25), 12122–12133 ( 2006). [CrossRef] [PubMed]

–9

Z. Wang, H. Du, S. Park, and H. Xie, “Three-dimensional shape measurement with a fast and accurate approach,” Appl. Opt. 48(6), 1052–1061 ( 2009). [CrossRef]

], but the expression of their conclusion is complicated and has lots of coefficients. All the above algorithms use the image ray tracing method, which cannot be used in the non-phase shift measuring profilometry. Anand et al. proposed the unified calibration techniques using the projection ray tracing method [10

A. Asundi and Z. Wensen, “Unified calibration technique and its applications in optical triangular profilometry,” Appl. Opt. 38(16), 3556–3561 ( 1999). [CrossRef] [PubMed]

], but the setup of their system is restricted to rotating work table systems. It requires the normal incident of the projection line stripes too. The restrictions of the above-mentioned systems mainly stem from the use of geometry in their deduction, which also makes the process inefficient and tedious.

Instead of deducing the calibration equations from the geometry of the system, the no-reference-plane approach sets up a world coordinates system in which all the points in the measuring volumes are represented by the three-dimensional (3D) coordinates. The projector and the camera are always modeled as pin-hole camera models [11

J. Heikkila, and O. Silven, “Calibration Procedure for short focal length off-the-shelf CCD cameras,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vienna, Austria, 1996), pp. 166–170.

–15

R. L. Saenz, T. Bothe, and W. P. Juptner, “Accurate procedure for the calibration of a structured light system,” Opt. Eng. 43, 467–471 ( 2004).

], and the intrinsic and extrinsic parameters can be calibrated by single device calibration techniques [11

–14

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 45(8), 083601 ( 2006). [CrossRef]

] or unite calibration techniques [15

R. L. Saenz, T. Bothe, and W. P. Juptner, “Accurate procedure for the calibration of a structured light system,” Opt. Eng. 43, 467–471 ( 2004).

]. The process of obtaining calibration equations is significantly simplified and generalized as the systems are described by the transitions of the matrices and the rotation and translation effects are considered.

In this work, we use the matrices to describe the measuring system under reference-plane approach as an attempt to simplify and generalize the process of deducing the calibration equations. The camera and projector are modeled as pin-hole camera model, and the reference (test) planes are modeled by parallel planes. For the generalization, the direction of the camera, projector and reference (test) planes is in arbitrary position and direction in the world coordinates system. When the projection point and image point positions are known, the world coordinates of the point can be obtained by solving the equations formed by the system parameters. So the relationship of the world coordinate position, projection position and image position can be acquired. By deducing from this relationship, for both the projection ray and image ray tracing systems, the linear relationship of the reciprocal of the height and the reciprocal of the phase shift (or pixel position vertical distance) can be acquired.

The rest of the paper is organized as follows. In Section 2, after introducing the setup of the measuring system, the calibration method of projection ray tracing line stripe profilometry is deduced. It is then extended to the phase shift measuring profilometry. And then the calibration method of image ray tracing phase shift measuring profilometry is deduced to get simple calibration equation. The distortion effects of the camera and projector lenses are considered at the end of Section 2. In Section 3, experiments are conducted to verify the proposed method. The paper is concluded in Section 4.

2. Methods

2.1 Projection ray tracing line stripe system calibration

Figure 1 gives the schematics of the system setup. The reference plane

S_{r e f}

is an arbitrary plane given by Eq. (1) in the world coordinate system

{o^{w}; x^{w}, y^{w}, z^{w}}

Fig. 1 Schematics of the system setup.

a x + b y + c z + d_{r e f} = 0.

(1)

a x + b y + c z + d_{i} = 0.

(2)

v_{s} = {[a, b, c]}^{T} .

(3)

The plane

S_{i}

, which parallels to the reference plane, is given by Eq. (2). The normal vector

v_{s}

of the planes is given by Eq. (3). So the height between surface

S_{i}

and

S_{r e f}

can be given by following Eq. (4).

h_{i} = (d_{i} - d_{r e f}) / \sqrt{a^{2} + b^{2} + c^{2}} .

(4)

The camera and the projector are described by pin-hole camera model [11

–13

S. Cui, X. Zhu, W. Wang, and Y. Xie, “Calibration of a laser galvanometric scanning system by adapting a camera model,” Appl. Opt. 48(14), 2632–2637 ( 2009). [CrossRef] [PubMed]

]. Point

o^{c}

is the optical lens center of the camera, and point

o^{p}

is the optical lens center of the projector. Camera image coordinate system

{o_{i c}; u_{c}, v_{c}}

is constructed in the camera image sensor plane with the origin at left top, and the principal point is

P_{c 0} (u_{c 0}, v_{c 0})

. The camera device coordinate system

{o^{c}; x^{c}, y^{c}, z^{c}}

is constructed with the origin at point

o^{c}

, with axis

o^{c} x^{c}

parallels to axis

o_{i c} u_{c}

and axis

o^{c} y^{c}

parallels to axis

o_{i c} v_{c}

. Axis

o^{c} z^{c}

is the main optical axis of the camera image lens. Through similar approach, the projector image coordinate system

{o_{i p}; u_{p}, v_{p}}

and the projector device coordinate system

{o^{p}; x^{p}, y^{p}, z^{p}}

are constructed, the principal point of the projector image plane is

P_{p 0} (u_{p 0}, v_{p 0})

. The distances between the image plane and the origin point are

f_{c}

(camera focus length) and

f_{p}

(projector focus length) for camera and projector systems respectively.

First of all, we will deduce the relationship of height

d_{i}

, projection position and image position. Assuming that

P_{p} (u_{p}, v_{p})

is an arbitrary point in the projector image plane, which is projected on the plane

S_{1}

at point

P_{1}

with the coordinate

X_{1} = {[x_{1}, y_{1}, z_{1}]}^{T}

and on the reference plane

S_{r e f}

at point

P_{r e f}

. This projection procedure can be described by the following sequence of transformations: the coordinate

X_{1}

of world coordinate system is translated to projector device coordinate system coordinate

X_{p}

by Eq. (5),

X_{p} = R_{p} X_{1} + t_{p} .

(5)

where

R_{p}

is a 3 × 3 orthogonal rotation matrix of the two coordinate systems, it is an expression of the rotation vector

r_{p} = {[r_{p x}, r_{p y}, r_{p z}]}^{T}

, which parallels the rotation axis and whose magnitude is equal to the rotation angle. The relationship between

R_{p}

and

r_{p}

can be described by Rodrigues equation [16

O. Faugeras, “Three-Dimensional Computer Vision: A Geometric Viewpoint,” (MIT Press, 1993).

t_{p} = {[t_{p x}, t_{p y}, t_{p z}]}^{T}

is a translation vector. The projector device coordinate

X_{p}

is translated to projector image plane by Eq. (6),

s_{p} {[\begin{matrix} u_{p} & v_{p} & 1 \end{matrix}]}^{T} = P_{p} X_{p} .

(6)

where

P_{p} = [\begin{matrix} f_{p u} & 0 & u_{p 0} \\ 0 & f_{p v} & v_{p 0} \\ 0 & 0 & 1 \end{matrix}]

f_{p u} = f_{p} / d_{p u}, f_{p v} = f_{p} / d_{p v}

, parameters

d_{p u}, d_{p v}

are the pixel sizes along horizontal and vertical direction of the projector image plane respectively, which are the priori-knowledge, and

s_{p}

is a scale factor. By substituting Eq. (5) to Eq. (6), Eq. (7) can be obtained.

s_{p} {[\begin{matrix} u_{p} & v_{p} & 1 \end{matrix}]}^{T} = P_{p} (R_{p} X_{1} + T_{p}) .

(7)

Similarly,

X_{1}

can be translated to camera image coordinate

(u_{c}, v_{c})

by following Eq. (8),

s_{c} {[\begin{matrix} u_{c} & v_{c} & 1 \end{matrix}]}^{T} = P_{c} (R_{c} X_{1} + T_{c}) .

(8)

where

P_{c} = [\begin{matrix} f_{c u} & 0 & u_{c 0} \\ 0 & f_{c v} & v_{c 0} \\ 0 & 0 & 1 \end{matrix}]

f_{c u} = f_{c} / d_{c u}, f_{c v} = f_{c} / d_{c v}

, parameters

d_{c u}, d_{c v}

are the pixel sizes along horizontal and vertical direction of the image plane respectively, which are the priori-knowledge, and

s_{c}

is a scale factor.

R_{c}

is the 3×3 orthogonal rotation matrix of the camera device coordinate system and world coordinate system, and

t_{c} = {[t_{c x}, t_{c y}, t_{c z}]}^{T}

is the translation vector. By defining

M_{p} = P_{p} R_{p}

t_{b p} = P_{p} t_{p}

M_{c} = P_{c} R_{c}

t_{b c} = P_{c} t_{c}

, Eq. (7) can be written as the following Eq. (9),

s_{p} {[\begin{matrix} u_{p} & v_{p} & 1 \end{matrix}]}^{T} = M_{p} X_{1} + t_{b p} .

(9)

Equation (8) can be written as following Eq. (10).

s_{c} {[\begin{matrix} u_{c} & v_{c} & 1 \end{matrix}]}^{T} = M_{c} X_{1} + t_{b c} .

(10)

Parameters

M_{c}, M_{p}

are 3 × 3 matrices, for the convenience of the induction, let

M_{c} = {[m_{c 1}, m_{c 2}, m_{c 3}]}^{T}

M_{p} = {[m_{p 1}, m_{p 2}, m_{p 3}]}^{T}

m_{c i}, m_{p i}

are the row vectors of matrices

M_{c}

M_{p}

. Similarly, let

t_{b c} = {[t_{b c x}, t_{b c y}, t_{b c z}]}^{T}

t_{b p} = {[t_{b p x}, t_{b p y}, t_{b p z}]}^{T}

. Once the system is set up, parameters

R_{c}, R_{p}, t_{p}, t_{c}, P_{p}, P_{c}

are changeless, so parameters

M_{c}, M_{p}, t_{b p}, t_{b c}

are changeless too. From Eq. (9) and Eq. (10), the following Eq. (11) can be constructed to acquire the world coordinate

X_{1}

when

u_{c}, v_{c}, v_{p}

are known.

A X_{1} = B .

(11)

where

A = [\begin{matrix} m_{c 1} - u_{c} m_{c 3} \\ m_{c 2} - v_{c} m_{c 3} \\ m_{p 2} - v_{p} m_{p 3} \end{matrix}]

B = - [\begin{matrix} t_{b c x} - u_{c} t_{b c z} \\ t_{b c y} - v_{c} t_{b c z} \\ t_{b p y} - v_{p} t_{b p z} \end{matrix}]

. Coordinate

X_{1}

is in plane

S_{1}

, substitute

X_{1}

and Eq. (3) to Eq. (2), the following Eq. (12) can be obtained.

v_{s}^{T} X_{1} + d_{i} = 0.

(12)

d_{i}

is given by following Eq. (13).

d_{i} = - v_{s}^{T} X_{1} .

(13)

By substituting the solution of Eq. (11) to Eq. (13), Eq. (14) can be obtained. The explicit expressions of Eq. (14) are calculated by symbol computation in MATLAB. In the calculation procedure, parameters

M_{c}, M_{p}, t_{b p}, t_{b c}, v_{s}

are constant, and

u_{c}, v_{c}, v_{p}

are variables. From the calculation result of Eq. (13), it can be found out that

d_{i}

is the sum of three fractions, of which denominator are the same, and both the numerator and denominator contain the variable items of

v_{c}, u_{c}, v_{p} v_{c}, v_{p} u_{c}, v_{p}

and the constant item. Therefore, Eq. (14) can be obtained.

d_{i} = \frac{a_{1} v_{c} + a_{2} u_{c} + a_{3} + a_{4} v_{p} v_{c} + a_{5} v_{p} u_{c} + a_{6} v_{p}}{b_{1} v_{c} + b_{2} u_{c} + b_{3} + b_{4} v_{p} v_{c} + b_{5} v_{p} u_{c} + b_{6} v_{p}} .

(14)

where

a_{1}, a_{2}, a_{3}, a_{4}, a_{5}, a_{6}

b_{1}, b_{2}, b_{3}, b_{4}, b_{5}, b_{6}

are constant parameters determined by system parameters

M_{p}, M_{c}, t_{b p}, t_{b c}

and reference plane parameters

a, b, c

. To simplify the description, define

c_{1} = (a_{1} + a_{4} v_{p}) / (b_{1} + b_{4} v_{p})

c_{2} = (a_{2} + a_{5} v_{p}) / (a_{1} + a_{4} v_{p})

c_{3} = (a_{3} + a_{6} v_{p}) / (a_{1} + a_{4} v_{p})

c_{4} = (b_{2} + b_{5} v_{p}) / (b_{1} + b_{4} v_{p})

c_{5} = (b_{3} + b_{6} v_{p}) / (b_{1} + b_{4} v_{p})

, then Eq. (14) turns to Eq. (15),

d_{i} = \frac{k_{c 1} u_{c} + k_{c 2}}{v_{c} + c_{4} u_{c} + c_{5}} + c_{1} .

(15)

where

k_{c 1} = c_{1} (c_{2} - c_{4})

k_{c 2} = c_{1} (c_{3} - c_{5})

Next, we will investigate the relationship between the height and image pixel vertical distance of the same projection points from Eq. (15) for the projection ray tracing line stripe pattern systems. Line stripe pattern

v_{p} = v_{p g i v e n}

of the projection image, shown in Fig. 2(a) , is projected on reference plane. The image is acquired by camera, which is shown in Fig. 2(b). As the surface plane

S_{i}

is placed ahead of the reference plane, the image acquired by camera is shown in Fig. 2(c). The two images are combined together for illustration, as shown in Fig. 2(d). For the two lines, the values of

v_{p}

are the same, so parameters

c_{1}, c_{2}, c_{3}, c_{4}, c_{5}, k_{c 1}, k_{c 2}

are the same. By substituting

v_{c r e f}

to Eq. (15), parameter

d_{r e f}

can be represented by the following Eq. (16):

d_{r e f} = \frac{k_{c 1} u_{c} + k_{c 2}}{v_{c r e f} + c_{4} u_{c} + c_{5}} + c_{1} .

(16)

Equation (15) and Eq. (16) are substituted to Eq. (4), so Eq. (17) can be obtained.

h_{i} = \frac{k_{n 1} u_{c} + k_{n 2}}{v_{c} + c_{4} u_{c} + c_{5}} - \frac{k_{n 1} u_{c} + k_{n 2}}{v_{c r e f} + c_{4} u_{c} + c_{5}} .

(17)

where

k_{n 1} = k_{c 1} \sqrt{a^{2} + b^{2} + c^{2}}

k_{n 2} = k_{c 2} \sqrt{a^{2} + b^{2} + c^{2}}

. By calculating the reciprocal of Eq. (17), the following Eq. (18) can be obtained,

\frac{1}{h_{i}} = - \frac{{(v_{c r e f} + c_{4} u_{c} + c_{5})}^{2}}{k_{n 1} u_{c} + k_{n 2}} \frac{1}{v_{c} - v_{c r e f}} - \frac{v_{c r e f} + c_{4} u_{c} + c_{5}}{k_{n 1} u_{c} + k_{n 2}} .

(18)

From Eq. (16),

v_{c r e f} + c_{4} u_{c} + c_{5} = (k_{c 1} u_{c} + k_{c 2}) / (d_{r e f} - c_{1})

can be known, by substituting it to Eq. (18), Eq. (19) can be obtained.

Fig. 2 Line stripe pattern of the projectors image and camera image. (a) A line stripe where

v_{p} = v_{p g i v e n}

on the projector image. (b) An image of the reference plane. (c) An image of the test plane. (d) The combination of two images.

\frac{1}{h_{i}} = - \frac{k_{c 1} u_{c} + k_{c 2}}{{(d_{r e f} - c_{1})}^{2} \sqrt{a^{2} + b^{2} + c^{2}}} \frac{1}{v_{c} - v_{c r e f}} - \frac{1}{(d_{r e f} - c_{1}) \sqrt{a^{2} + b^{2} + c^{2}}} .

(19)

Define

y = h_{i}^{- 1}

x = {(v_{c} - v_{c r e f})}^{- 1}

v_{c} - v_{c r e f}

is the vertical distance between the reference line and test lines, which is shown in Fig. 2(d), represented by the double row line, Eq. (19) turns to Eq. (20), parameter

p_{1}

is given by Eq. (21),

y = p_{1} (u_{c}, v_{c}) x + p_{2} .

(20)

p_{1} (u_{c}, v_{c}) = k_{b 1} u_{c} + k_{b 2} .

(21)

where

k_{b 1} = - k_{c 1} {(d_{r e f} - c_{1})}^{- 2} {(a^{2} + b^{2} + c^{2})}^{- 0.5}

k_{b 2} = - k_{c 2} {(d_{r e f} - c_{1})}^{- 2} {(a^{2} + b^{2} + c^{2})}^{- 0.5}

p_{2} = - {(d_{r e f} - c_{1})}^{- 1} {(a^{2} + b^{2} + c^{2})}^{- 0.5}

are constant for the stripe line.

p_{1} (u_{c}, v_{c})

is the coefficient at the camera image pixel position

(u_{c}, v_{c})

, it does not contain the item

v_{c}

. This equation can be used conveniently in optical stripe pattern measuring systems due to the projection ray tracing fact.

After the deduction of the calibration equation, the calibration procedure for the projection ray tracing line stripe profilometry can be described by the following steps:

step 1. Project the line stripe pattern to the reference planes and acquire the image (noted as reference image) by camera.
step 2. Place N(N≥2) parallel planes on top of the reference planes(noted as test plane 1, …,test plane N) and acquire image respectively(noted as test image 1,…, test image N).
step 3. Extract the line positions of the reference image and test images by algorithms, such as C. Steger method [17
C. Steger, “An Unbiased Detector of Curvilinear Structures,” IEEE Trans. Pattern Anal. Mach. Intell. 20(2), 113–125 ( 1998). [CrossRef]
].
step 4. For a horizontal position $u_{c}$ of the test images, calculate $v_{d i f} = v_{c} - v_{c r e f}$ to form $[v_{d i f 1}, v_{d i f 2}, ..., v_{d i f N}]$ , and the height between test planes and reference plane are $[h_{1}, h_{2}, ..., h_{N}]$ , $p_{1}, p_{2}$ at horizontal position $u_{c}$ can be determined by using the linear fitting algorithm of Eq. (20).
step 5. Repeat step 4 for every $u_{c}$ value to form a coefficient table.
step 6. The $u_{c}$ , $p_{1}$ values acquired in step 5 is fitted by linear algorithm of Eq. (21) to acquire coefficients $k_{b 1}, k_{b 2}$ , this will reduce the noise influence(relative to the single point).

After the system is calibrated and parameters

k_{b 1}, k_{b 2}, p_{2}

are determined, given an arbitrary specimen, its image is acquired by camera. After the extraction of the line position, the height can be calculated by Eq. (22).

h (u_{c}, v_{c}) = {(k_{b 1} u_{c} + k_{b 2}) {(v_{c} - v_{c r e f})}^{- 1} + p_{2}}^{- 1} .

(22)

2.2 Projection ray tracing phase shift measuring profilometery calibration

The above method can also be extended to be used in the phase shift measuring profilometry [18

P. S. Huang and S. Zhang, “Fast three-step phase-shifting algorithm,” Appl. Opt. 45(21), 5086–5091 ( 2006). [CrossRef] [PubMed]

], where

v_{p}

is a variable item. Since parameters

k_{c 1}, k_{c 2}, c_{1}

are not constant, the coefficients of Eq. (20) at pixel position

(u_{c}, v_{c})

turn to following Eq. (23) and Eq. (24),

p_{1} (u_{c}, v_{c}, v_{p}) = \frac{k_{p 1 a} v_{p} + k_{p 1 b}}{{(v_{p} + k_{p 0})}^{2}} u_{c} + \frac{k_{p 2 a} v_{p} + k_{p 2 b}}{{(v_{p} + k_{p 0})}^{2}} .

(23)

p_{2} (u_{c}, v_{c}, v_{p}) = (k_{p a} v_{p} + k_{p b}) / (v_{p} + k_{p 0}) .

(24)

v_{p}

can be translated from the phase by the following Eq. (25),

v_{p} = ϕ / 2 π .

(25)

Parameters

k_{p 0}, k_{p 1 a}, k_{p 1 b}, k_{p 2 a}, k_{p 2 b}, k_{p a}, k_{p b}

are the functions of

a_{1}, a_{2}, a_{3}, a_{4}, a_{5}, a_{6}

b_{1}, b_{2}, b_{3}, b_{4}

b_{5}, b_{6}

. The key point of the algorithms is to determine the reference line of the given phase valueϕ. For a given pixel position

(u_{c 1}, v_{c 1})

of the test plane image (noted as image 1), its extraction phase is

ϕ_{1}

, to calculate the height by Eq. (22), the pixel position

(u_{c 1 r e f}, v_{c 1 r e f})

of the same phase

ϕ_{1}

on the reference plane image (noted as image 2) should be found out. Then the relationship of

v_{c 1}

and

v_{p 1}

will be discussed. Define

D = [\begin{matrix} m_{c 1} - u_{c 1} m_{c 3} \\ m_{p 2} - v_{p 1} m_{p 3} \\ \begin{matrix} a & b & c \end{matrix} \end{matrix}]

E = - [\begin{matrix} t_{b c x} - u_{c 1} t_{b c z} \\ t_{b p y} - v_{p 1} t_{b p z} \\ d_{1} \end{matrix}]

, so the Point

P_{i}

’s coordinates can be got from equation

D X_{i} = E

, this equation is constructed from Eq. (2), Eq. (9) and Eq. (10). Then the image point of

X_{i}

in the camera can be calculated from Eq. (10). By substituting the solution of equation

D X_{i} = E

to Eq. (10), Eq. (26) can be obtained. The explicit expression of Eq. (26) is obtained by the symbol computing in Matlab. In the calculation procedure, parameters

M_{p}, M_{c}, t_{b p}, t_{b c}

a, b, c, d_{1}

and

u_{c 1}

are constant, the only variable is

v_{p 1}

. The calculation result is in the form of fraction after the simplification, where both the numerator and denominator contain single variable item

v_{p 1}

. So it can be written in the form of Eq. (26).

v_{c 1} = \frac{k_{r c 1}}{v_{p 1} + k_{r c 0}} + k_{r c 2} .

(26)

Parameters

k_{r c 1}, k_{r c 2}, k_{r c 0}

are determined by system constant parameters

M_{p}, M_{c}, t_{b p}, t_{b c}

a, b, c, d_{i}

and parameter

u_{c}

(known value).

After the deduction of the calibration equation, the calibration procedure of projection ray tracing phase shift measuring profilometery can be described by the following steps:

step 1. The phase shift images are projected on the reference planes and test planes, the images are acquired by camera.
step 2. The phase are calculated and unwrapped for each plane (noted as reference phase, test phase 1, …, test phase M) [19
J. Meneses, T. Gharbi, and P. Humbert, “Phase-unwrapping algorithm for images with high noise content based on a local histogram,” Appl. Opt. 44(7), 1207–1215 ( 2005). [CrossRef] [PubMed]
].
step 3. For each column pixels of the reference phase, variable $v_{c 1 r e f}$ , $v_{p 1}$ are known, so nonlinear fit algorithm can be used to determine the parameters $k_{r c 1}, k_{r c 2}, k_{r c 0}$ in Eq. (26).
step 4. Repeat step 3 for each test phases.
step 5. For one point $(u_{c k}, v_{c k i})$ in test plane i, its phase is $ϕ_{t}$ ,and corresponding projection image position is $v_{p 1}$ . By taking the parameters obtained in step3, step4, the reference pixel position $(u_{c k}, v_{c k r e f})$ , the corresponding pixel positions $(u_{c k}, v_{c k 1})$ , …, $(u_{c k}, v_{c k N})$ of the same phase on other test planes can be determined through Eq. (26), parameters $p_{1}, p_{2}$ can be determined by linear fit algorithms of Eq. (20).
step 6. Repeat step 5 for each points of the test plane i, parameters $p_{1} (u_{c 1}, v_{c 1}, v_{p 1})$ , $p_{1} (u_{c 2}, v_{c 2}, v_{p 2})$ , …, $p_{1} (u_{c M}, v_{c M}, v_{p M})$ and $p_{2} (u_{c 1}, v_{c 1}, v_{p 1})$ , $p_{2} (u_{c 2}, v_{c 2}, v_{p 2})$ ,…, $p_{2} (u_{c M}, v_{c M}, v_{p M})$ are acquired, and $u_{c 1}, v_{c 1}, v_{p 1}, u_{c 2}, v_{c 2}, v_{p 2}, ..., u_{c M}, v_{c M}, v_{p M}$ are known, so lest square methods can be used to determine the parameters $k_{p 0}, k_{p 1 a}, k_{p 1 b}, k_{p 2 a}, k_{p 2 b}, k_{p a}, k_{p b}$ in Eq. (23) and Eq. (24).

After the calibration is done, parameters can be used in the measure procedure. The phase image on the specimen is computed. For each pixel

(u_{c}, v_{c})

, its phaseϕis translated to

v_{p}

by Eq. (25), and then the reference pixel position is acquired by Eq. (26).

p_{1}, p_{2}

are determined by Eq. (23), Eq. (24), so height can be determined by Eq. (27).

h = {p_{1} (u_{c}, v_{c}, v_{p}) {(v_{c} - v_{c r e f})}^{- 1} + p_{2} (u_{c}, v_{c}, v_{p})}^{- 1}

(27)

2.3 Image ray tracing phase shift measuring profilometery calibration

From the above description of Section 2.2, it can be known that the calibration and calculation procedure is complicated for the phase shift measuring profilometry, and it’s not convenient, particularly in finding the reference position of given phase. If the image ray tracing method is used, we can use the phase shift information at the same pixel position for calibration. It’s convenient for calculating. Rewrite Eq. (14) as following Eq. (28),

d_{i} = t_{1} + \frac{t_{2}}{v_{p} + t_{3}} .

(28)

where

t_{1} = (a_{4} v_{c} + a_{5} u_{c} + a_{6}) {(b_{4} v_{c} + b_{5} u_{c} + b_{6})}^{- 1}

t_{2} = {(a_{1} v_{c} + a_{2} u_{c} + a_{3}) (b_{4} v_{c} + b_{5} u_{c} + b_{6}) - (b_{1} v_{c} + b_{2} u_{c} + b_{3}) (a_{4} v_{c} + a_{5} u_{c} + a_{6})} {(b_{4} v_{c} + b_{5} u_{c} + b_{6})}^{- 2}

t_{3} = (b_{1} v_{c} + b_{2} u_{c} + b_{3}) {(b_{4} v_{c} + b_{5} u_{c} + b_{6})}^{- 1}

. For the same pixel position

(u_{c}, v_{c})

of the camera image plane, parameters

t_{1}, t_{2}, t_{3}

are constant, so Eq. (19) turns to Eq. (29).

\frac{1}{h_{i}} = - \frac{1}{d_{r e f} - t_{1}} - \frac{t_{2}}{{(d_{r e f} - t_{1})}^{2}} \frac{1}{v_{p} - v_{p r e f}} .

(29)

This equation equals to Eq. (20) when

p_{1} = - t_{2} {(d_{r e f} - t_{1})}^{- 2}, p_{2} = - {(d_{r e f} - t_{1})}^{- 1}

v_{p}, v_{p r e f}

can be translated from the phase by Eq. (25). For each point, the reciprocal of the height and reciprocal of the phase shift satisfy Eq. (29).

Then the calibration procedure of image ray tracing phase shift measuring profilometery calibration can be described by the following steps:

step 1. The phase shift images are projected on the reference planes and test planes, the images are acquired by camera.
step 2. The phase are calculated and unwrapped for each plane (noted as reference phase, test phase 1, …, test phase M).
step 3. For pixel position $(u_{c k}, v_{c k})$ , the reference phase is $ϕ_{r e f}$ , its corresponding $v_{p}$ value is $v_{p r e f}$ , for test planes, $v_{p}$ values are $v_{p 1}$ , $v_{p 2}$ , …, $v_{p M}$ . Height $h_{i}^{- 1}$ is known and ${(v_{p i} - v_{p r e f})}^{- 1}$ is computed, so parameters $p_{1}, p_{2}$ are determined by linear fit algorithm of Eq. (20);
step 4. Repeat step 3 for each pixel position of reference image, the coefficients table for $p_{1}, p_{2}$ can be formed.

After calibration, the height can be calculated by Eq. (30).

h_{i} = {p_{1} (u_{c}, v_{c}) {(v_{p i} - v_{p r e f})}^{- 1} + p_{2} (u_{c}, v_{c})}^{- 1} .

(30)

2.4 Lens distortion compensation

All the above calculations base on the ideal model of the projector and camera lenses, but in the practical systems, the lens distortions of camera and the projector should be considered to achieve high measurement accuracies. The distortions of the lens usually contain radial and tangential distortion modeled by the following Eq. (31) [11

\begin{array}{l} u_{d} = u + 2 P_{1} u v + P_{2} (r^{2} + 2 u^{2}) + K_{1} u r^{2} + K_{2} u r^{4} + K_{3} u r^{6} \\ v_{d} = v + 2 P_{2} u v + P_{1} (r^{2} + 2 v^{2}) + K_{1} v r^{2} + K_{2} v r^{4} + K_{3} v r^{6} . \end{array}

(31)

where

(u, v)

is the ideal point, and

(u_{d}, v_{d})

is the distortion point,

r = \sqrt{u^{2} + v^{2}}

K_{1}, K_{2}

and

K_{3}

are the coefficients of radial distortion,

P_{1}

and

P_{2}

are the coefficients of tangential distortions. Calibration procedure [11

–13

S. Cui, X. Zhu, W. Wang, and Y. Xie, “Calibration of a laser galvanometric scanning system by adapting a camera model,” Appl. Opt. 48(14), 2632–2637 ( 2009). [CrossRef] [PubMed]

] can be used to acquire these distortion coefficients of the camera lens, and the projector lens distortions can be calibrated by the modify calibration procedure of Zhang et al. [14

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 45(8), 083601 ( 2006). [CrossRef]

After the distortion coefficients having been calibrated, iteration method will be used to calculate the ideal point

(u, v)

from

(u_{d}, v_{d})

for the camera image [15

R. L. Saenz, T. Bothe, and W. P. Juptner, “Accurate procedure for the calibration of a structured light system,” Opt. Eng. 43, 467–471 ( 2004).

]. For the projector image, the ideal point

(u, v)

is the known value, and then the distortion point

(u_{d}, v_{d})

is calculated by Eq. (31). When

(u_{d}, v_{d})

is projected on the test object, after the distortion of the projector lens, the approximate ideal point can be acquired on the test objects. In the proposed method of this work, all the images acquired by the camera are compensated before being used in the calibration procedure, and all the projector images are compensated before being projected.

3. Experiments

We conduct experiments to verify the proposed method. The line stripe pattern optical triangular measuring system is set up as shown in Fig. 1. The reference plane, the camera and projector are fixed after being set up. Blocks of the fixed thickness are placed on the reference plane to form the parallel test planes. The image of reference plane acquired by the camera is shown in Fig. 3(a) , one of the test images is shown in Fig. 3(b), and Fig. 3(c) are the combination of the reference and test line stripes extracted by Steger line position extraction algorithm from the images. By taking the procedures described in Section 2.1, parameters

p_{1}, p_{2}

of each point can be determined. Figure 3(d) shows the plot of

p_{1}

and

u_{c}

. From the figure, it can be found out that

p_{1}

is linear with

u_{c}

suggesting that the experiment result coincides with the Eq. (21). After the calibration being done, a block with thickness of 162.3um is measured, the results are shown in Fig. 3(e), and the errors are shown in Fig. 3(f). The mean error is 0.03um and root mean square error is 0.21um. In the experiment, the camera and projector are in arbitrary position and direction and the reference plane is in arbitrary direction too.

Fig. 3 Experiments result for the line stripe pattern profilometry. (a) Camera image of the reference plane. (b) Camera image of one of test planes. (c) The combination image of the extracted test and reference lines. (d) Plot of parameter

p_{1}

and variable

u_{c}

. (e) Measuring result of a given block. (f) Error of (e).

A phase shift measuring profilometry experiment is conducted to verify the proposed image ray tracing calibration method. Three steps shift algorithm is used [18

P. S. Huang and S. Zhang, “Fast three-step phase-shifting algorithm,” Appl. Opt. 45(21), 5086–5091 ( 2006). [CrossRef] [PubMed]

], the projector is NEC VT491 + digital projector, and the camera is SONY A-300, five test planes parallel to the reference plane are used, the distance from the reference plane are 200mm, 400mm, 600mm, 800mm, and 1000mm, the distance accuracy is ± 0.05mm, and the camera and projector are focused at the distance of 500 mm from the reference plane. The system is calibrated by procedures described in Section 2.3, and a coefficients table for each pixel position

(u_{c}, v_{c})

is acquired. These parameters are used to calculate the height of a given plane whose distance from the reference plane is 500mm, the measured distance is shown in Fig. 4(a) , the errors are shown in Fig. 4(b), the mean distance value of the measured plane is 499.92mm, and the root mean square error is 0.12mm. A face experiment is also conducted to verify the method for the complicated objects. Figure 5(a) shows 2D form of the measured face, the height of the face is shown as gray values, and Fig. 5(b) shows the 3-D mesh grid of the measured face.

Fig. 4 The measured result of the given plane. (a) Measured result. (b) The error of the measured result

Fig. 5 Measured result of face. (a) 2D show of the face, gray value represented the height dedicated by the color bar on the right side of the Fig. 5(b) 3D mesh grid of the measured face.

4. Conclusion

In this work, we propose a generalized calibration method in optical triangular profilometry by modeling the projector and camera as pin-hole camera models and the reference (test) planes as parallel planes. For the projection ray tracing line strip measuring systems, the reciprocal of height and the reciprocal of the pixel position vertical distance are in linear relationship, and for the image ray tracing phase shift measuring profilometry, the reciprocal of height and the reciprocal of the phase shift (change) of the same pixel position are in linear relationship. So for both the projection and image ray tracing systems, the simple linear calibration equation is obtained, which has fewer coefficients, so computer speed is fast for measuring. The distortion effects of images should be compensated before images being used in the proposed method. Experiments of the line stripe pattern and phase shift measuring profilometry are conducted to verify the proposed methods.

The calibration equations we obtained are consistent with the results from the geometry approaches. While the accuracy of the calibration was not significantly improved in our work as compared to the geometry approach [1

W. S. Zhou and X. Y. Su, “A direct mapping algorithm for phase-measuring profilometry,” J. Mod. Opt. 41(1), 89–94 ( 1994). [CrossRef]

–10

A. Asundi and Z. Wensen, “Unified calibration technique and its applications in optical triangular profilometry,” Appl. Opt. 38(16), 3556–3561 ( 1999). [CrossRef] [PubMed]

], our work is the first attempt to use matrices in the reference-plane approaches. With the use of matrices, our model has a number of important advantages over the traditional model derived from geometry: firstly, the deduction of the calibration is more convenient and simple, giving results that are easy to understand; secondly, it supports the complicated rotations and translations for the system setup; thirdly, the same linear calibration equation for both projection and image ray tracing system can be acquired.

There are some extended results for this paper, for the image ray tracing phase shift measuring profilometry, the coefficients can be also given by the expression of parameters

a_{1}, a_{2}, a_{3}, a_{4}, a_{5}, a_{6}, b_{1}, b_{2}, b_{3}, b_{4}, b_{5}, b_{6}

, which are determined by the system setup. The coefficients table and

(u_{c}, v_{c})

values can be used to determine these parameters by least square methods [7

Z. Wang, H. Du, and H. Bi, “Out-of-plane shape determination in generalized fringe projection profilometry,” Opt. Express 14(25), 12122–12133 ( 2006). [CrossRef] [PubMed]

–9

Z. Wang, H. Du, S. Park, and H. Xie, “Three-dimensional shape measurement with a fast and accurate approach,” Appl. Opt. 48(6), 1052–1061 ( 2009). [CrossRef]

], but it is complicated and unnecessary in practice.

Acknowledgement

The authors wish to express their thanks to Mr. Ho Simon Wang at HUST Academic Writing Center for improving the structure and presentation of the paper. We also wish to thank Changhong Zhu at HUST for his helpful discussions. We extended our thanks to the reviewers for their helpful and constructive comments.

References and links

1.	W. S. Zhou and X. Y. Su, “A direct mapping algorithm for phase-measuring profilometry,” J. Mod. Opt. 41(1), 89–94 ( 1994). [CrossRef]
2.	L. Chen and C. Quan, “Fringe projection profilometry with nonparallel illumination: a least-squares approach,” Opt. Lett. 30(16), 2101–2103 ( 2005). [CrossRef] [PubMed]
3.	L. Chen and C. J. Tay, “Carrier phase component removal: a generalized least-square approach,” J. Opt. Soc. Am. A 23(2), 435–443 ( 2006). [CrossRef]
4.	H. Guo, H. He, Y. Yu, and M. Chen, “Least-squares calibration method for fringe projection profilometry,” Opt. Eng. 44(3), 033603 ( 2005). [CrossRef]
5.	H. Guo, M. Chen, and P. Zheng, “Least-squares fitting of carrier phase distribution by using a rational function in fringe projection profilometry,” Opt. Lett. 31(24), 3588–3590 ( 2006). [CrossRef] [PubMed]
6.	B. A. Rajoub, M. J. Lalor, D. R. Burton, and S. A. Karout, “A new model for measuring object shape using non-collimated fringe-pattern projections,” J. Opt. A, Pure Appl. Opt. 9(6), S66–S75 ( 2007). [CrossRef]
7.	Z. Wang, H. Du, and H. Bi, “Out-of-plane shape determination in generalized fringe projection profilometry,” Opt. Express 14(25), 12122–12133 ( 2006). [CrossRef] [PubMed]
8.	H. Du and Z. Wang, “Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry system,” Opt. Lett. 32(16), 2438–2440 ( 2007). [CrossRef] [PubMed]
9.	Z. Wang, H. Du, S. Park, and H. Xie, “Three-dimensional shape measurement with a fast and accurate approach,” Appl. Opt. 48(6), 1052–1061 ( 2009). [CrossRef]
10.	A. Asundi and Z. Wensen, “Unified calibration technique and its applications in optical triangular profilometry,” Appl. Opt. 38(16), 3556–3561 ( 1999). [CrossRef] [PubMed]
11.	J. Heikkila, and O. Silven, “Calibration Procedure for short focal length off-the-shelf CCD cameras,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vienna, Austria, 1996), pp. 166–170.
12.	Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 ( 2000). [CrossRef]
13.	S. Cui, X. Zhu, W. Wang, and Y. Xie, “Calibration of a laser galvanometric scanning system by adapting a camera model,” Appl. Opt. 48(14), 2632–2637 ( 2009). [CrossRef] [PubMed]
14.	S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 45(8), 083601 ( 2006). [CrossRef]
15.	R. L. Saenz, T. Bothe, and W. P. Juptner, “Accurate procedure for the calibration of a structured light system,” Opt. Eng. 43, 467–471 ( 2004).
16.	O. Faugeras, “Three-Dimensional Computer Vision: A Geometric Viewpoint,” (MIT Press, 1993).
17.	C. Steger, “An Unbiased Detector of Curvilinear Structures,” IEEE Trans. Pattern Anal. Mach. Intell. 20(2), 113–125 ( 1998). [CrossRef]
18.	P. S. Huang and S. Zhang, “Fast three-step phase-shifting algorithm,” Appl. Opt. 45(21), 5086–5091 ( 2006). [CrossRef] [PubMed]
19.	J. Meneses, T. Gharbi, and P. Humbert, “Phase-unwrapping algorithm for images with high noise content based on a local histogram,” Appl. Opt. 44(7), 1207–1215 ( 2005). [CrossRef] [PubMed]

OCIS Codes
(120.2830) Instrumentation, measurement, and metrology : Height measurements
(120.5050) Instrumentation, measurement, and metrology : Phase measurement
(120.6650) Instrumentation, measurement, and metrology : Surface measurements, figure

ToC Category:
Instrumentation, Measurement, and Metrology

History
Original Manuscript: September 1, 2009
Revised Manuscript: October 21, 2009
Manuscript Accepted: October 22, 2009
Published: October 28, 2009

Citation
Suochao Cui and Xiao Zhu, "A generalized reference-plane-based calibration method in optical triangular profilometry," Opt. Express 17, 20735-20746 (2009)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-17-23-20735

Sort: Author | Year | Journal | Reset

References

W. S. Zhou and X. Y. Su, “A direct mapping algorithm for phase-measuring profilometry,” J. Mod. Opt. 41(1), 89–94 (1994). [CrossRef]
L. Chen and C. Quan, “Fringe projection profilometry with nonparallel illumination: a least-squares approach,” Opt. Lett. 30(16), 2101–2103 (2005). [CrossRef] [PubMed]
L. Chen and C. J. Tay, “Carrier phase component removal: a generalized least-square approach,” J. Opt. Soc. Am. A 23(2), 435–443 (2006). [CrossRef]
H. Guo, H. He, Y. Yu, and M. Chen, “Least-squares calibration method for fringe projection profilometry,” Opt. Eng. 44(3), 033603 (2005). [CrossRef]
H. Guo, M. Chen, and P. Zheng, “Least-squares fitting of carrier phase distribution by using a rational function in fringe projection profilometry,” Opt. Lett. 31(24), 3588–3590 (2006). [CrossRef] [PubMed]
B. A. Rajoub, M. J. Lalor, D. R. Burton, and S. A. Karout, “A new model for measuring object shape using non-collimated fringe-pattern projections,” J. Opt. A, Pure Appl. Opt. 9(6), S66–S75 (2007). [CrossRef]
Z. Wang, H. Du, and H. Bi, “Out-of-plane shape determination in generalized fringe projection profilometry,” Opt. Express 14(25), 12122–12133 (2006). [CrossRef] [PubMed]
H. Du and Z. Wang, “Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry system,” Opt. Lett. 32(16), 2438–2440 (2007). [CrossRef] [PubMed]
Z. Wang, H. Du, S. Park, and H. Xie, “Three-dimensional shape measurement with a fast and accurate approach,” Appl. Opt. 48(6), 1052–1061 (2009). [CrossRef]
A. Asundi and Z. Wensen, “Unified calibration technique and its applications in optical triangular profilometry,” Appl. Opt. 38(16), 3556–3561 (1999). [CrossRef] [PubMed]
J. Heikkila, and O. Silven, “Calibration Procedure for short focal length off-the-shelf CCD cameras,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vienna, Austria, 1996), pp. 166–170.
Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). [CrossRef]
S. Cui, X. Zhu, W. Wang, and Y. Xie, “Calibration of a laser galvanometric scanning system by adapting a camera model,” Appl. Opt. 48(14), 2632–2637 (2009). [CrossRef] [PubMed]
S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 45(8), 083601 (2006). [CrossRef]
R. L. Saenz, T. Bothe, and W. P. Juptner, “Accurate procedure for the calibration of a structured light system,” Opt. Eng. 43, 467–471 (2004).
O. Faugeras, “Three-Dimensional Computer Vision: A Geometric Viewpoint,” (MIT Press, 1993).
C. Steger, “An Unbiased Detector of Curvilinear Structures,” IEEE Trans. Pattern Anal. Mach. Intell. 20(2), 113–125 (1998). [CrossRef]
P. S. Huang and S. Zhang, “Fast three-step phase-shifting algorithm,” Appl. Opt. 45(21), 5086–5091 (2006). [CrossRef] [PubMed]
J. Meneses, T. Gharbi, and P. Humbert, “Phase-unwrapping algorithm for images with high noise content based on a local histogram,” Appl. Opt. 44(7), 1207–1215 (2005). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Click here to see a list of articles that cite this paper