Flying triangulation—an optical 3D sensor for the motion-robust acquisition of complex objects

doi:10.1364/AO.51.000281

« Show journal navigation

Flying triangulation—an optical 3D sensor for the motion-robust acquisition of complex objects

Svenja Ettl, Oliver Arold, Zheng Yang, and Gerd Häusler »View Author Affiliations

Applied Optics, Vol. 51, Issue 2, pp. 281-289 (2012)
http://dx.doi.org/10.1364/AO.51.000281

View Full Text Article

Acrobat PDF (877 KB)

Journal Search
Article Lookup

Article Tools

Citations

Alert me when this article is cited
Export Citation/Save

Full Text
Article Info
References (31)
Cited By
Figures (8)
Multimedia (1)
Metrics

Abstract

Three-dimensional (3D) shape acquisition is difficult if an all-around measurement of an object is desired or if a relative motion between object and sensor is unavoidable. An optical sensor principle is presented—we call it “flying triangulation”—that enables a motion-robust acquisition of 3D surface topography. It combines a simple handheld sensor with sophisticated registration algorithms. An easy acquisition of complex objects is possible—just by freely hand-guiding the sensor around the object. Real-time feedback of the sequential measurement results enables a comfortable handling for the user. No tracking is necessary. In contrast to most other eligible sensors, the presented sensor generates 3D data from each single camera image.

1. Introduction

In many fields of application, an optical measurement of the three-dimensional (3D) topography of complex objects is desired. In art preservation, a complete optical 3D acquisition of a sculpture (see Fig. 1(a)) is necessary for documentation or for producing a duplicate [1

X. Laboureux and G. Häusler, “Localization and registration of three-dimensional objects in space—where are the limits?,” Appl. Opt. 40, 5206–5216 (2001). [CrossRef]

]. In the medical field, 3D surface data of human body parts support a surgeon. In oral and maxillofacial surgery, the displacement of bones can be detected and corrected more easily [2

M. Benz, J. Hartmann, T. Maier, E. Nkenke, K. Veit, A. Stellzig-Eisenhauer, F. W. Neukam, and G. Häusler, “Optical 3d-metrology for medical applications,” in Biomedizinische Technik. Proc. ICMP 2005 and BMT 2005 , U. Boenick, A. Bolz, W. Kalender, E. G. Hahn, and A. M. Schulte, eds. (Schiele und Schön, 2005), pp. 48–49.

]. For the manufacturing of tooth crowns, an optical intraoral measurement of teeth reduces the discomfort for the patient in comparison to the standard method based on taking an impression.

Fig. 1. (a) An example of a complex object difficult to measure is the “Trombone Angel” of the Bamberg Dome. (b) A virtual copy of the Trombone Angel [1

X. Laboureux and G. Häusler, “Localization and registration of three-dimensional objects in space—where are the limits?,” Appl. Opt. 40, 5206–5216 (2001). [CrossRef]

]. (c) Desired is a sensor that enables a motion-robust acquisition of an object surface by freely hand-guiding the sensor around the object (chain of dots mark an example of a sensor path; a projected line pattern is seen on the dental cast).

However, such measurement tasks bear essential challenges. First, most common sensors for high-quality data require a series of several exposures in order to generate one single 3D view. During the recording of this series, the sensor and the object have to stand still. Paradigm for a large number of sensors is the “fringe projection method” [3

M. Halioua, H. Liu, and V. Srinivasan, “Automated phase-measuring profilometry of 3-D diffuse objects,” Appl. Opt. 23, 3105–3108 (1984). [CrossRef]

3D–Shape GmbH, “FaceScan3D,” http://www.3d-shape.com.

], which needs at least three exposures; commonly many more shots are taken. Again, during the acquisition of each single 3D view, an uncontrolled motion of the object relative to the sensor causes a complete failure of the measurement. Second, for a complete surface acquisition, single 3D views have to be captured from many different perspectives. For this purpose, the sensor needs to be repositioned various times. This stop-and-go can be an elaborate and time-consuming task. For complex surfaces, such as the “Trombone Angel” at the Dome of Bamberg (see Fig. 1(a)), commonly 3D views from several hundred positions have to be taken. Third, these 3D views need to be merged in order to obtain one complete 3D surface model. Only after registering all 3D views to each other—usually performed a posteriori—can missing surface parts be detected and measured. So, it took the authors of [1

X. Laboureux and G. Häusler, “Localization and registration of three-dimensional objects in space—where are the limits?,” Appl. Opt. 40, 5206–5216 (2001). [CrossRef]

] an entire month to acquire the data for a complete virtual copy of the “Trombone Angel” (see Fig. 1(b)).

Our goal is to overcome these severe drawbacks of common 3D data acquisition. The acquisition should be motion robust, and the sensor should be freely hand guided around the object [5

S. Ettl, O. Arold, P. Vogt, O. Hybl, Z. Yang, W. Xie, and G. Häusler, “Flying triangulation”: a motion-robust optical 3D sensor principle,” in Proceedings of Fringe 2009, The 6th International Workshop on Advanced Optical Metrology , W. Osten and M. Kujawinska, eds. (Springer, 2009), pp. 768–771.

] (see Fig. 1(c)). The registration of the single 3D views should not depend on external tracking. And last, visual feedback about the measurement progress should be provided in real time in order to enable an efficient and comfortable guidance of the sensor. To attain this goal, the following approach is pursued: combining simple technology with thorough optical design and complex algorithms—shifting the major part of the work load to the algorithm side.

The focus of this paper is twofold: first, the sensor principle and second, the optical design optimized to work at the physical limits. The latter aspect is important in order to achieve a measurement uncertainty and resolution that are not limited by technology or by algorithms but rather by physics. Further, an overview of the employed algorithms (such as registration) will be given. A detailed description of these algorithms will be published soon.

2. Related Work

First of all, a principal distinction between employing uncoded and encoded patterns has to be drawn. In general, encoding the pattern is an option to ensure a more robust pattern indexing in comparison to employing an uncoded pattern. An overview of different encoding methods is, for example, given in [6

J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004). [CrossRef]

]. However, pattern encoding requires bandwidth: For time encoding, several images need to be acquired in order to generate 3D data, as is the case for the well-established “fringe projection” method [3

M. Halioua, H. Liu, and V. Srinivasan, “Automated phase-measuring profilometry of 3-D diffuse objects,” Appl. Opt. 23, 3105–3108 (1984). [CrossRef]

3D–Shape GmbH, “FaceScan3D,” http://www.3d-shape.com.

]. For space encoding, several adjacent camera pixels have to be combined to form a pattern. Further, for direct encoding, either color or gray levels can be employed, requiring a certain signal strength for the encoding.

In more detail, for the fringe projection method, at least three camera images (two-dimensional (2D) raw images) are necessary to extract 3D information in each camera pixel [7

C. Wagner and G. Häusler, “Information theoretical optimization for optical range sensors,” Appl. Opt. 42, 5418–5426 (2003). [CrossRef]

]. We call the result of this type of acquisition “dense 3D data” in contrast to other sensor principles that generate only “sparse 3D data.” During the acquisition, the sensor and the object have to stand still. One realization for the intraoral measurement of teeth based on this principle is the CEREC scanner by Sirona Dental Systems, Inc. [8

Sirona Dental Systems, Inc., “CEREC scanner,” http://www.sirona.com.

]. However, this scanner conceptually lacks motion robustness due to the required acquisition time for generating 3D information.

To enable some sort of motion robustness for time-encoded patterns, the 2D raw images necessary to generate 3D data have to be acquired within a very short time window. One option is to speed up the above-described multishot approach by employing fast projection and acquisition technology (see, e.g., [9

G. Frankowski, M. Chen, and T. Huth, “Real-time 3D shape measurement with digital stripe projection by Texas Instruments Micromirror Devices (DMD),” Proc. SPIE 3958, 90–105 (2000). [CrossRef]

,10

M. Schaffer, M. Grosse, and R. Kowarschik, “High-speed pattern projection for three-dimensional shape measurement using laser speckles,” Appl. Opt. 49, 3622–3629 (2010). [CrossRef]

]). However, this is a rather costly and conceptually not perfect option.

What about generating 3D data from one single camera image? There are several options for such a single-shot approach. One well-designed method is described in [11

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph. 21, 438–446 (2002). [CrossRef]

]. The sensor is based on structured-light triangulation employing several line patterns. Time encoding is used to correctly index the line pattern, while 3D data is generated from each camera image. However, this method possesses a severe drawback: The sensor motion has to be restricted due to the indexing method: It is limited to a half-line distance per frame in the camera image [12

O. Hall-Holt and S. Rusinkiewicz, “Stripe boundary codes for real-time structured-light range scanning of moving objects,” in Proceedings of 8th IEEE International Conference on Computer Vision (IEEE, 2001), pp. 359–366.

]. Since motion robustness is important for our measurement tasks, this method is not sufficient for our purpose.

For space encoding, one real-time acquisition method is described in [13

C. Albitar, P. Graebling, and C. Doignon, “Robust structured light coding for 3D reconstruction,” in Proceedings of 11th IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–6.

]. The underlying principle is structured-light triangulation. It employs a so-called M-array approach (explained also in [6

J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004). [CrossRef]

]) based on a pattern consisting of some geometric primitives. The method is real-time capable; however, the lateral resolution is reduced in comparison to methods employing uncoded patterns: The boundaries of primitives of size

5 mm \times 5 mm

are determined, and the centers are estimated to generate (sparse) 3D points. Thus, small details cannot be detected. We aim for a lateral resolution close to the physical (diffraction) limit and a longitudinal resolution close to the speckle-noise limit [14

R. G. Dorsch, G. Häusler, and J. M. Herrmann, “Laser triangulation: fundamental uncertainty in distance measurement,” Appl. Opt. 33, 1306–1314 (1994). [CrossRef]

]. Since the lateral resolution of a triangulation principle is directly related to the longitudinal resolution (height resolution), whose minimization is crucial for us, this approach cannot be used for most of our purposes.

Theoretically optimal is a method that employs direct pattern encoding, more precise than the use of an additional modality such as color (see, e.g., [15

G. Häusler and D. Ritter, “Parallel three-dimensional sensing by color-coded triangulation,” Appl. Opt. 32, 7164–7169 (1993). [CrossRef]

L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in Proceedings of 1st IEEE International Symposium on 3D Data Processing, Visualization, and Transmission (IEEE, 2002), pp. 24–36.

–17

F. Forster, “A high-resolution and high accuracy real-time 3D sensor based on structured light,” in Proceedings of 3rd IEEE International Symposium on 3D Data Processing, Visualization, and Transmission (IEEE, 2006), pp. 208–215.

]), where a spectrum of colors is projected onto the object instead of noncolored fringes. But how sensitive is such a method to textured surfaces? The authors’ experience [15

G. Häusler and D. Ritter, “Parallel three-dimensional sensing by color-coded triangulation,” Appl. Opt. 32, 7164–7169 (1993). [CrossRef]

] is that an illumination with spectrally pure colors is insensitive to textured surfaces. Dense 3D data can be extracted from one single camera image. However, the efficiency of the observed light may be noticeably reduced. Illumination with spectrally nonpure colors (for example, a slide with printed color stripes) is more efficient, but sensitive to texture. Hence, the resulting precision may be noticeably reduced. Further, color-encoded illumination is energetically inefficient, because the light has to pass a narrow spectrometer slit. So, this type of sensor is used only for applications that do not require high precision or high lateral resolution.

For strongly band-limited (“smoothly varying”) surfaces, dense 3D data can be generated from one single, noncolored camera image [18

M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Appl. Opt. 22, 3977–3982 (1983). [CrossRef]

]. However, most of the objects of interest are not strongly band limited, and hence this approach cannot be used in many cases.

So far, we have discussed only sensor principles that actively project a pattern onto the object surface for feature detection. Of course, there also exist passive methods that directly detect features in the camera images. A well-known passive triangulation principle is “structure from motion” [19

E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, “Generic and real-time structure from motion using local bundle adjustment,” Image Vis. Comput. 27, 1178–1193 (2009). [CrossRef]

]. It enables a single-shot real-time acquisition of 3D data. The registration of such data is based on tracking feature points detected in each frame. The feature points need to be distinguishable, meaning that they can only be sparsely distributed. The 3D points are generated from these features, and hence only a small number, commonly in the order of some hundred points, is acquired over several frames. The density of the resulting point cloud remains insufficient for our applications.

From this overview, it can be seen that for a single-shot concept without additional modality, something has to be sacrificed. If any kind of object should be measurable, it appears that dense 3D data cannot be obtained anymore. Indeed, single-shot 3D sensors appear only possible with sparse 3D data. The paradigm is “light sectioning” [20

G. Häusler and W. Heckel, “Light sectioning with large depth and high resolution,” Appl. Opt. 27, 5165–5169 (1988). [CrossRef]

]. There is a commercialized prior art sensor that is based on light sectioning: the “Artec 3D Scanner” by the Artec Group [21

Artec Group, Inc., “Artec 3D scanner,” http://artec-group.com.

]. It enables a motion-robust measurement of objects. However, this sensor exploits spatially encoded patterns and displays limited resolution, as described above. In the next section, a solution is presented that enables a motion-robust and freely hand-guided 3D measurement of object surfaces with optimized precision and resolution.

3. Flying Triangulation

In this section, the new principle, which we call “flying triangulation,” is introduced. We will present the measurement principle, then give the specifications of the single-shot sensor, and last describe the registration of the acquired data.

An important property of flying triangulation is that the measurement principle is scalable. It permits the design of sensors having measurement volumes in the range from some millimeters up to meters, enabling the measurement of objects such as teeth inside a patient’s mouth, heads and sculptures, bodies, and entire rooms. In this paper we will present a sensor enabling an intraoral measurement of teeth. However, this will only serve as exemplary representation of the described measurement principle.

A. Measurement Principle

The sensor is based on the well-known light-sectioning principle. A line pattern is projected along a projection axis onto the surface under test and observed by a camera along an observation axis. The angle between these two axes is the triangulation angle

θ

(see Fig. 2(a)). From each camera image, a sparse 3D view (see Section 2) is generated. For registration reasons, we use a trick: Two line patterns—orthogonal to each other—are alternately projected onto the object surface, as shown in Fig. 2(b) and 2(c). For this purpose, the sensor is equipped with two perpendicularly aligned projection units as displayed in Fig. 3(a). The registration of the 3D views is described in Section 3.C.

Fig. 2. (a) The presented sensor is based on the well-established light-sectioning principle: A line pattern is projected onto the surface under test and observed by a camera from a different direction, via a triangulation angle

θ

. From the captured pattern, sparse profile data of the object can be extracted. (b) and (c) A key feature of the new single-shot sensor is the alternating projection of orthogonal patterns.

Fig. 3. (a) Schematic setup of the flying triangulation sensor consisting of two orthogonal projection units and one observation unit. (b) For each single 3D view, a measurement uncertainty of less than 30 μm is achieved within the entire measurement volume of

20 mm \times 15 mm \times 15 mm

To obtain metric data

(x, y, z)

from each camera image, a standard model-free calibration method for active triangulation sensors, e.g., as described in [22

K. Veit and G. Häusler, “Metrical calibration of a phase measuring triangulation sensor,” in Proceedings of Vision Modeling and Visualization , B. Girod, G. Greiner, H. Niemann, and H.-P. Seidel, eds. (Aka GmbH, 2000), pp. 33–38.

B. Girod, G. Greiner, and H. Niemann, Principles of 3D Image Analysis and Synthesis (Kluwer Academic Publishers, 2000), pp. 335–347.

–24

A. W. Koch, M. W. Ruprecht, O. Toedter, and G. Häusler, Optische Metechnik an technischen Oberflächen (Expert-Verlag, 1998), pp 113–139.

], is employed. The main idea of the method can be briefly described as follows: First, for relating the line shift to the depth information through the entire measurement volume, a longitudinal

z

-calibration is performed by observing the projected line pattern for a planar object placed at known

z

-positions through the entire measurement depth range. The lateral shift on the CCD chip of the observed pattern yields a depth

z

for pixel

(i, j)

via polynomial fit. This shift is of course dependent on the triangulation angle. To ensure a correct line referencing, the distance between neighboring lines is chosen so that each line has its own “region of uniqueness,” meaning that its lateral shift on the CCD chip will not interfere with the pixel range of adjacent line shifts. Second, in order to correct for distortion errors, a lateral

x - y

-calibration is performed by observing a marker plate of known marker positions placed at known

z

-positions. Mapping the observed marker positions onto the correct positions via polynomial fit yields an

(x, y)

-pair for

(i, j)

. The above-introduced regions of uniqueness for the lines imply that the later range measurement is restricted in depth in a way that the line indexing is only unique when measuring within the measurement volume. This is not too restrictive if the measurement situation is controllable.

The work flow of flying triangulation is as follows (see Fig. 4): The sensor is hand guided around the object while continuously capturing a “movie” of camera images. After applying the calibration information, each image delivers a sparse 3D view, which is registered automatically “on the fly” to the already acquired 3D data. The current result is visualized in real time, providing the user with feedback to detect yet unmeasured object parts. Eventually, commonly after some seconds, a dense 3D point cloud of the object’s topography is obtained.

Fig. 4. Measurement principle of flying triangulation: Measuring an object by moving the sensor or the object yields a series of single 3D views. These sparse 3D views are registered to each other in real time, yielding a dense 3D model of the object.

B. Single-Shot Sensor

One challenge for the sensor is to achieve the best possible measurement uncertainty in each single 3D view. The major source of noise of an active triangulation sensor is speckle noise. The physically achievable measurement uncertainty

δ z

is given by [14

R. G. Dorsch, G. Häusler, and J. M. Herrmann, “Laser triangulation: fundamental uncertainty in distance measurement,” Appl. Opt. 33, 1306–1314 (1994). [CrossRef]

]

δ z = \frac{C}{2 π} \frac{λ}{\sin u_{obs} \sin θ},

(1)

where

C

is the speckle contrast,

λ

is the wave length,

\sin u_{obs}

is the observation aperture, and

θ

is the triangulation angle.

We will now describe the optimal choice of these parameters exemplary for a sensor designed to enable an intraoral acquisition of teeth. The captured data have to fulfill certain requirements: In order to avoid an intrusion of bacteria, the measurement uncertainty of the gap between the prepared tooth and the prospective crown has to be smaller than 50 μm. The triangulation angle should not exceed 7° to prevent shadowing effects, e.g., between neighboring teeth. Further, the measurement volume should contain at least one complete tooth, hence being not smaller than

20 mm \times 15 mm \times 15 mm

Certain aspects for the choice of the parameters have to be considered: First, since height information of object points within a certain measurement volume is desired, the depth of field of both the projection and the observation has to be taken into account. Unfortunately, increasing the depth of field of our diffraction-limited system—by choosing smaller apertures according to Rayleigh [25

D. Malacara, ed., Geometrical and Instrumental Optics (Academic Press, 1988) p. 167.

]

δ z_{R} = \frac{λ}{2 n \sin^{2} u},

(2)

where

n

is the refraction index—will also increase the measurement uncertainty given in Eq. (1). Second, a triangulation angle as small as possible has to be used in order to minimize shadowing effects. Third, a high observation aperture is required for energetic reasons. As described above, motion robustness requires short exposure times.

These contradictory requirements are handled by thorough design of the components: The speckle contrast

C

in Eq. (1) is minimized: An extended white-light source is used, which fills the full pupil of the projection lens. The projection aperture is chosen as large as possible and as small as necessary to achieve a measurement uncertainty below 30 μm within the entire measurement volume of

20 mm \times 15 mm \times 15 mm

for an intraoral sensor (see Fig. 3(b)). However, since the apertures and the triangulation angle cannot be chosen arbitrarily large in order to reduce the speckle noise, the key component to address is the speckle contrast

C

given by the product [26

G. Häusler, “Ubiquitous coherence—boon and bale of the optical metrologist,” Proc. SPIE 4933, 48–52 (2003). [CrossRef]

,27

C. Wagner, “Informationstheoretische Grenzen optischer 3D-Sensoren,” dissertation (University Erlangen-Nuremberg, 2003).

]

C = C_{spat} \cdot C_{temp} \cdot C_{pix} \cdot C_{pol},

(3)

where

C_{spat}

describes in Eq. (3) the component of the speckle contrast resulting from the spatial coherence, with

C_{spat} = \min (1, \frac{\sin u_{obs}}{\sin u_{proj}}),

(4)

where

\sin u_{proj}

is the projection aperture and

C_{temp}

is the component of

C

in Eq. (3) resulting from the temporal coherence, with

C_{temp} = \frac{1}{{(1 + {(\frac{4 σ_{h}}{l_{c}})}^{2})}^{1 / 4}},

(5)

where

σ_{h}

is the surface roughness and

l_{c}

is the coherence length of the light source.

C_{pix}

is the component of the speckle contrast in Eq. (3), taking into account the pixel size relative to the speckle size: When speckle patterns are observed by a camera, several speckles might be averaged within a single camera pixel, hence reducing the contrast.

C_{pol}

is the component of

C

resulting from polarization: The employment of nonpolarized light sometimes reduces the contrast

C_{pol}

, depending on the surface properties.

The observation aperture is chosen to be smaller than the projection aperture in order to create some spatial incoherence, useful to reduce the speckle contrast. Since teeth are volume scatterers, a measurement of pure teeth would lead to an error of some 100 μm exceeding the above mentioned specifications. Therefore, we spray the teeth with titanium dioxide having a thickness of about 20 μm. Since titanium dioxide is also a volume scatterer with a surface roughness larger than the coherence length below 3 μm of the white-light source, the speckle noise in the observed camera image is not significantly increased. By these measures,

C

could be scaled down by almost a factor of 10 compared to a laser light source. Hence, by exploiting the limits of physics and technology, the entire optical sensor setup could be optimized.

C. Registration

In order to obtain a dense 3D model of the captured object, a method is required that robustly registers sparse 3D views. In a first step, a coarse registration is employed to roughly align two data sets to each other. In a second step, a fine registration is applied to improve the accuracy. Most existing coarse registration methods are based on detecting common surface features that are commonly curvature related and rely on neighboring 3D points (in

x

- and

y

-direction). These features are then detected in both data sets and aligned to each other. This can only be done if the underlying 3D views are dense [28

N. Schön and G. Häusler, “Automatic coarse registration of three-dimensional surfaces by information theoretic selection of salient points,” Appl. Opt. 45, 6539–6550 (2006). [CrossRef]

T. Maier and G. Häusler, “Segmentation based fast registration of free form surfaces in the Euclidean space,” in Proceedings of Vision Modeling and Visualization , L. Kobbelt, T. Kuhlen, T. Aach, and R. Westermann, eds. (Aka GmbH, 2006), pp. 17–24.

–30

J. Kaminski, M. Struck, T. Maier, S. Ettl, and G. Häusler, “Robust automatic coarse registration of specular free-form surfaces,” in Vol. 108 of DGaO Proceedings (2007).

For flying triangulation, a new registration method had to be developed, since the sparse 3D data do not provide sufficient surface information. Our method is based on detecting common 3D points in subsequent 3D views. Each view provides only 3D data along the projected lines. As the sensor is moved during the measurement, consecutive views generally would have no common 3D points as depicted in Fig. 5(a). We apply a trick to guarantee common 3D points: Two perpendicular line patterns are alternately projected onto the surface (see Fig. 5(b)).

Fig. 5. (a) Employing only one light pattern of lines generally yields no common 3D points in consecutive 3D views. (b) Using two alternating orthogonal line patterns guarantees the existence of common 3D points (circles). (c) Basic idea of the coarse registration: finding the best position of a table with

N

legs on a hilly landscape. The arrow represents the remaining gap. (d) From the already registered views at times

T_{n}

, the sensor path can be reconstructed in order to detect registration outliers and to estimate transformations of subsequent 3D views at later times.

The main task of the registration is to find the common 3D points in subsequent 3D views and to map them onto each other. The basic concept of the coarse registration can be illustrated as the procedure of finding the most stable position of a “multilegged” table on a hilly landscape, as depicted in Fig. 5(c). The distance of the table legs to the heaps yields a measure of the goodness of the registration. The employed fine registration method is based on an iterative-closest-point algorithm (see, for example, [31

P. Besl and N. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14, 239–256 (1992). [CrossRef]

]) specifically adapted to the sparseness of the data. More details will be presented in a separate paper.

In order to obtain a dense 3D data set of the object, some hundred 3D views have to be acquired. However, it cannot be avoided that the registration error accumulates along the constituent registration steps. In order to reduce this error, we employ two further tricks: First, registration outliers are detected and corrected by reconstructing the sensor path from the already determined transformation parameters (see Fig. 5(d)). Here, prior knowledge is used, assuming this path to be continuous. Second, several overlapping 3D views are registered in one step to each other. By time-optimized algorithms and proper software architecture, the registration is quite fast, and it allows a real-time visualization to help the user to efficiently capture the object.

4. Results

A. Sensor Realization

As emphasized above, the measurement principle is scalable and permits us to measure a wide range of objects. In this section, the exemplary operation of one of the implemented sensors is demonstrated (see Fig. 6(a)), which enables an intraoral measurement of teeth, e.g., to facilitate the manufacturing of dental crowns. The sensor specifications are the same as described in Section 3.B. However, we demonstrate in Fig. 7(b) that the same sensor principle can also be employed to measure objects of larger scales, such as human faces.

Fig. 6. (a) The miniaturized sensor with a line pattern projected onto a dental cast (Media 1). (b) The object under test is a dental cast. (c) Before registration, the series of 3D views is located almost at the same position. (d) After registration, a dense model of the acquired part is obtained.

Fig. 7. (a) The measured 3D data of the entire dental cast. (b) The measured 3D data of a human head. (c) The 3D model of a dental jaw used for an investigation of the registration error (the chain of dots marks the sensor path chosen for the simulation of the 3D data acquisition). (d) The registered 3D data of the teeth acquired by simulation.

Figure 3(b) displays the achieved measurement uncertainty on a flat mirror. The mirror is sprayed with titanium dioxide in order to obtain results comparable to real measurement results. The generated 3D data is smoothed by using a moving-average filter. The noise is obtained by subtracting the smoothed data from the original data. For single 3D views, the statistical measurement uncertainty is below 30 μm within the entire measurement volume.

The sensor acquires 30 camera images per second, each with an exposure time of 15 ms. As a first example, a freely hand-guided measurement of a part of a dental cast is presented. Within 17 seconds, a set of 500 single 3D views is acquired, resulting in about 3.5 million 3D points (see Fig. 6(b)). The unregistered 3D views are depicted in Fig. 6(c). Applying the registration method described in Section 3.C yields a dense point cloud of the acquired part of the dental cast (see Fig. 6(d)). As a second measurement example, an entire dental cast is acquired. For this experiment, 600 single 3D views of the dental cast depicted in Fig. 6(b) are measured in 20 seconds, resulting in about 4.2 million 3D points. The registered result is shown in Fig. 7(a).

4. Registration Error

As can be seen from these experiments, the new sensor acquires an entire “movie,” usually consisting of several hundred camera images, in order to entirely capture an object. We will conclude with an investigation of the error resulting from the registration of data from a sequence of camera images of typical length. The data of each 3D view have a statistical measurement uncertainty below 30 μm (as described above and depicted in Fig. 3(b)). One has to distinguish two further error sources: the registration error and a systematic error caused by calibration imperfections. The paper will conclude with an investigation of the registration error. This is a crucial error of our measurement principle because of the possibility of an error propagation, in contrast to other common principles that acquire dense data.

If the sensor were calibrated perfectly, no systematic error would accumulate, and the registration error would originate only from the statistical measurement uncertainty. In order to investigate this error, the following simulation procedure is performed (see Fig. 8): A perfect 3D model

M_{p}

of a dental cast serves as master data set. From this master, a series of perfect 3D views

{s}

is obtained by performing a computer simulation of the proposed sensor (see Fig. 7(c)).

Fig. 8. Flow diagram for the investigation of the registration error of flying triangulation via simulation procedure.

The registration error of such noise-free virtual data is almost zero up to errors caused by the discrete nature of the underlying data. Hence, statistical noise extracted from real measurements is added to each simulated 3D view

{s + n}

. Why not use simulated noise such as white noise? As it is difficult to simulate data that reflects the speckle nature of the noise observed in measured data, the noise is extracted from real measurements in the manner described in Section 4.A.

After registration of these 3D views

{(s + n)}^{reg}

, a noisy 3D model

M_{i}^{noisy}

is obtained (see Fig. 7(d)). Subtracting the known, separable statistical noise from this 3D model results in a noise-free 3D model

M_{i}

, which is imperfect due to registration errors. Table 1 displays some results: If the noise-free imperfect 3D model

M_{i}

is compared to the original model

M_{p}

, the mean pure registration error, defined as the mean of the pointwise Euclidean distances of the two datasets, amounts to only 10 μm. This shows that the error caused by registration of 3D data containing statistical noise is significantly smaller than the statistical measurement uncertainty. This can be explained by the fact that the registration is performed on several hundred 3D points at the same time, thus averaging over the statistical noise on the 3D data. And it proves that the realized sensor based on the proposed measurement principle functions well, even when capturing the 3D topography of an object by acquiring hundreds of sparse 3D views.

Table 1. Statistical Results for Registration Method Applied to Simulated 3D Views With Real Noise

Measurement field	2 teeth, 30 mm
Acquired single 3D views	330
Acquired 3D points	2.4 millions
Mean pure registration error (without noise)	10 μm

5. Conclusion

The presented novel measurement principle “flying triangulation” enables a motion-robust and comfortable acquisition of virtually any kind of object. An embodiment of flying triangulation is shown, which enables an intraoral measurement of teeth. The sensor works at its physical limits, achieving a statistical measurement uncertainty of below 30 μm (based on single 3D views) within the entire measurement volume of

20 mm \times 15 mm \times 15 mm

. Measurements and simulations demonstrate the successful operation of the developed sensor. Data acquisition and registration perform in real time and enable an instantaneous display of the current measurement progress. The sensor has high precision and can be manufactured with low cost. Flying triangulation is scalable and allows sensors for small objects such as teeth, as well as for large objects such as cars or entire rooms.

References

1.	X. Laboureux and G. Häusler, “Localization and registration of three-dimensional objects in space—where are the limits?,” Appl. Opt. 40, 5206–5216 (2001). [CrossRef]
2.	M. Benz, J. Hartmann, T. Maier, E. Nkenke, K. Veit, A. Stellzig-Eisenhauer, F. W. Neukam, and G. Häusler, “Optical 3d-metrology for medical applications,” in Biomedizinische Technik. Proc. ICMP 2005 and BMT 2005 , U. Boenick, A. Bolz, W. Kalender, E. G. Hahn, and A. M. Schulte, eds. (Schiele und Schön, 2005), pp. 48–49.
3.	M. Halioua, H. Liu, and V. Srinivasan, “Automated phase-measuring profilometry of 3-D diffuse objects,” Appl. Opt. 23, 3105–3108 (1984). [CrossRef]
4.	3D–Shape GmbH, “FaceScan3D,” http://www.3d-shape.com.
5.	S. Ettl, O. Arold, P. Vogt, O. Hybl, Z. Yang, W. Xie, and G. Häusler, “Flying triangulation”: a motion-robust optical 3D sensor principle,” in Proceedings of Fringe 2009, The 6th International Workshop on Advanced Optical Metrology , W. Osten and M. Kujawinska, eds. (Springer, 2009), pp. 768–771.
6.	J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004). [CrossRef]
7.	C. Wagner and G. Häusler, “Information theoretical optimization for optical range sensors,” Appl. Opt. 42, 5418–5426 (2003). [CrossRef]
8.	Sirona Dental Systems, Inc., “CEREC scanner,” http://www.sirona.com.
9.	G. Frankowski, M. Chen, and T. Huth, “Real-time 3D shape measurement with digital stripe projection by Texas Instruments Micromirror Devices (DMD),” Proc. SPIE 3958, 90–105 (2000). [CrossRef]
10.	M. Schaffer, M. Grosse, and R. Kowarschik, “High-speed pattern projection for three-dimensional shape measurement using laser speckles,” Appl. Opt. 49, 3622–3629 (2010). [CrossRef]
11.	S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph. 21, 438–446 (2002). [CrossRef]
12.	O. Hall-Holt and S. Rusinkiewicz, “Stripe boundary codes for real-time structured-light range scanning of moving objects,” in Proceedings of 8th IEEE International Conference on Computer Vision (IEEE, 2001), pp. 359–366.
13.	C. Albitar, P. Graebling, and C. Doignon, “Robust structured light coding for 3D reconstruction,” in Proceedings of 11th IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–6.
14.	R. G. Dorsch, G. Häusler, and J. M. Herrmann, “Laser triangulation: fundamental uncertainty in distance measurement,” Appl. Opt. 33, 1306–1314 (1994). [CrossRef]
15.	G. Häusler and D. Ritter, “Parallel three-dimensional sensing by color-coded triangulation,” Appl. Opt. 32, 7164–7169 (1993). [CrossRef]
16.	L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in Proceedings of 1st IEEE International Symposium on 3D Data Processing, Visualization, and Transmission (IEEE, 2002), pp. 24–36.
17.	F. Forster, “A high-resolution and high accuracy real-time 3D sensor based on structured light,” in Proceedings of 3rd IEEE International Symposium on 3D Data Processing, Visualization, and Transmission (IEEE, 2006), pp. 208–215.
18.	M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Appl. Opt. 22, 3977–3982 (1983). [CrossRef]
19.	E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, “Generic and real-time structure from motion using local bundle adjustment,” Image Vis. Comput. 27, 1178–1193 (2009). [CrossRef]
20.	G. Häusler and W. Heckel, “Light sectioning with large depth and high resolution,” Appl. Opt. 27, 5165–5169 (1988). [CrossRef]
21.	Artec Group, Inc., “Artec 3D scanner,” http://artec-group.com.
22.	K. Veit and G. Häusler, “Metrical calibration of a phase measuring triangulation sensor,” in Proceedings of Vision Modeling and Visualization , B. Girod, G. Greiner, H. Niemann, and H.-P. Seidel, eds. (Aka GmbH, 2000), pp. 33–38.
23.	B. Girod, G. Greiner, and H. Niemann, Principles of 3D Image Analysis and Synthesis (Kluwer Academic Publishers, 2000), pp. 335–347.
24.	A. W. Koch, M. W. Ruprecht, O. Toedter, and G. Häusler, Optische Metechnik an technischen Oberflächen (Expert-Verlag, 1998), pp 113–139.
25.	D. Malacara, ed., Geometrical and Instrumental Optics (Academic Press, 1988) p. 167.
26.	G. Häusler, “Ubiquitous coherence—boon and bale of the optical metrologist,” Proc. SPIE 4933, 48–52 (2003). [CrossRef]
27.	C. Wagner, “Informationstheoretische Grenzen optischer 3D-Sensoren,” dissertation (University Erlangen-Nuremberg, 2003).
28.	N. Schön and G. Häusler, “Automatic coarse registration of three-dimensional surfaces by information theoretic selection of salient points,” Appl. Opt. 45, 6539–6550 (2006). [CrossRef]
29.	T. Maier and G. Häusler, “Segmentation based fast registration of free form surfaces in the Euclidean space,” in Proceedings of Vision Modeling and Visualization , L. Kobbelt, T. Kuhlen, T. Aach, and R. Westermann, eds. (Aka GmbH, 2006), pp. 17–24.
30.	J. Kaminski, M. Struck, T. Maier, S. Ettl, and G. Häusler, “Robust automatic coarse registration of specular free-form surfaces,” in Vol. 108 of DGaO Proceedings (2007).
31.	P. Besl and N. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14, 239–256 (1992). [CrossRef]

OCIS Codes
(100.6890) Image processing : Three-dimensional image processing
(110.0110) Imaging systems : Imaging systems
(110.6880) Imaging systems : Three-dimensional image acquisition
(120.0120) Instrumentation, measurement, and metrology : Instrumentation, measurement, and metrology

ToC Category:
Imaging Systems

History
Original Manuscript: July 12, 2011
Revised Manuscript: October 6, 2011
Manuscript Accepted: October 6, 2011
Published: January 9, 2012

Citation
Svenja Ettl, Oliver Arold, Zheng Yang, and Gerd Häusler, "Flying triangulation—an optical 3D sensor for the motion-robust acquisition of complex objects," Appl. Opt. 51, 281-289 (2012)
http://www.opticsinfobase.org/ao/abstract.cfm?URI=ao-51-2-281

Sort: Author | Year | Journal | Reset

References

X. Laboureux and G. Häusler, “Localization and registration of three-dimensional objects in space—where are the limits?,” Appl. Opt. 40, 5206–5216 (2001). [CrossRef]
M. Benz, J. Hartmann, T. Maier, E. Nkenke, K. Veit, A. Stellzig-Eisenhauer, F. W. Neukam, and G. Häusler, “Optical 3d-metrology for medical applications,” in Biomedizinische Technik. Proc. ICMP 2005 and BMT 2005, U. Boenick, A. Bolz, W. Kalender, E. G. Hahn, and A. M. Schulte, eds. (Schiele und Schön, 2005), pp. 48–49.
M. Halioua, H. Liu, and V. Srinivasan, “Automated phase-measuring profilometry of 3-D diffuse objects,” Appl. Opt. 23, 3105–3108 (1984). [CrossRef]
3D–Shape GmbH, “FaceScan3D,” http://www.3d-shape.com.
S. Ettl, O. Arold, P. Vogt, O. Hybl, Z. Yang, W. Xie, and G. Häusler, “Flying triangulation”: a motion-robust optical 3D sensor principle,” in Proceedings of Fringe 2009, The 6th International Workshop on Advanced Optical Metrology, W. Osten and M. Kujawinska, eds. (Springer, 2009), pp. 768–771.
J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004). [CrossRef]
C. Wagner, and G. Häusler, “Information theoretical optimization for optical range sensors,” Appl. Opt. 42, 5418–5426 (2003). [CrossRef]
Sirona Dental Systems, Inc., “CEREC scanner,” http://www.sirona.com.
G. Frankowski, M. Chen, and T. Huth, “Real-time 3D shape measurement with digital stripe projection by Texas Instruments Micromirror Devices (DMD),” Proc. SPIE 3958, 90–105(2000). [CrossRef]
M. Schaffer, M. Grosse, and R. Kowarschik, “High-speed pattern projection for three-dimensional shape measurement using laser speckles,” Appl. Opt. 49, 3622–3629 (2010). [CrossRef]
S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph. 21, 438–446 (2002). [CrossRef]
O. Hall-Holt and S. Rusinkiewicz, “Stripe boundary codes for real-time structured-light range scanning of moving objects,” in Proceedings of 8th IEEE International Conference on Computer Vision (IEEE, 2001), pp. 359–366.
C. Albitar, P. Graebling, and C. Doignon, “Robust structured light coding for 3D reconstruction,” in Proceedings of 11th IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–6.
R. G. Dorsch, G. Häusler, and J. M. Herrmann, “Laser triangulation: fundamental uncertainty in distance measurement,” Appl. Opt. 33, 1306–1314 (1994). [CrossRef]
G. Häusler and D. Ritter, “Parallel three-dimensional sensing by color-coded triangulation,” Appl. Opt. 32, 7164–7169 (1993). [CrossRef]
L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in Proceedings of 1st IEEE International Symposium on 3D Data Processing, Visualization, and Transmission (IEEE, 2002), pp. 24–36.
F. Forster, “A high-resolution and high accuracy real-time 3D sensor based on structured light,” in Proceedings of 3rd IEEE International Symposium on 3D Data Processing, Visualization, and Transmission (IEEE, 2006), pp. 208–215.
M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Appl. Opt. 22, 3977–3982 (1983). [CrossRef]
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, “Generic and real-time structure from motion using local bundle adjustment,” Image Vis. Comput. 27, 1178–1193(2009). [CrossRef]
G. Häusler and W. Heckel, “Light sectioning with large depth and high resolution,” Appl. Opt. 27, 5165–5169 (1988). [CrossRef]
Artec Group, Inc., “Artec 3D scanner,” http://artec-group.com.
K. Veit and G. Häusler, “Metrical calibration of a phase measuring triangulation sensor,” in Proceedings of Vision Modeling and Visualization, B. Girod, G. Greiner, H. Niemann, and H.-P. Seidel, eds. (Aka GmbH, 2000), pp. 33–38.
B. Girod, G. Greiner, and H. Niemann, Principles of 3D Image Analysis and Synthesis (Kluwer Academic Publishers, 2000), pp. 335–347.
A. W. Koch, M. W. Ruprecht, O. Toedter, and G. Häusler, Optische Metechnik an technischen Oberflächen (Expert-Verlag, 1998), pp 113–139.
D. Malacara, ed., Geometrical and Instrumental Optics(Academic Press, 1988) p. 167.
G. Häusler, “Ubiquitous coherence—boon and bale of the optical metrologist,” Proc. SPIE 4933, 48–52 (2003). [CrossRef]
C. Wagner, “Informationstheoretische Grenzen optischer 3D-Sensoren,” dissertation (University Erlangen-Nuremberg, 2003).
N. Schön and G. Häusler, “Automatic coarse registration of three-dimensional surfaces by information theoretic selection of salient points,” Appl. Opt. 45, 6539–6550 (2006). [CrossRef]
T. Maier and G. Häusler, “Segmentation based fast registration of free form surfaces in the Euclidean space,” in Proceedings of Vision Modeling and Visualization, L. Kobbelt, T. Kuhlen, T. Aach, and R. Westermann, eds. (Aka GmbH, 2006), pp. 17–24.
J. Kaminski, M. Struck, T. Maier, S. Ettl, and G. Häusler, “Robust automatic coarse registration of specular free-form surfaces,” in Vol. 108 of DGaO Proceedings (2007).
P. Besl and N. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14, 239–256 (1992). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Click here to see a list of articles that cite this paper