OSA's Digital Library

Optics Express

Optics Express

  • Editor: Andrew M. Weiner
  • Vol. 21, Iss. 5 — Mar. 11, 2013
  • pp: 5974–5987
« Show journal navigation

Dark-field microscopic image stitching method for surface defects evaluation of large fine optics

Dong Liu, Shitong Wang, Pin Cao, Lu Li, Zhongtao Cheng, Xin Gao, and Yongying Yang  »View Author Affiliations


Optics Express, Vol. 21, Issue 5, pp. 5974-5987 (2013)
http://dx.doi.org/10.1364/OE.21.005974


View Full Text Article

Acrobat PDF (1414 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

One of the challenges in surface defects evaluation of large fine optics is to detect defects of microns on surfaces of tens or hundreds of millimeters. Sub-aperture scanning and stitching is considered to be a practical and efficient method. But since there are usually few defects on the large aperture fine optics, resulting in no defects or only one run-through line feature in many sub-aperture images, traditional stitching methods encounter with mismatch problem. In this paper, a feature-based multi-cycle image stitching algorithm is proposed to solve the problem. The overlapping areas of sub-apertures are categorized based on the features they contain. Different types of overlapping areas are then stitched in different cycles with different methods. The stitching trace is changed to follow the one that determined by the features. The whole stitching procedure is a region-growing like process. Sub-aperture blocks grow bigger after each cycle and finally the full aperture image is obtained. Comparison experiment shows that the proposed method is very suitable to stitch sub-apertures that very few feature information exists in the overlapping areas and can stitch the dark-field microscopic sub-aperture images very well.

© 2013 OSA

1. Introduction

The demand for high quality optics has increased to satisfy requirements for modern optical systems, and quality control becomes one of the most important subjects in the optical manufacturing industry. Surface defects, such as small scratches and digs, are one area of optical quality that has become more important. While such defects do not affect imaging systems much (unless they are located in a conjugate plane), they can lead to significant damage in high-power laser and short-wave optical systems. For example, small defects can lead to dramatic damage in large-aperture optics in the ICF (Inertial Confinement Fusion) system, including catastrophic failure after repeated laser shots [1

1. S. Gomez, K. Hale, J. Burrows, and B. Griffiths, “Measurements of surface defects on optical components,” Meas. Sci. Technol. 9(4), 607–616 (1998). [CrossRef]

3

3. M. Gebhardt, H. Truckenbrodt, and B. Harnisch, “Surface defect detection and classification with light scattering,” Proc. SPIE 1500, 135–143 (1991). [CrossRef]

].

It is considered challenging to evaluate the surface defects of high quality optical components because the defects are usually of microns while the test samples are of hundreds of millimeters. AFM (Atomic Force Microscope) and SEM (Scanning Electron Microscope) are used to detect and characterize the surface defects on fine optics [1

1. S. Gomez, K. Hale, J. Burrows, and B. Griffiths, “Measurements of surface defects on optical components,” Meas. Sci. Technol. 9(4), 607–616 (1998). [CrossRef]

, 2

2. H. Ota, M. Hachiya, Y. Ichiyasu, and T. Kurenuma, “Scanning surface inspection system with defect-review SEM and analysis system solutions,” Hatachi Review 55, 78–82 (2006).

]. These methods are suitable for measuring defects within small areas of the samples. However, if the test area is hundreds of millimeters or larger, a huge amount of data will be obtained and the time to acquire and process this data becomes impractical. Scattering energy distribution of surface defects is studied to determine the defects area and even perform defects classification [3

3. M. Gebhardt, H. Truckenbrodt, and B. Harnisch, “Surface defect detection and classification with light scattering,” Proc. SPIE 1500, 135–143 (1991). [CrossRef]

]. The DAMOCLES system developed by LLNL (Lawrence Livermore National Lab) employs back and edge illuminations, and a mega-pixel CCD(charged-coupled device) to simultaneously get both the surface defects and sub-surface damages of fine optics of up to 1 meter size [4

4. F. Rainer, R. K. Dickson, R. T. Jennings, J. F. Kimmons, S. M. Maricle, R. P. Mouser, S. Schwartz, and C. L. Weinzapfel, “Development of practical damage mapping and inspection systems,” Proc. SPIE 3492, 556–563 (1999). [CrossRef]

].

A dark-field microscopic scattering imaging system was proposed to detect and evaluate surface defects of large fine optics [5

5. Y. Yang, C. Lu, J. Liang, D. Liu, L. Yang, and R. Li, “Microscopic dark-field scattering imaging and digitalization evaluation system of defects on optical devices precision surface,” Guangxue Xuebao/Acta Opt. Sin. 27, 1031–1038 (2007).

7

7. D. Sun, Y. Yang, F. Wang, L. Yang, and R. Li, “Microscopic scattering imaging system of defects on ultra-smooth surface suitable for digital image processing,” H. Xun, Y. Jiahu, C. W. James, W. Hexin, and H. Sen, eds. (SPIE, 2006), p. 615012.

]. The test sample is illuminated by circular white light source. The scattering light of the defect is captured by the CCD detector through a microscopic imaging system and dark-field image [8

8. N. Wei, J. You, K. Friehs, E. Flaschel, and T. W. Nattkemper, “In situ dark field microscopy for on-line monitoring of yeast cultures,” Biotechnol. Lett. 29(3), 373–378 (2007). [CrossRef] [PubMed]

, 9

9. K. Rebner, M. Schmitz, B. Boldrini, A. Kienle, D. Oelkrug, and R. W. Kessler, “Dark-field scattering microscopy for spectral characterization of polystyrene aggregates,” Opt. Express 18(3), 3116–3127 (2010). [CrossRef] [PubMed]

] of the defect is recorded. Sub-aperture scanning and stitching techniques are employed to obtain the full image of the test sample. As a result, large optical surfaces of hundreds of millimeters can be examined with a resolution of submicron. The mapping, classification and ranking of surface defects are achieved through processing of the full image. It takes about 60 minutes to evaluate an optical component of 800mm × 450mm.

The original system works well except that sub-aperture stitching encounters problems in some special conditions. Since the test optics are usually of high quality, there is very little characteristic information of the defects in the overlapping areas. In the original stitching method, template matching [6

6. D. Liu, Y. Y. Yang, L. Wang, Y. M. Zhuo, C. H. Lu, L. M. Yang, and R. J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun. 278(2), 240–246 (2007). [CrossRef]

, 10

10. S. Muramatsu, Y. Kobayashi, K. Takahashi, and E. Shimizu, “Development of template matching hardware and its high-speed processing strategy,” Electron. Commun. Jpn. Part III Fundam. Electron. Sci. 84(11), 1–10 (2001). [CrossRef]

] is employed to deal with defect features in the overlapping area of two adjacent sub-aperture images. For overlapping areas without defect features, stitching will be performed with nominal position calculated from resolution of CCD, size of the field of view (FOV), and stepping length of the translation stage, etc. For those sub-apertures that have only one run-through line feature in the overlapping area, the stitching process may fail because it is very difficult to find an accurate template. Also, as the stitching process follows the same order as sub-aperture scanning, stitching error will be yielded by the accumulated positioning error of the translation stage.

In this paper, a feature-based dark-field microscopic image stitching method is proposed to solve the problem above. In this method, the overlapping areas are firstly categorized by the features they contain: template-friendly feature, only one run-through line feature, or no feature. The sub-aperture images with template-friendly feature in the overlapping areas are stitched first, producing some large blocks of sub-apertures; then the sub-apertures with only one run-through line feature are treated with the guidance of their nominal positions; finally, those with no features are stitched following the nominal positions. Experiments show that the method proposed in this paper could effectively stitch the dark-field microscopic sub-aperture images obtained in the defect detection system, and is very suitable to stitch those with very few features all over the aperture.

This paper is constructed as follows: a simple description of the dark-field microscopic scattering imaging system for surface defects evaluation and the current problems with the original stitching method will be presented in Section 2; Section 3 gives a detailed illustration of the new proposed dark-field microscopic image stitching method; comparison experiments with the original and new proposed stitching method are given in Section 4; a conclusion of this paper will be summarized in Section 5.

2. Dark-field microscopic imaging system for surface defects evaluation

2.1 System layout

The dark-field microscopic imaging system for surface defects evaluation of large fine optical components is described in detail in Ref [6

6. D. Liu, Y. Y. Yang, L. Wang, Y. M. Zhuo, C. H. Lu, L. M. Yang, and R. J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun. 278(2), 240–246 (2007). [CrossRef]

]. A brief summary description is presented here.

Figure 1
Fig. 1 Principle of the microscopic scattering imaging system for surface defects evaluation, (a) system layout, and (b) collection process of the sub-aperture images.
illustrates the principle of the dark-field microscopic imaging system for surface defects evaluation, where Fig. 1(a) is the system layout, and Fig. 1(b) is the collection process of the sub-aperture images. As is shown in Fig. 1(a), the test sample, which can be as large as 800mm × 450mm, is illuminated by circular spaced white LED (Light Emitting Diode) light source. The scattering light of the surface defect is captured by a digital microscope (DM), thus the dark-field microscopic image of the defect is obtained. The view field of the DM is about 3.5mm × 2.5mm (@1 × magnification). To evaluate a sample of up to hundreds of millimeters, the DM is controlled by an X-Y translation stage to collect sub-aperture images. As is shown in Fig. 1(b), the sub-aperture images are collected following an “S” shape trace, with an overlapping area between every two adjacent sub-apertures. Sub-aperture stitching is employed to obtain the full aperture image of the test sample. As a result, large optical surfaces of hundreds of millimeters can be examined with a resolution of submicron. Currently, it takes about 30 minutes to collect the more than 60,000 sub-aperture images and another 30 minutes to stitch and produce evaluation report of the surface defects for a fine optical component of 800mm × 450mm (Matlab R2011b, 64 bit Windows 7 operating system, Intel Xeon 2.13 Ghz X2 processor, 32 GB DDR3 1333 MHz memory).

2.2 Mismatch of sub-stitching

A perfect translation stage will not have positioning error, and therefore every sub-aperture image could be directly placed at its nominal position in the full aperture image of the test sample. A practical translation stage, however, has positioning error, so sub-aperture stitching should be employed to compensate for this. In this system, the stitching process follows the same order as sub-aperture scanning. For algorithmic simplicity and higher degree of flexibility to the change of recognition target classes, a template matching method [6

6. D. Liu, Y. Y. Yang, L. Wang, Y. M. Zhuo, C. H. Lu, L. M. Yang, and R. J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun. 278(2), 240–246 (2007). [CrossRef]

, 10

10. S. Muramatsu, Y. Kobayashi, K. Takahashi, and E. Shimizu, “Development of template matching hardware and its high-speed processing strategy,” Electron. Commun. Jpn. Part III Fundam. Electron. Sci. 84(11), 1–10 (2001). [CrossRef]

] is employed in the stitching process to deal with any defect features in the overlapping area of two adjacent sub-aperture images. For overlapping areas without defect features, stitching will be performed with the nominal position calculated from the resolution of CCD, the size of the FOV, and the stepping length of the translation stage, etc. Currently, the stitching algorithm works well for most of the cases, especially for small samples with more defects. However, it may fail when evaluating large samples with very few defects, especially with the two conditions below:

  • Sub-aperture images with no feature in the overlapping areas;
  • Sub-aperture images with only ONE run-through line feature in the overlapping areas.

(a) Mismatch caused by images with no feature in overlapping area

Since surface defects are often sparse for fine optics, the overlapping areas of some sub-aperture images may contain no defect features at all. As is indicated above, nominal positions will be used to determine the position of the specific sub-apertures. The stitching process will be affected by the cumulative positioning error of the translation stage. As is shown in Fig. 2
Fig. 2 Stitching mismatch caused by sub-aperture images with no feature in overlapping area by the original method, (a) scanning trace of sub-aperture sampling, (b) stitching result of the original method.
, mismatch occurs due to absent of features in the overlapping area. There are some defects like scratches (s1,s2,s3) and pits (p1,p2) on the test part. Sub-aperture images are taken at the order of Ai,j, Ai+1,j,Ai+1,j+1, andAi,j+1. The translation stage has positioning errors and the sub-apertures are not taken at the nominal positions, as is shown in Fig. 2(a). If stitched correctly, the full aperture image should be the same as Fig. 2(a). But since there is no feature in the overlapping area between Ai,j andAi+1,j, nominal position is used to determine the relative position between these two sub-apertures. Also, because the stitching is performed following the blue trace shown in Fig. 2(b), and template matching is employed for the overlapped areas between Ai+1,jandAi+1,j+1, Ai+1,j+1andAi,j+1, respectively, mismatch will be yielded between Ai,jand Ai,j+1. As can be found in Fig. 2(b), the scratchs1 falls into two scratches s1'ands1''.

(b) Mismatch caused by images with only ONE run-through line feature in overlapping area

There is another case where mismatch can be found between the adjacent sub-apertures. That is the overlapping areas contain only one run-through line feature that both the start and end points of the line are outside of the overlapping area. Figure 3(a)
Fig. 3 Stitching mismatch caused by sub-aperture images with only one fun-through line feature in overlapping area by the original method, (a) scanning trace of sub-aperture sampling, (b) stitching result of the original method.
shows a similar figure of Fig. 2(a) except that there is one run-through line feature in the overlapping area between Ai,jand Ai+1,j. The stitched image is shown in Fig. 3(b), there is an apparent mismatch for scratch s1. But there is another mismatch that is not as apparent as s1, and that is scratch s. Since the cross sections of this line feature are very similar to each other, template matching is carried out with template that only contain run-through line feature. Pseudo-matching [11

11. M. Ghodsi, M. Hajiaghayi, M. Mahdian, and V. Mirrokni, “Length-constrained path-matchings in graphs,” Networks 39(4), 210–215 (2002). [CrossRef]

, 12

12. J. Cohen, P. Fraigniaud, J. Konig, and A. Raspaud, “Optimized broadcasting and multicasting protocols in cut-through routed networks,” IEEE Trans. Parallel Distrib. Syst. 9(8), 788–802 (1998). [CrossRef]

] points will be yielded and what followed is mismatch along the direction of the run-through line feature. It should be noted that, the mismatch for scratch s in this case will not cause breaking of feature as in Fig. 3, but only affects the length evaluation of the line feature. However, these mismatches will influence the following sub-aperture stitching process, as can be seen in Fig. 3(b), the scratchs1 is broken into two scratches s1'ands1''.

It should also be noted that the condition discussed here is for ONE run-through line feature only. If the overlapping area contains not only ONE run-through line feature but also one or more other features, for example, scratches, pits, or even another run-through line feature, template matching can still work well.

From the analysis above, we can conclude that the original stitching method may fail in sub-aperture stitching with no feature or only one run-through line feature in the overlapping areas. One of the reasons is the stitching process follows the same order as sub-aperture sampling. When there is no feature in the overlapping area, direct stitching is used. The accumulated positioning error of the translation stage will affect the stitching that follows. Another reason for the failure of the original method is that it is difficult to find the calibration point for the one run-through line feature.

3. Dark-field microscopic image stitching method

In order to make good use of the information in the overlapping areas, a feature-based multi-cycle dark-field microscopic image stitching method is proposed. In this method, the overlapping areas are firstly categorized by the features they contain: template-friendly feature, only run-through line feature, or no feature. Unlike the linear stitching process in the original method that follows the sub-aperture collection trace, the new method is a region-growing [13

13. A. Mehnert and P. Jackway, “An improved seeded region growing algorithm,” Pattern Recognit. Lett. 18(10), 1065–1071 (1997). [CrossRef]

] like image stitching process. The sub-aperture images with template-friendly feature in the overlapping areas are stitched first, producing some large groups of sub-apertures; then the sub-apertures with only one run-through line feature can be treated with the guidance of their nominal positions; finally, along with the sub-aperture groups obtained, those with no features in the overlapping areas are stitched following the nominal positions.

The flow chart of the stitching method is shown in Fig. 4
Fig. 4 Flow chart of the feature-based multi-cycle dark-field microscopic image stitching method.
. The method is divided into two parts: overlapping area classification and multi-cycle image stitching, as shown in red dash-dot rectangle at the top and blue at the bottom of the chart, respectively. Overlapping area classification consists of extraction of overlapping areas, image preprocessing, connected pixel area labeling, feature extraction, and finally overlapping areas classification based on the features that are contained. According to the classification results, multi-cycle image stitching will be carried out.

3.1 Feature-based classification of overlapping areas

At the very beginning, overlapping areas will be extracted from each sub-aperture image and preprocessed properly. Smoothing filtering is used to eliminate the influence of non-uniform illumination and CCD noise on the correlation operation. Then label each connected pixel area as one feature, as is shown in Fig. 5
Fig. 5 The recorded information of the feature, (a) the ellipse that has the same secondary moment with the rectangular pixel area of the feature, (b) the pose of the ellipse shown in (a).
, and record the following information:

  • 1) the rectangular pixel area Rect of the feature (the red rectangle in Fig. 5(a));
  • 2) the axes length ratio R of the ellipse that has the same secondary moment with Rect(R=LL/LS, where LL and LSare the lengths of the two axes of the ellipse, as are shown in Fig. 5(b));
  • 3) the angle between the major axis of the ellipse and X axis, θ.

Here, if the feature is line feature, Rcan be considered to be the long side of the Minimum Bounding Rectangle (MBR) [14

14. D. Chaudhuri and A. Samal, “A simple method for fitting of bounding rectangle to closed regions,” Pattern Recognit. 40(7), 1981–1989 (2007). [CrossRef]

] of the ellipse, and θ is the angle between the line feature and X axis. If the starting and ending points of the feature are not located at the same side of the overlapping area, and R is larger than the threshold T(usually 4:1 as a rule of thumb), it is a run-through line feature. Note that, the run-through line feature can be straight or arc line. Considering the efficiency of the system as well as some other practical aspects (for example, the microscope’s distortion can reduce the effective size of the sub-aperture image), the overlapping area is usually about 1/6 of the sub-aperture. The run-through arc line feature usually does not change its curvature within the overlapping area and it is still difficult for template matching method to find a suitable template.

As is stated above, template matching method is employed in the stitching of the full aperture image. Suitable templates can be obtained for those defects such as scratches, pits, line features that have dead-end in the overlapping area, etc. But template matching may encounter problems if there is only one run-through line feature or no feature in the overlapping area. So the features can be categorized into three types: a) template-friendly feature, b) only one run-through feature, and c) no information feature. Note that if there are two or more run-through line features in an overlapping area, template matching method can also produce good results. This is because the template can be chosen to contain all the feature information and they are also template-friendly features.

Then the overlapping areas can also be categorized based on the defect features contained in them. The overlapping areas which contain template-friendly features are template-friendly overlapping areas (TFOAs); those that have no features are no feature overlapping areas (NFOAs).

Conditions for those overlapping areas with only one run-through line feature are a little different. In the original stitching method, the sub-apertures with only one run-through line feature in their overlapping area were not stitched properly by the template matching method, as is shown in Section 2.2. But the relative positions of some sub-apertures can also be determined by proper design of the stitching trace. Figure 6
Fig. 6 Illustration of some configurations that the relative positions of the sub-apertures or image blocks can be determined by their mutual positions, (a) to (d) are four different 2 × 2 sub-aperture configurations, (e) to (h) are four example 3 × 3 sub-aperture configurations of image blocks.
shows some configurations where the relative positions of the sub-apertures or image blocks can be determined by their mutual positions, where Figs. 6(a) to 6(d) are four different 2 × 2 sub-aperture configurations, Figs. 6(e) to 6(h) are four example 3 × 3 sub-aperture configurations of image blocks. The n×n (n = 2, 3) grids represent n×n sub-aperture images Ai,j (i = 1~n, j = 1~n), the “-” represent one run-through line feature, respectively. The relative positions of sub-apertures with the same color are all set. It should be noted that overlapping areas are not shown and the line features in overlapping areas are just placed on the boundary line of grids. We can find in Fig. 6(a) that even each sub-aperture within the 2 × 2 sub-aperture group has one line feature in their overlapping areas, all the relative positions of the four sub-apertures can be set because of their mutual positions connected by the four line features. Any pseudo-matching along one line will cause ruptures to another line feature. This condition is called “mutual positioning” condition, and all the overlapping areas between the 2 × 2 sub-aperture group are mutual positioning overlapping areas (MPOAs).

Note that, the 2 × 2 mutual positioning sub-aperture group has three simplified versions, as are shown in Figs. 6(b) to 6(d). In Fig. 6(b), two of the 2 × 2 sub-apertures have already been stitched together while the relative positions of three sub-apertures within the 2 × 2 group have been obtained in Fig. 6(c). In Fig. 6(d), the four sub-apertures are divided into two pairs and the relative positions of the sub-apertures with each pair have been set. Obviously, the above three configurations can be stitched by template matching as the 2 × 2 group shown in Fig. 6(a).

In fact, not only the four sub-apertures within the 2 × 2 group can meet the mutual positioning condition. The relative positions of bigger image blocks, as examples presented in Figs. 6(e) to 6(g), can also be determined. This is because the image blocks in Figs. 6(e) to 6(g) can be converted to the equivalent 2 × 2 mutual positioning configuration as shown in Figs. 6(a) to 6(d), respectively. At this sense, the image blocks in Fig. 6(e) to 6(g) also meet the mutual positioning condition and we can call them mutual positioning image blocks (MPIBs). Note that, the MPIBs can be extended to larger image blocks, given that the sub-apertures of the image block have two shared overlapping areas, which contain only one run-through line feature, with those of another adjacent image block.

Finally, if the overlapping area which contains only one run-through line feature does not meet the mutual positioning condition, it is a discrete through line overlapping area (DLOA).

3.2 Multi-cycle image stitching

The dark-field microscopic image stitching method proposed in this paper is a feature-based multi-cycle image stitching procedure. As is shown in the section above, the overlapping areas are categorized by the features contained in them. Different types of overlapping areas will be stitched in different cycles.

Cycle I: Stitching of template-friendly overlapping area

Cycle II: Stitching of 2 × 2 mutual positioning overlapping areas

As is indicated in Section 3.1, the overlapping areas that meet the mutual positioning condition can still be stitched accurately even if they contain only one run-through line feature. For the general 2 × 2 mutual positioning sub-apertures, as is shown in Fig. 6(a), the process is performed by scanning the four sub-apertures to the suitable positions, as is expressed below:

  • 1) set the stitching template that contains the run-through feature;
  • 2) fix the position of Ai,j, and move the other three sub-apertures to all the possible positions within an area determined by the sub-aperture sampling system;
  • 3) calculate the mean correlation coefficient of the four templates for each possibility.

The correct stitching should have the largest summation of correlation coefficients.

A similar stitching process can be applied to the simplified 2 × 2 mutual positioning sub-aperture groups shown in Fig. 6(b) to 6(d) and mutual positioning image blocks in Fig. 6(e) to 6(g). In short, the key of the new method in dealing with the only one run-through line feature is to change the stitching order from following the sampling order to a self-adapting manner.

Note that new sub-aperture blocks that meet the mutual positioning condition can be yielded after each stitching of the MPOAs or mutual positioning image blocks. Re-definition of the new constructed mutual positioning image blocks is necessary. A detailed illustration is shown in Section 4 Experiment results.

Cycle III: Stitching of discrete run-through line error

Since the sub-apertures with only one run-through line feature in the overlapping area could not be stitched accurately, and there would not be more defects reported due to incorrect stitching, template matching will be employed in the stitching process. Run-through features would not be broken due to incorrect stitching but can be a little shorter or longer (usually smaller than 2%, determined by the positioning accuracy), as is shown in Fig. 6(b). This kind of error is not eliminated by the software. A hardware improvement, however, (for instance, improvement on the positioning accuracy of the translation stage) could reduce the length error of an isolated run-through feature.

Cycle IV: Stitching of no feature

When the stitching process comes to this cycle, there is no feature in the overlapping areas between the two adjacent sub-apertures, and template matching method can be employed. Only the nominal position determined by the translation stage and the microscopic system can be used to determine the position of the image stack.

3.3 Graphical expression of multi-cycle image stitching

Figure 7
Fig. 7 Graphical expression of multi-cycle image stitching, (a) sub-aperture images and features in the overlapping area, (b) sub-aperture blocks after Cycle I: template matching method for TFOAs, (c) sub-aperture blocks after Cycle II: mutual positioning method for MPOAs, (d) sub-aperture blocks after Cycle III: modified template-matching method for DLOAs, (e) sub-aperture blocks after Cycle IV: direct stitching method for OFOAs.
is a graphical expression of the multi-cycle image stitching process, where Fig. 7(a) shows the sub-aperture images and features in the overlapping area, and 7(b)-7(e) show the resulting sub-aperture blocks after each cycle of stitching process. The 5 × 5 grids represent 5 × 5 sub-aperture images Ai,j (i = 1~5, j = 1~5). The “” and “” represent template-friendly features and run-through line features, respectively. It should be noted that overlapping areas are not shown and features in overlapped areas are just placed on the boundary line of grids. If there is nothing on the boundary line of grids means there is no feature in the overlapping areas. Also, defects that are not in the overlapping areas are not present either. As is shown in Fig. 7(a), there are template-friendly features in the overlapping area of A1,3 and A1,4, A2,1 and A2,2,A2,4 and A2,5,A2,4 and A3,4,A3,4 and A4,4,A4,2 and A5,2,A5,2 and A5,3; and there are also some sub-apertures contain only one run-through line feature in their overlapping areas such as A1,4 and A2,4,A2,2 and A2,3,A2,2 and A3,2,A2,3 and A3,3,A2,5 and A3,5,A3,2 and A3,3,A3,4 and A3,5,A4,4 and A4,5.

The multi-cycle stitching process of the 5 × 5 sub-aperture images is shown in Figs. 7(b)-7(e). The sub-aperture blocks are processed from top to bottom and from left to right. As is illustrated in Section 3.2, sub-apertures with template-friendly feature in the overlapping areas will be stitched in Cycle I with template matching method. As is shown in Fig. 7(b), A1,3 will be combined with A1,4, producing an sub-aperture block B1(1) (filled with green); A2,1 and A2,2 will form B2(1) (filled with red); A2,4, A2,5,A3,4and A4,4will construct a larger sub-aperture block B3(1) (filled with pink); A4,2,A5,2 and A5,3 are stitched together and we can get B4(1) (filled with blue). Sub-aperturesA2,3, A3,2,A3,3, A3,5 and A4,5, which has only one run-through line feature in the overlapping areas, are labeled as B5(1), B6(1), B7(1), B8(1)and B9(1), respectively. All the sub-aperture blocks, Bk(1) (k = 1,2,3,…), will join the stitching process in Cycle II. Note that, the sub-apertures which have no feature in the overlapping areas are left unlabeled because they are left to be treated in Cycle IV.

After the two cycles above, all the sub-apertures that can be stitched correctly have been stitched together. The resulted sub-aperture blocks have only one run-through line feature in the overlapping areas (if they have feature in the overlapping area) but do not meet the mutual positioning condition. All these sub-aperture blocks will be stitched with the modified template matching method in Cycle III. As is shown in Fig. 7(d), sub-aperture blocks B2(2), B3(2)and B5(2) are stitched together, producing B1(3). After this cycle, neither of the resulted sub-aperture blocks has feature information in the overlapping area. Since it is difficult to exactly locate all the sub-aperture blocks over the full aperture, direct stitching will be applied under their nominal positions that are determined by the system parameters such as the resolution of CCD, the size of the field of view (FOV), and the stepping length of the translation stage, etc. So in Cycle IV, all the sub-aperture blocks are directly stitched together and form the full aperture image (Fig. 7(e)).

From the expression above, we can find that the new multi-cycle image stitching method employs four cycles to obtain the full aperture image: sub-apertures with different kinds of overlapping areas are processed in separate cycles; after each cycle, sub-aperture blocks may grow bigger and bigger, until to the full aperture image at the end of the stitching procedure.

4. Experimental results

Experiments on the new dark-field microscopic image stitching method were carried out and are presented in this section, including a comparison with the original method. The test sample is a standard reticle plate that is used to calibrate the real size of the defects [6

6. D. Liu, Y. Y. Yang, L. Wang, Y. M. Zhuo, C. H. Lu, L. M. Yang, and R. J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun. 278(2), 240–246 (2007). [CrossRef]

]. It is a 127mm × 127mm square fused silica plate with a thickness of 2.54mm. A group of standard defects, such as scratches, pits, etc, are grooved on it by electron beam exposure (EBE) [15

15. M. Stepanova, S. Dew, M. Mohammad, M. Muhammad, and S. Dew, “Fundamentals of electron beam exposure and development,” in Nanofabrication (Springer, 2012), pp. 11–41.

] and ion beam etching (IBE) [16

16. V. Y. Guzhov, “Ion-beam etching technology in the production of optical elements,” J. Opt. Technol. 69(9), 685 (2002). [CrossRef]

]. The sub-apertures collected by the microscopic system are of 700 × 500 pixels and the object field of the 2 × digital microscope is 1.75mm × 1.25mm. The positioning error of the translation stage that is used for sub-aperture sampling is ± 10μm. The full aperture image is obtained by stitching the sub-apertures together.

The same sub-aperture groups are stitched together by the new proposed multi-cycle dark-field microscopic image stitching method. The results are shown in Fig. 9
Fig. 9 Sub-aperture block of 4 × 4 sub-aperture images obtained using the new proposed dark-field microscopic image stitching method (on the left) with detailed views of some critical areas (on the right).
. Similarly, the sub-aperture grids and detailed view of some critical areas are also presented. It is demonstrated in Fig. 9 that unlike the mismatches obtained with the original method, the stitching result with the new method is more encouraging. In fact, it is not easy to find ruptures with the new method because categorized overlapping areas are stitched in different cycles. The overlapping areas containing template-friendly features are processed first, then those meeting the mutual positioning condition and those containing discrete run-through line feature are treated successively, and finally those with no feature are directly placed at the nominal positions.

Note that there are some run-through line features in the overlapping area between A1,2 and A2,3, and between A1,4 and A2,4. They are parallel run-through line features and sometimes act as only one run-through line feature with template matching stitching. But such features are practically nonexistent on “regular” optical surfaces; they only exist on special calibration artifacts, such as the standard reticle plate. This is also the reason that parallel features are not considered in the feature categorization in Section 3. In this experiment, the parallel run-through line features are treated as only one run-through line feature in the stitching process.

Note that, some connections in the full aperture image obtained above, for instance, the detailed view of “B”, “C” and “D”, are not so smooth. It is because the illumination across the view field of the DM is not very uniform. Shadows of the defect features can be found, and deviates from each other by the shape, size, position, illumination direction of the defects. This effect could not be eliminated by software but can be improved by the upgrade of the illumination system of the defect evaluation system.

5. Conclusions

It is difficult to evaluate surface defects of large aperture fine optical components because the micron-sized defects need to be positioned and quantified accurately over samples of hundreds of millimeters. Sub-aperture scanning and stitching are indispensable to the successful evaluation of defects. The positioning error of the translation stage may accumulate, causing small features to register poorly. But since there are usually very few defects on the test fine optics (and therefore few features exist in the overlapping areas of two adjacent sub-apertures), the original template matching method could not stitch all the sub-apertures together accurately. The dark-field microscopic image stitching method proposed in this paper is a feature-based multi-cycle stitching process. The overlapping areas are categorized based on the features they contain: template-friendly overlapping area, mutual positioning overlapping area, discrete run-through line overlapping area, and no feature overlapping area. Different types of overlapping areas are then stitched in different cycles with different methods. The stitching trace is adaptively determined by the features instead of by the scanning trace. The whole stitching procedure is like a region growing process. Sub-aperture blocks grow bigger after each cycle. A comparison experiment on a reticle plate shows that the new stitching method can successfully avoid the mismatches caused by the original method and stitch the dark-field microscopic sub-aperture images very well. The new proposed method is much more robust to overlapping areas that lack feature information, preventing such areas from contaminating the overall stitch result.

Acknowledgments

We would like to express our great appreciation to the editor, Dr. Paul Murphy, for his time and effect devoted to the significant improvement of this paper. This work was supported by the National Defense Key Program of China under the contract No. 0205010803.18 and the State Key Lab of Modern Optical Instrumentation Innovation Program under the contract No. MOI201208.

References and links

1.

S. Gomez, K. Hale, J. Burrows, and B. Griffiths, “Measurements of surface defects on optical components,” Meas. Sci. Technol. 9(4), 607–616 (1998). [CrossRef]

2.

H. Ota, M. Hachiya, Y. Ichiyasu, and T. Kurenuma, “Scanning surface inspection system with defect-review SEM and analysis system solutions,” Hatachi Review 55, 78–82 (2006).

3.

M. Gebhardt, H. Truckenbrodt, and B. Harnisch, “Surface defect detection and classification with light scattering,” Proc. SPIE 1500, 135–143 (1991). [CrossRef]

4.

F. Rainer, R. K. Dickson, R. T. Jennings, J. F. Kimmons, S. M. Maricle, R. P. Mouser, S. Schwartz, and C. L. Weinzapfel, “Development of practical damage mapping and inspection systems,” Proc. SPIE 3492, 556–563 (1999). [CrossRef]

5.

Y. Yang, C. Lu, J. Liang, D. Liu, L. Yang, and R. Li, “Microscopic dark-field scattering imaging and digitalization evaluation system of defects on optical devices precision surface,” Guangxue Xuebao/Acta Opt. Sin. 27, 1031–1038 (2007).

6.

D. Liu, Y. Y. Yang, L. Wang, Y. M. Zhuo, C. H. Lu, L. M. Yang, and R. J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun. 278(2), 240–246 (2007). [CrossRef]

7.

D. Sun, Y. Yang, F. Wang, L. Yang, and R. Li, “Microscopic scattering imaging system of defects on ultra-smooth surface suitable for digital image processing,” H. Xun, Y. Jiahu, C. W. James, W. Hexin, and H. Sen, eds. (SPIE, 2006), p. 615012.

8.

N. Wei, J. You, K. Friehs, E. Flaschel, and T. W. Nattkemper, “In situ dark field microscopy for on-line monitoring of yeast cultures,” Biotechnol. Lett. 29(3), 373–378 (2007). [CrossRef] [PubMed]

9.

K. Rebner, M. Schmitz, B. Boldrini, A. Kienle, D. Oelkrug, and R. W. Kessler, “Dark-field scattering microscopy for spectral characterization of polystyrene aggregates,” Opt. Express 18(3), 3116–3127 (2010). [CrossRef] [PubMed]

10.

S. Muramatsu, Y. Kobayashi, K. Takahashi, and E. Shimizu, “Development of template matching hardware and its high-speed processing strategy,” Electron. Commun. Jpn. Part III Fundam. Electron. Sci. 84(11), 1–10 (2001). [CrossRef]

11.

M. Ghodsi, M. Hajiaghayi, M. Mahdian, and V. Mirrokni, “Length-constrained path-matchings in graphs,” Networks 39(4), 210–215 (2002). [CrossRef]

12.

J. Cohen, P. Fraigniaud, J. Konig, and A. Raspaud, “Optimized broadcasting and multicasting protocols in cut-through routed networks,” IEEE Trans. Parallel Distrib. Syst. 9(8), 788–802 (1998). [CrossRef]

13.

A. Mehnert and P. Jackway, “An improved seeded region growing algorithm,” Pattern Recognit. Lett. 18(10), 1065–1071 (1997). [CrossRef]

14.

D. Chaudhuri and A. Samal, “A simple method for fitting of bounding rectangle to closed regions,” Pattern Recognit. 40(7), 1981–1989 (2007). [CrossRef]

15.

M. Stepanova, S. Dew, M. Mohammad, M. Muhammad, and S. Dew, “Fundamentals of electron beam exposure and development,” in Nanofabrication (Springer, 2012), pp. 11–41.

16.

V. Y. Guzhov, “Ion-beam etching technology in the production of optical elements,” J. Opt. Technol. 69(9), 685 (2002). [CrossRef]

OCIS Codes
(100.0100) Image processing : Image processing
(120.0120) Instrumentation, measurement, and metrology : Instrumentation, measurement, and metrology
(120.4630) Instrumentation, measurement, and metrology : Optical inspection
(150.1835) Machine vision : Defect understanding

ToC Category:
Instrumentation, Measurement, and Metrology

History
Original Manuscript: November 23, 2012
Revised Manuscript: January 27, 2013
Manuscript Accepted: February 5, 2013
Published: March 4, 2013

Citation
Dong Liu, Shitong Wang, Pin Cao, Lu Li, Zhongtao Cheng, Xin Gao, and Yongying Yang, "Dark-field microscopic image stitching method for surface defects evaluation of large fine optics," Opt. Express 21, 5974-5987 (2013)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-21-5-5974


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. S. Gomez, K. Hale, J. Burrows, and B. Griffiths, “Measurements of surface defects on optical components,” Meas. Sci. Technol.9(4), 607–616 (1998). [CrossRef]
  2. H. Ota, M. Hachiya, Y. Ichiyasu, and T. Kurenuma, “Scanning surface inspection system with defect-review SEM and analysis system solutions,” Hatachi Review55, 78–82 (2006).
  3. M. Gebhardt, H. Truckenbrodt, and B. Harnisch, “Surface defect detection and classification with light scattering,” Proc. SPIE1500, 135–143 (1991). [CrossRef]
  4. F. Rainer, R. K. Dickson, R. T. Jennings, J. F. Kimmons, S. M. Maricle, R. P. Mouser, S. Schwartz, and C. L. Weinzapfel, “Development of practical damage mapping and inspection systems,” Proc. SPIE3492, 556–563 (1999). [CrossRef]
  5. Y. Yang, C. Lu, J. Liang, D. Liu, L. Yang, and R. Li, “Microscopic dark-field scattering imaging and digitalization evaluation system of defects on optical devices precision surface,” Guangxue Xuebao/Acta Opt. Sin.27, 1031–1038 (2007).
  6. D. Liu, Y. Y. Yang, L. Wang, Y. M. Zhuo, C. H. Lu, L. M. Yang, and R. J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun.278(2), 240–246 (2007). [CrossRef]
  7. D. Sun, Y. Yang, F. Wang, L. Yang, and R. Li, “Microscopic scattering imaging system of defects on ultra-smooth surface suitable for digital image processing,” H. Xun, Y. Jiahu, C. W. James, W. Hexin, and H. Sen, eds. (SPIE, 2006), p. 615012.
  8. N. Wei, J. You, K. Friehs, E. Flaschel, and T. W. Nattkemper, “In situ dark field microscopy for on-line monitoring of yeast cultures,” Biotechnol. Lett.29(3), 373–378 (2007). [CrossRef] [PubMed]
  9. K. Rebner, M. Schmitz, B. Boldrini, A. Kienle, D. Oelkrug, and R. W. Kessler, “Dark-field scattering microscopy for spectral characterization of polystyrene aggregates,” Opt. Express18(3), 3116–3127 (2010). [CrossRef] [PubMed]
  10. S. Muramatsu, Y. Kobayashi, K. Takahashi, and E. Shimizu, “Development of template matching hardware and its high-speed processing strategy,” Electron. Commun. Jpn. Part III Fundam. Electron. Sci.84(11), 1–10 (2001). [CrossRef]
  11. M. Ghodsi, M. Hajiaghayi, M. Mahdian, and V. Mirrokni, “Length-constrained path-matchings in graphs,” Networks39(4), 210–215 (2002). [CrossRef]
  12. J. Cohen, P. Fraigniaud, J. Konig, and A. Raspaud, “Optimized broadcasting and multicasting protocols in cut-through routed networks,” IEEE Trans. Parallel Distrib. Syst.9(8), 788–802 (1998). [CrossRef]
  13. A. Mehnert and P. Jackway, “An improved seeded region growing algorithm,” Pattern Recognit. Lett.18(10), 1065–1071 (1997). [CrossRef]
  14. D. Chaudhuri and A. Samal, “A simple method for fitting of bounding rectangle to closed regions,” Pattern Recognit.40(7), 1981–1989 (2007). [CrossRef]
  15. M. Stepanova, S. Dew, M. Mohammad, M. Muhammad, and S. Dew, “Fundamentals of electron beam exposure and development,” in Nanofabrication (Springer, 2012), pp. 11–41.
  16. V. Y. Guzhov, “Ion-beam etching technology in the production of optical elements,” J. Opt. Technol.69(9), 685 (2002). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited