OSA's Digital Library

Optics Express

Optics Express

  • Editor: J. H. Eberly
  • Vol. 3, Iss. 5 — Aug. 31, 1998
  • pp: 190–197
« Show journal navigation

A fast, robust pattern recognition system for low light level image registration and its application to retinal imaging

A. R. Wade and F. W. Fitzke  »View Author Affiliations

Optics Express, Vol. 3, Issue 5, pp. 190-197 (1998)

View Full Text Article

Acrobat PDF (422 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



We describe an image processing system which we have developed to align autofluorescence and high-magnification images taken with a laser scanning ophthalmoscope. The low signal to noise ratio of these images makes pattern recognition a non-trivial task. However, once n images are aligned and averaged, the noise levels drop by a factor of √n and the image quality is improved. We include examples of autofluorescence images and images of the cone photoreceptor mosaic obtained using this system.

© Optical Society of America

1. Introduction

Much of our understanding of the underlying mechanisms of retinal abnormalities has come from investigations of post-mortem material. These have revealed details about the basis of pathologic changes that lead to loss of vision. However, post-mortem investigations are limited in providing information about the functional consequences of pathologies and it is difficult to determine the natural history of change or the effects of interventions. Two recent advances in ophthalmoscopic imaging based on new technological developments allow investigations that can be carried out in the living human eye and can contribute to our understanding of pathologies in a complementary way to the post-mortem studies.

A second technique [3

3. A. R. Wade and F. W. Fitzke, “In-vivo imaging of the human cone photoreceptor mosaic using a confocal LSO,” Lasers and Light in Ophthalmology 1998. (In Press).

] is based upon an optical attachment to the SLO that narrows the scan from 20 degrees to 2.5 degrees and so provides a very high magnification view of a small portion of the retina. At eccentricities of approx. 2 degrees from the fovea, the cone photoreceptor mosaic can be resolved and we hope to use this technique to monitor photoreceptor health in patients with a variety of degenerative retinal diseases.

The SNR of individual cSLO autofluorescence or high-magnification images is very low. To improve image quality, sequences of images are averaged to reduce noise and make visible the spatial structure present in the image. This requires each image in the sequence to be aligned or registered with the others to compensate for eye movements. Alignment is generally necessary for IFA imaging and is an absolute requirement for photoreceptor imaging where the effects of small eye movements are far more significant. Although several techniques are available for aligning images in this way, their effectiveness can be limited due to the high levels of noise present in the images combined with the lack of high-contrast structure. For these reasons most image registration algorithms are either not effective in correcting for eye movements or operate too slowly to be of routine use.

In this paper, we describe an image alignment system that we have developed specifically for noisy images. The system processes high-resolution SVHS video images at a rate of up to 70 frames per minute and has proven to be robust and tolerant of high noise levels in the raw image frames.

2. Materials and methods

Our high-magnification imaging method has been described previously [3

3. A. R. Wade and F. W. Fitzke, “In-vivo imaging of the human cone photoreceptor mosaic using a confocal LSO,” Lasers and Light in Ophthalmology 1998. (In Press).

]. For autofluorescence imaging, the patients’ pupils were dilated using 1% Tropicamide and imaged using the 488nm Argon laser of the Zeiss prototype cSLO with a scan angle of 40°. Images were recorded on a S-VHS video system and digitised in groups of 32 frames using a real-time frame grabber (Pulsar, Matrox Electronic Imaging Inc., Montreal, Canada) with an intensity resolution of 10 bits/pixel. Reflectance images of the region of interest were recorded first and a 521nm low-pass filter was then placed in the path of the photodetector to eliminate directly reflected light and pass only the light emitted by the auto-fluorescent substances in the retina. Images obtained with this filter in place are referred to here as autofluorescence (AF) image. All image processing software was written in-house using the Matrox Image processing Library (MIL) supplied with the digitising board and runs under Windows NT4 on a desktop Pentium PC at 133MHz. No additional image processing hardware was used.

The main stages in the image acquisition and processing procedure are summarised in Fig.1.

Fig. 1. Stages in image averaging procedure

2.1 Image alignment and averaging

2.1.1 Pattern matching algorithm

At its core, a spatial cross-correlation pattern-detector that is searching for a model M in an image I must compute


Where X and Y are the height and width of the model and (p,q) is a position in the image I. This function is evaluated at all locations in I (width P, height Q) and the location which yields the maximum value of r is deemed to be the location of the model in the image. In order to eliminate the effect of linear intensity changes (offset and gain) in the image and model, the MIL algorithm evaluates a slightly more complicated normalised correlation function


With all summations being taken over X and Y in the model and P and Q in the image. In addition, to increase calculation speed the value of r2 rather than r is calculated.

2.1.2 Image pre-processing

A typical image from a high-magnification imaging session is shown in Fig. 2.

Fig. 2. Raw SLO video frame

Raw images such as Fig. 2 contain extremely high levels of noise: a combination of normally-distributed electronic and thermal noise from the SLO photo-diode and Poisson-distributed photon noise. When presented with images of this type, the MIL algorithm fails, either finding false matches to the model or failing to find a match at all. One solution to this problem is to low pass filter the images to attenuate the high spatial frequencies where most of the noise is concentrated.

2.1.3 Model allocation

We have found that a model size of 200x200 pixels works well with our 40° reflectance fundus images (where, for example, the pigmented macular region is around 150 pixels across) and this is the size of model we use for auto-fluorescence images. With high-magnification images, the choice of landmark features is usually more limited and the model size can be selected by the operator. Occasionally the operator will choose a model which cannot be detected in subsequent images; usually because the subject’s eye moves so far in a drift or saccade that the field of view no longer contains the original model region. For this reason, it is essential that the operator monitors the alignment process until it has finished.

2.1.4 Localised search

2.1.5 Rotation

Our method assumes that only translational changes have occurred between the reference and target images. Cross-correlation peak detection in the spatial domain can fail when a rotated version of the original image is presented. The MIL routine tolerates between 10 and 12 degrees of rotation when the model has no obvious rotational symmetry. This has been found to be sufficient to ensure the pattern matching procedure works with all subjects encountered so far. Even greater tolerance to rotation can be achieved if a model with some rotational symmetry (for example, the fovea in an IFA image) is used. The MIL library contains additional routines to find patterns with an unknown amount of rotation as well as translation. However, we have not found that rotational compensation significantly improves the quality of averaged images and the images shown here were aligned using translational displacements only.

3. Results

3.1 Alignment Accuracy

We have tested our alignment routine on sample images with varying amounts of added noise to gauge its accuracy. The results are shown in figure 3. It can be seen that decreasing the signal to noise ratio (SNR), where


leads to an increase in alignment errors. However, the algorithm continues to operate even at extremely low SNRs and at the SNR encountered in a typical autofluorescence image (-15dB) the mean positional error is less than 0.3 pixels. Moreover, increasing the size of the blurring kernel (or equivalently, lowering the low pass filter cutoff frequency) causes only a slight decrease in accuracy. The results shown here are for images with normally-distributed noise added but we have obtained similar results for images with Poisson-distributed noise.

Fig. 3. Effects of increasing blurring kernel size and noise levels on image alignment accuracy. An image set with SNR of -14dB was used to test the effect of blurring kernel size.

3.2 Image alignment examples

Fig. 4. Examples of processed images. Top - High-magnification cone photoreceptor imaging. Bottom - IFA imaging

Figure 4 illustrates the effectiveness of the averaging procedure. Raw IFA and high-magnification images are shown along with their averaged counterparts. Little or no detail is visible in the raw images but in averaged IFA images discrete regions of high autofluorescence are clearly defined and in averaged high-magnification images individual cone photoreceptors can be resolved.

3.3 Processing time

The most significant effect of automating the image alignment process is the reduction in processing time compared to manual alignment. Manual alignment of 32 IFA frames takes an experienced operator at least 15 minutes and in a clinical situation, time constraints effectively limit the maximum size of the image sequence which it is practical to deal with. Using a Pentium 133MHz processor, the automated system takes approximately 30 seconds to align 32 images held in RAM and the limit to sequence length is determined by available disk storage space. We have successfully aligned sequences of 256 images in under 5 minutes. Photoreceptor imaging requires the alignment of at least 64 frames to obtain good quality images and this would not have been possible without the development of an automatic alignment system. Since the processing time is almost entirely dependent upon CPU speed, upgrading to faster CPU clock speeds brings a proportional increase in alignment speed. Tests with a 400 MHz Pentium have shown a three-fold increase in speed as expected.

4. Discussion

High-magnification and IFA imaging promise to be a powerful tools in the study of a wide range of eye diseases. IFA imaging to date has been limited by the time taken to produce good quality images using the technique of noise reduction by signal averaging. High-magnification photoreceptor imaging is a new technique where the noise levels in individual frames are so high that manual alignment of the required number of frames is impractical. The algorithm described above has enabled us to develop a fast, robust image registration system. The total time taken to generate averaged images from video sequences has been reduced by at least a factor of 20 and work is underway to make this process faster still. In addition to speeding up the process of IFA imaging, the system has enabled us to perform high-magnification photoreceptor imaging where previously the alignment time would have prohibited it.



F. C. Delori, “Spectrophotometer for noninvasive measurement of intrinsic fluorescence and reflectance of the ocular fundus,” Appl. Opt. 33, 7439–7452 (1994). [CrossRef] [PubMed]


A. von Ruckman, F. W. Fitzke, and A. C. Bird, “Distribution of fundus autofluorescence with a scanning laser ophthalmoscope,” Br. J. Ophthal. 79, 407–412 (1995). [CrossRef]


A. R. Wade and F. W. Fitzke, “In-vivo imaging of the human cone photoreceptor mosaic using a confocal LSO,” Lasers and Light in Ophthalmology 1998. (In Press).


L. G. Brown, “A survey of image registration techniques,” Computing Surveys 24, 325–376 (1992). [CrossRef]

OCIS Codes
(100.5010) Image processing : Pattern recognition
(330.4300) Vision, color, and visual optics : Vision system - noninvasive assessment

ToC Category:
Research Papers

Original Manuscript: May 11, 1998
Revised Manuscript: May 7, 1998
Published: August 31, 1997

Alex Wade and Frederick Fitzke, "A fast, robust pattern recognition asystem for low light level image registration and its application to retinal imaging," Opt. Express 3, 190-197 (1998)

Sort:  Journal  |  Reset  


  1. F. C. Delori, "Spectrophotometer for noninvasive measurement of intrinsic fluorescence and reflectance of the ocular fundus," Appl. Opt. 33, 7439-7452 (1994). [CrossRef] [PubMed]
  2. A. von Ruckman, F. W. Fitzke and A. C. Bird, "Distribution of fundus autofluorescence with a scanning laser ophthalmoscope," Br. J. Ophthal. 79, 407-412 (1995). [CrossRef]
  3. A. R. Wade, and F. W. Fitzke, " In-vivo imaging of the human cone photoreceptor mosaic using a confocal LSO," Lasers and Light in Ophthalmology 1998. (In Press).
  4. L. G. Brown, "A survey of image registration techniques," Computing Surveys 24, 325-376 (1992). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


Fig. 1. Fig. 2. Fig. 3.
Fig. 4.

« Previous Article

OSA is a member of CrossRef.

CrossCheck Deposited