## Scanning ophthalmoscope retinal image registration using one-dimensional deformation fields |

Optics Express, Vol. 19, Issue 5, pp. 4157-4169 (2011)

http://dx.doi.org/10.1364/OE.19.004157

Acrobat PDF (1297 KB)

### Abstract

We present a new, robust and automated method for registering sequences of images acquired from scanning ophthalmoscopes. The method uses a multi-scale B-spline representation of the deformation field to map images to each other and an hierarchical optimization method. We applied the method to video sequences acquired from different parts of the retina. In all cases, the registration was successful, even in the presence of large distortions from microsaccades, and the resulting deformation fields describe the fixational motion of the eye. The registration reveals examples of dynamic photoreceptor behaviour in the sequences.

© 2011 Optical Society of America

## 1. Introduction

1. M. Stetter, R. A. Sendtner, and G. T. Timberlake, “A novel method for measuring saccade profiles using the scanning laser ophthalmoscope,” Vis. Res. **36**, 1987–1994 (1996). [PubMed]

3. S. Martinez-Conde, S. L. Macknik, and D. H. Hube, “The role of fixational eye movements in visual perception,” Nat. Rev. Neurosci. **5**, 229–240 (2004). [PubMed]

6. C. R. Vogel, D. W. Arathorn, A. Roorda, and A. Parker, “Retinal motion estimation in adaptive optics scanning laser ophthalmoscopy,” Opt. Express **14**(2), 487–497 (2006). [PubMed]

7. D. W. Arathorn, Q. Yang, C. R. Vogel, Y. Zhang, P. Tiruveedhula, and A. Roorda, “Retinally stabilized cone-targeted stimulus delivery,” Opt. Express **15**(21), 13731–13744 (2007) [PubMed]

*et al.*but with the points tracked manually [9

9. V. Nourrit, J. M. Bueno, B. Vohnsen, and P. Artal, “Nonlinear registration for scanned retinal images: application to ocular polarimetry,” Appl. Opt. **47**(29), 5341–5347 (2008). [PubMed]

*et al.*estimated transformation models that included a range of global and local mapping strategies.

- For a small field of view, the non-planar geometry of the retina can be neglected. Then rotation of the eye about the horizontal and vertical axes can be treated as a rigid translation of the image.
- We shall assume that the fixational eye motion during a single horizontal line scan can be neglected. A single horizontal scan is captured in tens of microseconds, and even at the peak velocity of a microsaccade, the line shift in this time is of the order of a fraction of a pixel. A similar assumption is made in [5].
- We shall assume that rotation about the torsional axis can be neglected. The rotation around the torsional axis, which is known as ocular torsion, may give rise to complicated image distortions. However, these ocular torsions are neglected in most papers, leading nevertheless to good results.

10. V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, “3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization,” IEEE Trans. Image Process. **14**, 553–566 (2005). [PubMed]

## 2. Method

### 2.1. Problem formalization

**I**

_{reg}at coordinate vector

**s**is finally obtained as The registration of the floating image

**I**

_{float}onto the reference image

**I**

_{ref}can be stated as the following constrained optimization problem: where

*θ*represents the parameters that define the transformation

**h**

*, Θ is the set of admissible parameters and*

_{θ}*E*(.,.) is the similarity criterion, or cost function.

### 2.2. Properties of the mapping between the reference image and the floating image

**s**is a point on the reference image and

**u**is the displacement vector field (representing the deformation). Note that if

**u**is the null function,

**h**corresponds to the identity transformation.

**u**(

**s**) is constant for a horizontal line and we can write where

*u*(.), and

_{x}*u*(.) are translations in the vertical (

_{y}*x*), and horizontal (

*y*) axes (see Fig. 1). Notice that these two functions depend only on

*s*: this means that the displacement field

_{x}**u**is a function of one dimension.

### 2.3. Modelling of the displacement vector field

*u*(.), and

_{x}*u*(.) can be represented with a one-dimensional B-spline of order 1. An advantage of using a low-order B-spline over other representations such as polynomial, harmonic functions, radial basis functions or wavelets is that they have a small overlap. This reduces the interdependency between the parameters which makes the minimization problem easier to solve [11]. Additionally, B-splines have been shown to have very good approximation properties [11].

_{y}*l*, the function is represented as a weighted sum of 2

^{l+1}– 1 triangular basis functions

*ϕ*(

*t*) = {1 – 2|

*t*| for |

*t*| < 1; 0 elsewhere}. Then, for the spline representation of the deformation field at scale

*l*we have where

*= [−1/2, 1/2]. Using a larger support than the region of interest is necessary to allow the represented functions to have non-zero values at the boundary of Ω*

_{x}*. However, this also means that at higher scales, some of the spline elements fall completely outside Ω*

_{x}*and so have no influence on the registration mapping.*

_{x}### 2.4. Optimization

**I**

_{ref}and

**I**

_{float}are the reference and floating image (with gray-levels linearly scaled to have the same mean and variance). The summation is over those image pixels in the reference image (the discrete space Ω

_{d}) that map onto the domain of the floating image (Ω).

*N*(

**h**) is the number of terms in the summation, which depends on the image mapping

**h**. Note that whereas a normalized cross-correlation criterion would use a modulus-squared, here we use the absolute value for

*f*which is less sensitive to outliers.

10. V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, “3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization,” IEEE Trans. Image Process. **14**, 553–566 (2005). [PubMed]

*λ*is a weighting factor of the regularization term and

*C*a scaling factor to make

*E*

^{(sim)}and

*E*

^{(reg)}comparable. In fact here we present results both using with and without a regularisation term in the cost function.

*l*= 0, and then move successively through finer scales until

*l*=

*l*

_{max}. The solution obtained at each scale is used as the starting point for optimizing at the next scale (note that the scalable property of the splines means that the displacement vector at each scale

*l*can be represented exactly at the next scale

*l*+ 1). Note also that the final resulting displacement vector, including the initial translation can be completely represented in the final B-spline field We also use a hierarchical representation of the images, starting with reduced resolution images of

**I**

_{ref}and of

**I**

_{float}at the lowest scales and increasing the resolution throughout the algorithm.

*l*, the optimization parameters are the set of spline weights for the displacement vector. Joint estimation of parameters is non-trivial, thus we use a sequential optimization procedure. We optimize the two parameters associated to one basis function at a time, keeping the rest constant. The result of a sequential optimization procedure in general can depend on the order of visitation of the parameters. However, the finite support of the spline basis functions allows us to split the basis functions into two sets (one comprising the odd indexed, the other the even indexed parameters). Within each set, the basis functions are completely disjoint, therefore the visitation order does not matter. Our procedure is to iterate alternately between the odd and the even sets of basis functions. The detailed procedure is described in Algorithm (1).

*l*is finally described in Algorithm (2).

### 2.5. Desinusoiding and Registration

**h**

_{des}and its inverse

**h**is again the registration transformation that satisfies the rigid translation properties we require. To avoid any extra interpolations (which can introduce artefacts) the desinusoiding transformations are incorporated directly so that

**I**

_{float}is warped according to

**I**

_{ref}○

**h**

_{des}and the desinusoided registered image can be computed as

**I**

_{float}○

**h**

_{des}○

**h**.

## 3. Results

*μs*.

*λ*= 0 in Eq. (8)). Since there was no need to favour smoothness at coarser scales,

*λ*was always set to 0 at the beginning of the registration for scales with functions with domain larger or equal than 16 pixels.

### 3.1. Video stabilization

14. R. S. Jonnal, J. Rha, Y. Zhang, B. Cense, W. Gao, and D. T. Miller, “In vivo functional imaging of humancone photoreceptors,” Opt. Express **15**(24), 16141–16160 (2007). [PubMed]

^{2}). Regularization becomes necessary when there is insufficient data in the image to guide the registration and, for example, non-rigid registration can easily become an ill-posed problem where homogeneous regions in an image have to be mapped. However, this is unlikely to happen when registering complete lines of retinal images which have fine structure everywhere in the image.

### 3.2. Microsaccade motion

*u*and

_{x}*u*). In the microsaccade near the bottom of

_{y}**I**

_{float}(extending approximately from line 160 to line 230) the horizontal component (

*u*) is particularly severe, with a horizontal shift of approximately 35 pixels over 70 lines - i.e., half a pixel shift per line scan. The black regions near the edges of the registered image, which correspond to regions that were outside the floating image, give a further indication of the severity of the motion that is nevertheless correctly registered here. This example corresponds to a microsaccade of approximate magnitude 16° s

_{y}^{−1}(assuming the reference image is fixed). Martinez-Conde

*et al.*refer to studies that indicate microsaccades up to 120° s

^{−1}[3

3. S. Martinez-Conde, S. L. Macknik, and D. H. Hube, “The role of fixational eye movements in visual perception,” Nat. Rev. Neurosci. **5**, 229–240 (2004). [PubMed]

*u*in Fig. 7). In this case, eye motion is not completely responsible for the deformation: the vertical scan speed of this instrument is not constant near the top of the frame and small vertical motion of the eye can give larger displacement. The registration method has corrected for this motion as well as the fixation motion, illustrating the robustness of the approach. Although not necessary for the registration, a desinusoiding procedure could be used in the vertical direction to remove such scanner distortion from the images.

_{x}## 4. Conclusions and discussion

6. C. R. Vogel, D. W. Arathorn, A. Roorda, and A. Parker, “Retinal motion estimation in adaptive optics scanning laser ophthalmoscopy,” Opt. Express **14**(2), 487–497 (2006). [PubMed]

## Acknowledgments

## References and links

1. | M. Stetter, R. A. Sendtner, and G. T. Timberlake, “A novel method for measuring saccade profiles using the scanning laser ophthalmoscope,” Vis. Res. |

2. | A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, “A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina,” IEEE Trans. Pattern Anal. Mach. Intell. |

3. | S. Martinez-Conde, S. L. Macknik, and D. H. Hube, “The role of fixational eye movements in visual perception,” Nat. Rev. Neurosci. |

4. | J. B. Mulligan, “Recovery of motion parameters from distortions in scanned images,” Proceedings of the NASA Image Registration Workshop (IRW97), NASA Goddard Space Flight Center, MD, 1997. |

5. | S. B. Stevenson and A. Roorda, “Correcting for miniature eye movements in high resolution scanning laser ophthalmoscopy,” in Ophthalmic Technologies XV , edited by Fabrice Manns, Per Soderberg, and Arthur Ho, Proc. of SPIE Vol. |

6. | C. R. Vogel, D. W. Arathorn, A. Roorda, and A. Parker, “Retinal motion estimation in adaptive optics scanning laser ophthalmoscopy,” Opt. Express |

7. | D. W. Arathorn, Q. Yang, C. R. Vogel, Y. Zhang, P. Tiruveedhula, and A. Roorda, “Retinally stabilized cone-targeted stimulus delivery,” Opt. Express |

8. | H. Li, J. Lu, G. Shi, and Y. Zhang, “Tracking features in retinal images of adaptive optics confocal scanning laser ophthalmoscope using KLT-SIFT algorithm,” Biomed. Opt. Express |

9. | V. Nourrit, J. M. Bueno, B. Vohnsen, and P. Artal, “Nonlinear registration for scanned retinal images: application to ocular polarimetry,” Appl. Opt. |

10. | V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, “3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization,” IEEE Trans. Image Process. |

11. | J. Kybic and M. Unser, “Fast parametric elastic image registration,” IEEE Trans. Image Process. |

12. | H. Lester and S. R. Arridge, “A survey of hierarchical non-linear medical image registration,” Pattern Recogn. |

13. | V. Chvátal, |

14. | R. S. Jonnal, J. Rha, Y. Zhang, B. Cense, W. Gao, and D. T. Miller, “In vivo functional imaging of humancone photoreceptors,” Opt. Express |

**OCIS Codes**

(100.0100) Image processing : Image processing

(170.4470) Medical optics and biotechnology : Ophthalmology

(170.5810) Medical optics and biotechnology : Scanning microscopy

(170.5755) Medical optics and biotechnology : Retina scanning

**ToC Category:**

Medical Optics and Biotechnology

**History**

Original Manuscript: October 8, 2010

Revised Manuscript: December 17, 2010

Manuscript Accepted: December 19, 2010

Published: February 17, 2011

**Virtual Issues**

Vol. 6, Iss. 3 *Virtual Journal for Biomedical Optics*

**Citation**

S. Faisan, D. Lara, and C. Paterson, "Scanning ophthalmoscope retinal image registration using one-dimensional deformation fields," Opt. Express **19**, 4157-4169 (2011)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-19-5-4157

Sort: Year | Journal | Reset

### References

- M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996). [PubMed]
- A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).
- S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004). [PubMed]
- J. B. Mulligan, "Recovery of motion parameters from distortions in scanned images," Proceedings of the NASA Image Registration Workshop (IRW97), NASA Goddard Space Flight Center, MD, 1997.
- S. B. Stevenson, and A. Roorda, "Correcting for miniature eye movements in high resolution scanning laser ophthalmoscopy," in Ophthalmic Technologies XV, edited by Fabrice Manns, Per Soderberg, Arthur Ho, Proc. of SPIE Vol. 5688A, pp. 145-151 (2005).
- C. R. Vogel, D. W. Arathorn, A. Roorda, and A. Parker, "Retinal motion estimation in adaptive optics scanning laser ophthalmoscopy," Opt. Express 14(2), 487-497 (2006). [PubMed]
- D. W. Arathorn, Q. Yang, C. R. Vogel, Y. Zhang, P. Tiruveedhula, and A. Roorda, "Retinally stabilized conetargeted stimulus delivery," Opt. Express 15(21), 13731-13744 (2007). [PubMed]
- H. Li, J. Lu, G. Shi, and Y. Zhang, "Tracking features in retinal images of adaptive optics confocal scanning laser ophthalmoscope using KLT-SIFT algorithm," Biomed. Opt. Express 1, 31-40 (2010).
- V. Nourrit, J. M. Bueno, B. Vohnsen, and P. Artal, "Nonlinear registration for scanned retinal images: application to ocular polarimetry," Appl. Opt. 47(29), 5341-5347 (2008). [PubMed]
- V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005). [PubMed]
- J. Kybic, and M. Unser, "Fast parametric elastic image registration," IEEE Trans. Image Process. 12, 1427-1442 (2003).
- H. Lester, and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit. 32(1), 129-149 (1999).
- V. Chvátal, Linear programming, (Freeman, 1983).
- R. S. Jonnal, J. Rha, Y. Zhang, B. Cense, W. Gao, and D. T. Miller, "In vivo functional imaging of human cone photoreceptors," Opt. Express 15(24), 16141-16160 (2007). [PubMed]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Supplementary Material

» Media 1: AVI (654 KB)

» Media 2: AVI (619 KB)

» Media 3: AVI (654 KB)

» Media 4: AVI (631 KB)

» Media 5: AVI (654 KB)

» Media 6: AVI (625 KB)

» Media 7: AVI (47 KB)

« Previous Article | Next Article »

OSA is a member of CrossRef.