## Robust disparity estimation based on color monogenic curvature phase |

Optics Express, Vol. 20, Issue 14, pp. 14971-14979 (2012)

http://dx.doi.org/10.1364/OE.20.014971

Acrobat PDF (2687 KB)

### Abstract

Disparity estimation for binocular images is an important problem for many visual tasks such as 3D environment reconstruction, digital hologram, virtual reality, robot navigation, etc. Conventional approaches are based on brightness constancy assumption to establish spatial correspondences between a pair of images. However, in the presence of large illumination variation and serious noisy contamination, conventional approaches fail to generate accurate disparity maps. To have robust disparity estimation in these situations, we first propose a model - color monogenic curvature phase to describe local features of color images by embedding the monogenic curvature signal into the quaternion representation. Then a multiscale framework to estimate disparities is proposed by coupling the advantages of the color monogenic curvature phase and mutual information. Both indoor and outdoor images with large brightness variation are used in the experiments, and the results demonstrate that our approach can achieve a good performance even in the conditions of large illumination change and serious noisy contamination.

© 2012 OSA

## 1. Introduction

6. A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE **69**, 529–541 (1981). [CrossRef]

7. M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. **49**, 3136–3144 (2001). [CrossRef]

8. M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision **21**, 5–26 (2004). [CrossRef]

9. G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision **40**, 269–284 (2011). [CrossRef]

10. D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image **18**, 81–99 (2007). [CrossRef]

11. D. Zang, J. Li, and D. Zhang, “Robust visual correspondence computation using monogenic curvature phase based mutual information,” Opt. Lett. **37**, 10–12 (2012). [CrossRef] [PubMed]

## 2. Color monogenic curvature phase

### 2.1. Monogenic curvature signal

*f*(

*x*,

*y*), (

*x*,

*y*) ∈

*R*

^{2}, the monogenic curvature signal [10

10. D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image **18**, 81–99 (2007). [CrossRef]

*f*

_{1}can be obtained as where * represents the convolution operator,

7. M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. **49**, 3136–3144 (2001). [CrossRef]

12. F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. **2006**, 98145 (2006). [CrossRef]

*f*

_{1}will yield the other two components

*f*

_{2}and

*f*

_{3}of the monogenic curvature signal. In the frequency domain, the second order Hilbert transform reads

*H*

_{2}= [cos2

*α*sin2

*α*]

*, where*

^{T}*α*is the polar coordinate. The other two components of the monogenic curvature signal are respectively given by where

*ℱ*

^{−1}refers to the inverse Fourier transform and

*F*

_{1}is the Fourier transformed result of

*f*

_{1}.

**f**

*(*

_{mc}*x*,

*y*,

*s*) with

*s*being the scale parameter. The monogenic curvature scale-space performs a split of identity, from it, three independent local features, i.e. the amplitude, main orientation and monogenic curvature phase, can be simultaneously obtained as where atan2(·) ∈ (−

*π*,

*π*] and

**u**(

*x*,

*y*,

*s*) = [

*f*

_{2}(

*x*,

*y*,

*s*)

*f*

_{3}(

*x*,

*y*,

*s*)]

*.*

^{T}### 2.2. Color monogenic curvature scale-space

13. S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett. **32**, 1979–1980 (1996). [CrossRef]

*i*,

*j*and

*k*are three imaginary units,

*f*,

_{r}*f*and

_{g}*f*indicate the red, green and blue channels of the color image. We are thus inspired to extend the monogenic curvature scale-space to the color domain by embedding it into the framework of quaternion. Similar to the color image representation, corresponding components of monogenic curvature scale-space are considered as three channels to be encoded in a pure quaternion. Therefore, the color monogenic curvature scale-space

_{b}**f**

*can be constructed as where*

_{cmc}**f**

*refers to the monogenic curvature scale-space of the*

_{nmc}*n*th color channel.

*is given by Figure 1 illustrates the computed color monogenic curvature phase results at the first scale. Top row contains three test images taken from [15], they are captured under different camera exposure and lighting conditions. Bottom row includes corresponding color monogenic curvature phase images. It is shown that the color monogenic curvature phase is very robust against large illumination variation.*

_{cmc}## 3. Disparity estimation

*I*and

_{l}*I*, the corresponding phase information Φ

_{r}*and Φ*

_{l}*can be extracted by applying the color monogenic curvature phase model. Based on Φ*

_{r}*and Φ*

_{l}*, two pyramids are correspondingly constructed by down-sampling the original phase images. At each scale*

_{r}*s*, the disparity map can be computed by using the mutual information of two phase images Φ

*and Φ*

_{l,s}*as the matching cost. From the coarsest scale, the estimated disparity map is used in the next scale for initialization, and this continues to the finest scale.*

_{r,s}*and Φ*

_{l,s}*can be defined as where*

_{r,s}*H*(Φ

*) and*

_{l,s}*H*(Φ

*) are the Shannon entropy which can be given by where*

_{r,s}*E*

_{Φ}indicates the expected value function of Φ,

*P*(Φ) is the probability of Φ, Ω

*refers to the domain over which the random variable can range and*

_{ϕ}*ϕ*is an event in this domain.

_{i}*H*(Φ

*, Φ*

_{l,s}*) indicates the joint entropy of Φ*

_{r,s}*and Φ*

_{l,s}*, it is represented in the following form where*

_{r,s}*E*refers to the expectation,

*P*(Φ

*, Φ*

_{l,s}*) is the joint distribution of Φ*

_{r,s}*and Φ*

_{l,s}*. Since Eq. (13) defines the mutual information for the whole phase image, similar to [16], we approximate the whole mutual information as the sum of the pixel-wise mutual information and use it as a data cost, that is where*

_{r,s}*d*refers to the disparity at the pixel

_{p}*p*.

*E*is a matching cost which works as a similarity measure and

_{data}*E*is the smooth energy which penalizes disparity differences. In this paper, we use the mutual information of the color monongeic curvature phase image as a matching cost. Based on the approximation, the pixel-wise data energy

_{smooth}*E*can be formulated as We use a truncated quadratic function as the smoothness energy, which is defined as where

_{data}*𝒩*(

*p*) is the neighbourhood pixels of the pixel p, and

*V*is represented as with

_{pq}*λ*being a weighting parameter. The Graph-cuts expansion algorithm proposed in [17

17. Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. **23**, 1222–1239 (2001). [CrossRef]

## 4. Experimental results

11. D. Zang, J. Li, and D. Zhang, “Robust visual correspondence computation using monogenic curvature phase based mutual information,” Opt. Lett. **37**, 10–12 (2012). [CrossRef] [PubMed]

11. D. Zang, J. Li, and D. Zhang, “Robust visual correspondence computation using monogenic curvature phase based mutual information,” Opt. Lett. **37**, 10–12 (2012). [CrossRef] [PubMed]

**37**, 10–12 (2012). [CrossRef] [PubMed]

## 5. Conclusions

## Acknowledgment

## References and links

1. | S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell. |

2. | H. Moravec, “Toward automatic visual obstacle avoidance,” in Proceedings of 5th International Joint Conference on Artificial Intelligence, (Morgan Kaufmann, 1977), pp. 584–590. |

3. | C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940. |

4. | I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern. |

5. | A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38. |

6. | A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE |

7. | M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. |

8. | M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision |

9. | G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision |

10. | D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image |

11. | D. Zang, J. Li, and D. Zhang, “Robust visual correspondence computation using monogenic curvature phase based mutual information,” Opt. Lett. |

12. | F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. |

13. | S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett. |

14. | N. L. Bihan and S. J. Sangwine, “Quaternion principal component analysis of color images,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2003), pp. 809–812. |

15. | |

16. | J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040. |

17. | Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. |

18. | D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2007), pp. 1–8. |

**OCIS Codes**

(100.2000) Image processing : Digital image processing

(100.2960) Image processing : Image analysis

(330.1400) Vision, color, and visual optics : Vision - binocular and stereopsis

(100.3008) Image processing : Image recognition, algorithms and filters

**ToC Category:**

Image Processing

**History**

Original Manuscript: February 6, 2012

Revised Manuscript: May 21, 2012

Manuscript Accepted: June 7, 2012

Published: June 20, 2012

**Virtual Issues**

Vol. 7, Iss. 9 *Virtual Journal for Biomedical Optics*

**Citation**

Di Zang, Jie Li, Dongdong Zhang, and Junqi Zhang, "Robust disparity estimation based on color monogenic curvature phase," Opt. Express **20**, 14971-14979 (2012)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-20-14-14971

Sort: Year | Journal | Reset

### References

- S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell.20, 401–406 (1998). [CrossRef]
- H. Moravec, “Toward automatic visual obstacle avoidance,” in Proceedings of 5th International Joint Conference on Artificial Intelligence, (Morgan Kaufmann, 1977), pp. 584–590.
- C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940.
- I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern.37, 1009–1014 (2007). [CrossRef]
- A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38.
- A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE69, 529–541 (1981). [CrossRef]
- M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process.49, 3136–3144 (2001). [CrossRef]
- M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision21, 5–26 (2004). [CrossRef]
- G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision40, 269–284 (2011). [CrossRef]
- D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image18, 81–99 (2007). [CrossRef]
- D. Zang, J. Li, and D. Zhang, “Robust visual correspondence computation using monogenic curvature phase based mutual information,” Opt. Lett.37, 10–12 (2012). [CrossRef] [PubMed]
- F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci.2006, 98145 (2006). [CrossRef]
- S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett.32, 1979–1980 (1996). [CrossRef]
- N. L. Bihan and S. J. Sangwine, “Quaternion principal component analysis of color images,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2003), pp. 809–812.
- http://vision.middlebury.edu/stereo/ .
- J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040.
- Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell.23, 1222–1239 (2001). [CrossRef]
- D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2007), pp. 1–8.

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.