## Compression of 3D color integral images

Optics Express, Vol. 12, Issue 8, pp. 1632-1642 (2004)

http://dx.doi.org/10.1364/OPEX.12.001632

Acrobat PDF (1271 KB)

### Abstract

In this paper, we discuss the compression results of full color 3D Integral Images (II) by MPEG-2 (Motion Picture Experts Group). II is a popular three-dimensional image video recording and display technique. The huge size of II data has become a practical issue for storing and transmitting of 3D scenes. The MPEG is a standard coded representation of moving pictures. We model the elemental images in II as consecutive frames in a moving picture. Therefore, MPEG scheme can be applied to take advantage of the high cross-correlations between elemental images. We also introduce several scanning topologies along the elemental image sequences and investigate their performance with different number of pictures in GOP (Group of Picture). Experimental results are presented to illustrate the image quality of the MPEG-2 and the baseline JPEG with the same compression rate. We show that a well-known and widely-available MPEG-2 scheme can be a good alternative for II compression.

© 2004 Optical Society of America

## 1. Introduction

1. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. **36**, 1598–1603 (1997). [CrossRef] [PubMed]

1. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. **36**, 1598–1603 (1997). [CrossRef] [PubMed]

7. C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. **58**, 71–76 (1968). [CrossRef]

10. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. **26**, 157–159 (2001). [CrossRef]

12. A. Stern and B. Javidi, “3-D computational synthetic aperture integral imaging (COMPSAII),” Opt. Express **11**, 2446–2451 (2003), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-19-2446. [CrossRef] [PubMed]

13. Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. **41**, 5488–5496 (2002). [CrossRef] [PubMed]

11. J. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. **27**, 1144–1146 (2002). [CrossRef]

12. A. Stern and B. Javidi, “3-D computational synthetic aperture integral imaging (COMPSAII),” Opt. Express **11**, 2446–2451 (2003), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-19-2446. [CrossRef] [PubMed]

13. Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. **41**, 5488–5496 (2002). [CrossRef] [PubMed]

14. M. C. Forman and A. Aggoun, “Quantisation strategies for 3D-DCT-BASED compression of full parallax 3D images,” in *Proceedings of Int. Conf. on Image processing and its applications, 6th*, (Ireland, 1997), pp. 32–35. [CrossRef]

15. R. Zaharia, A. Aggoun, and M. McCormick, “Adaptive 3D-DCT compression algorithm for continuous parallax 3D integral imaging,” Signal Processing: Image Communication **17**, 231–242 (2002). [CrossRef]

## 2. Background on MPEG-2

*Generic coding of moving pictures and associated audio*[17

17. V. Bhaskaran and K. Konstantinides, *Image and video compression standards*2nd edition, (Kluwer Academic Publishers, 1997). [CrossRef]

*Digital compression and coding of continuous-tone still images*or to the ITU-T Recommendation T. 81~4. It has been developed to compress still images and became widely used [17

17. V. Bhaskaran and K. Konstantinides, *Image and video compression standards*2nd edition, (Kluwer Academic Publishers, 1997). [CrossRef]

25. T. Nomura, A. Okazaki, M. Kameda, Y. Morimoto, and B. Javidi, “Digital holographic data reconstruction with data compression,” in *Algorithms and Systems for Optical Information Processing V*, B. Javidi and D. Psaltis, eds., Proc. SPIE4471, (2001). [CrossRef]

*Y*is the luminance component and

*Cb*and

*Cr*are the chrominance components. MPEG has two strategies for video compression. One is “intraframe coding” and the other is “interframe coding.” The former one is similar to still image compression by JPEG. For the latter one, MPEG subtracts macroblocks established with the motion vector and performs DCT coding of their difference. For the interframe coding, motion estimation yields the optimal motion vector of macroblocks. During the motion compensation, the prediction errors of moving pictures are computed between the best-matching macroblocks in different frames. The macroblock is the smallest coded unit. It consists of four 8×8 blocks of

*Y*, one 8×8 block of

*Cb*, and one 8×8 block of

*Cr*.

*N*=6 and

*M*=3.

*N*is distance between I pictures and

*M*is distance between consecutive I or P pictures.

17. V. Bhaskaran and K. Konstantinides, *Image and video compression standards*2nd edition, (Kluwer Academic Publishers, 1997). [CrossRef]

*Image and video compression standards*2nd edition, (Kluwer Academic Publishers, 1997). [CrossRef]

## 3. Compressing integral images using MPEG-2

*I*is the image obtained after decompression. MSE (Mean Square Error) is defined as:

_{u}*M*,

_{s}*N*) are the number of pixels in the

_{s}*x*and

*y*axis of the image. Another metric for image quality is SNR (Signal to Noise Ratio) defined as:

*VAR*(

*I*) is the variance of the original image

_{o}*I*.

_{o}## 4. Experimental results

*f*-number of the imaging lens is 2.5. The imaging lens is inserted between the lens array and CCD camera due to the short focal length of the lenslets.

26. MPEG-2 Video Codec (with Source Code), http://www.mpeg.org/MPEG/MSSG/#source

^{5}bits/s, and the frame rate was set at 30 frames/s for II-(1) in Fig. 5(b) and II-(2) in Fig. 5(c). The quality factor of JPEG was set at 20 for II-(1) and 23 for II-(2) to provide the same compression rates between MPEG-2 and JPEG. The sizes of compressed files are 80 Kbytes for both methods. Figure 6(b) and 6(d) show decompressed elemental images of II-(1) and II-(2) after MPEG-2 compression. Table 2 shows detailed experimental results for II-(1) and II-(2). The PSNR and SNR of MPEG-2 are larger than those of JPEG. It is noted that MPEG-2 depends on high similarities between elemental images.

*N*=3, 6, 9, 12, 15, and

*M*=3. As shown in the Figures, PSNR and SNR are mostly higher when

*N*=6 and

*M*=3 for three different scanning topologies shown in Fig. 4. The spiral scanning topology provides better result for II-(1) scene and the perpendicular and spiral scanning topologies provide better results for II-(2) scene. This is because elemental images for interframe coding are closer to each other when the perpendicular and spiral scanning topologies are used. Closer elemental images result in smaller and more exact motion compensation. This may be also due to the effect of having blurred elemental images which are located near the edge of 3D scenes.

*N*=6 and

*M*=3. Several compression rates of II are achieved by changing the bit rate with the fixed frame rate in MPEG-2 and changing the quality factor in JPEG. MPEG-2 TM codec supports adaptive quantization which can be controlled by bit rate or frame rate [27

27. MPEG-2 Test Model 5, http://www.mpeg.org/MPEG/MSSG/tm5

^{5}, 5×10

^{5}, 4×10

^{5}, 3×10

^{5}, and 2×10

^{5}bits/s. The quality factor for JPEG is 41, 31, 20, 10, 5 for II-(1) and 44, 34, 22, 12, 9 for II-(2). Figures 11 and 12 show movies of elemental images of II-(1) and II-(2) for original and decompressed elemental images after MPEG-2, respectively. It is noted that the compression rate is far less than theoretical value (30×208

^{2}×24/2×10

^{5}≈155.75) when bit rate is 2×10

^{5}bits/s.

## 5. Conclusions

## References and links

1. | F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. |

2. | F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. |

3. | S. A. Benton, ed., |

4. | D. H. McMahon and H. J. Caulfield, “A technique for producing wide-angle holographic displays,” Appl. Opt. |

5. | T. Okoshi, “Three-dimensional displays,” in |

6. | G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. |

7. | C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. |

8. | M. Martinez-Corral, C. Ibáñez-López, and G. Saavedra, “Axial gain resolution in optical sectioning fluorescence microscopy by shaded-ring filters,” Opt. Express |

9. | P. Ambs, L. Bigue, Y. Fainman, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “Image reconstruction using electrooptic holography,” |

10. | H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. |

11. | J. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. |

12. | A. Stern and B. Javidi, “3-D computational synthetic aperture integral imaging (COMPSAII),” Opt. Express |

13. | Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. |

14. | M. C. Forman and A. Aggoun, “Quantisation strategies for 3D-DCT-BASED compression of full parallax 3D images,” in |

15. | R. Zaharia, A. Aggoun, and M. McCormick, “Adaptive 3D-DCT compression algorithm for continuous parallax 3D integral imaging,” Signal Processing: Image Communication |

16. | J. S. Jang and B. Javidi, “Compression of ray information in three-dimensional integral imaging using the Karhunen-Loeve transform,” submitted to Opt. Lett. (2004). |

17. | V. Bhaskaran and K. Konstantinides, |

18. | M. Rabbani, |

19. | R. L. Joshi, M. Rabbani, and M. A. Lepley, “Comparison of multiple compression cycle performance for JPEG and JPEG 2000,” in |

20. | J. A. Saghri, A. G. Tescher, and A. M. Planinac, “KLT/JPEG 2000 multispectral bandwidth compression with region-of-interest prioritization capability,” in |

21. | T. J. Naughton, Y. Frauel, B. Javidi, and E. Tajahuerce, “Compression of digital holograms for three-dimensional object reconstruction and recognition,” Appl. Opt. |

22. | A. Mahalanobis and C. Daniell, “Data compression and correlation filtering,” in |

23. | M. Rabbani, |

24. | T. A. Welch, “A technique for high performance data compression,” IEEE Computer |

25. | T. Nomura, A. Okazaki, M. Kameda, Y. Morimoto, and B. Javidi, “Digital holographic data reconstruction with data compression,” in |

26. | MPEG-2 Video Codec (with Source Code), http://www.mpeg.org/MPEG/MSSG/#source |

27. | MPEG-2 Test Model 5, http://www.mpeg.org/MPEG/MSSG/tm5 |

**OCIS Codes**

(100.2000) Image processing : Digital image processing

(100.6890) Image processing : Three-dimensional image processing

(110.3000) Imaging systems : Image quality assessment

(110.6880) Imaging systems : Three-dimensional image acquisition

**ToC Category:**

Research Papers

**History**

Original Manuscript: March 2, 2004

Revised Manuscript: March 29, 2004

Published: April 19, 2004

**Citation**

Sekwon Yeom, Adrian Stern, and Bahram Javidi, "Compression of 3D color integral images," Opt. Express **12**, 1632-1642 (2004)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-12-8-1632

Sort: Journal | Reset

### References

- F. Okano, H. Hoshino, J. Arai, and I. Yuyama, �??Real-time pickup method for a three-dimensional image based on integral photography,�?? Appl. Opt. 36, 1598-1603 (1997). [CrossRef] [PubMed]
- F. Okano, H. Hoshino, J. Arai, and I. Yuyama, �??Three-dimensional video system based on integral photography,�?? Opt. Eng. 38, 1072-1077 (1997). [CrossRef]
- S. A. Benton, ed., Selected Papers on Three-Dimensional Displays (SPIE Optical Engineering Press, Bellingham, WA., 2001).
- D. H. McMahon and H. J. Caulfield, �??A technique for producing wide-angle holographic displays,�?? Appl. Opt. 9, 91-96 (1970). [CrossRef] [PubMed]
- T. Okoshi, �??Three-dimensional displays,�?? in Proceedings of IEEE 68, 548-564 (1980). [CrossRef]
- G. Lippmann, �??La photographic intergrale,�?? C. R. Acad. Sci. 146, 446-451 (1908).
- C. B. Burckhardt, �??Optimum parameters and resolution limitation of integral photography,�?? J. Opt. Soc. Am. 58, 71-76 (1968). [CrossRef]
- M. Martinez-Corral, C. Ibáñez-López, and G. Saavedra, �??Axial gain resolution in optical sectioning fluorescence microscopy by shaded-ring filters,�?? Opt. Express 11, 1740-1745 (2003). [CrossRef] [PubMed]
- P. Ambs, L. Bigue, Y. Fainman, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, �??Image reconstruction using electrooptic holography,�?? in Proceedings of IEEE Conference on the 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, 1 (IEEE, Piscataway, NJ., 2003), pp. 179-180.
- H. Arimoto and B. Javidi, �??Integral three-dimensional imaging with digital reconstruction,�?? Opt. Lett. 26, 157-159 (2001). [CrossRef]
- J. Jang and B. Javidi, �??Three-dimensional synthetic aperture integral imaging,�?? Opt. Lett. 27, 1144-1146 (2002). [CrossRef]
- A. Stern and B. Javidi, �??3-D computational synthetic aperture integral imaging (COMPSAII),�?? Opt. Express 11, 2446-2451 (2003), <a href="http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-19-2446">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-19-2446</a>. [CrossRef] [PubMed]
- Y. Frauel and B. Javidi, �??Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,�?? Appl. Opt. 41, 5488-5496 (2002). [CrossRef] [PubMed]
- M. C. Forman and A. Aggoun, �??Quantisation strategies for 3D-DCT-BASED compression of full parallax 3D images,�?? in Proceedings of Int. Conf. on Image processing and its applications, 6th, (Ireland, 1997), pp. 32-35. [CrossRef]
- R. Zaharia, A. Aggoun, and M. McCormick, �??Adaptive 3D-DCT compression algorithm for continuous parallax 3D integral imaging,�?? Signal Processing: Image Communication 17, 231-242 (2002). [CrossRef]
- J. S. Jang and B. Javidi, �??Compression of ray information in three-dimensional integral imaging using the Karhunen-Loeve transform,�?? submitted to Opt. Lett. (2004).
- V. Bhaskaran and K. Konstantinides, Image and video compression standards 2nd edition, (Kluwer Academic Publishers, 1997). [CrossRef]
- M. Rabbani, Fundamentals of Wavelet Image compression and the emerging JPEG-2000 standard VT080, (SPIE Press, Bellingham, WA., 2000).
- R. L. Joshi, M. Rabbani, and M. A. Lepley, �??Comparison of multiple compression cycle performance for JPEG and JPEG 2000,�?? in Applications of Digital Image Processing XXIII, A. G. Tescher, ed., Proc. SPIE 4115, 492-501 (2000).
- J. A. Saghri, A. G. Tescher, and A. M. Planinac, �??KLT/JPEG 2000 multispectral bandwidth compression with region-of-interest prioritization capability,�?? in Applications of Digital Image Processing XXVI, A. G. Tescher, ed., Proc. SPIE 5203, 226-235, (Nov 2003).
- T. J. Naughton, Y. Frauel, B. Javidi, and E. Tajahuerce, �??Compression of digital holograms for threedimensional object reconstruction and recognition,�?? Appl. Opt. 41, 4124-4132 (2002). [CrossRef] [PubMed]
- A. Mahalanobis and C. Daniell, �??Data compression and correlation filtering,�?? in Smart Imaging Systems (SPIE Press, 2001).
- M. Rabbani, Selected Papers on Image Coding and Compression, SPIE Milestone Series MS48 (SPIE Press, 1992).
- T. A. Welch, �??A technique for high performance data compression,�?? IEEE Computer 17, 8-19 (1984). [CrossRef]
- T. Nomura, A. Okazaki, M. Kameda, Y. Morimoto, and B. Javidi, �??Digital holographic data reconstruction with data compression,�?? in Algorithms and Systems for Optical Information Processing V, B. Javidi and D. Psaltis, eds., Proc. SPIE 4471, (2001). [CrossRef]
- MPEG-2 Video Codec (with Source Code), <a href="http://www.mpeg.org/MPEG/MSSG/#source">http://www.mpeg.org/MPEG/MSSG/#source</a>
- MPEG-2 Test Model 5, <a href="http://www.mpeg.org/MPEG/MSSG/tm5">http://www.mpeg.org/MPEG/MSSG/tm5</a>

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Figures

Fig. 1. |
Fig. 2. |
Fig. 3. |

Fig. 4. |
Fig. 5. |
Fig. 6. |

Fig. 7. |
Fig. 8. |
Fig. 9. |

Fig. 10. |
Fig. 11. |
Fig. 12. |

### Multimedia

Multimedia Files | Recommended Software |

» Media 1: AVI (6214 KB) | |

» Media 2: AVI (6214 KB) | |

» Media 3: AVI (6214 KB) | |

» Media 4: AVI (6214 KB) | |

» Media 5: AVI (6214 KB) | |

» Media 6: AVI (6214 KB) | |

» Media 7: AVI (6214 KB) | |

» Media 8: AVI (6214 KB) | |

» Media 9: AVI (6214 KB) | |

» Media 10: AVI (6214 KB) | |

» Media 11: AVI (6214 KB) | |

» Media 12: AVI (6214 KB) | |

» Media 13: AVI (2284 KB) | |

» Media 14: AVI (2284 KB) | |

» Media 15: AVI (2284 KB) | |

» Media 16: AVI (2284 KB) | |

» Media 17: AVI (2284 KB) | |

» Media 18: AVI (2284 KB) | |

» Media 19: AVI (2284 KB) | |

» Media 20: AVI (2284 KB) | |

» Media 21: AVI (2284 KB) | |

» Media 22: AVI (2284 KB) | |

» Media 23: AVI (2284 KB) | |

» Media 24: AVI (2284 KB) | |

» Media 25: AVI (2284 KB) |

« Previous Article | Next Article »

OSA is a member of CrossRef.