## Mixel camera––a new push-broom camera concept for high spatial resolution keystone-free hyperspectral imaging |

Optics Express, Vol. 21, Issue 9, pp. 11057-11077 (2013)

http://dx.doi.org/10.1364/OE.21.011057

Acrobat PDF (3359 KB)

### Abstract

Current high-resolution push-broom hyperspectral cameras introduce keystone errors to the captured data. Efforts to correct these errors in hardware severely limit the optical design, in particular with respect to light throughput and spatial resolution, while at the same time the residual keystone often remains large. The mixel camera solves this problem by combining a hardware component – an array of light mixing chambers – with a mathematical method that restores the hyperspectral data to its keystone-free form, based on the data that was recorded onto the sensor with large keystone. A Virtual Camera software, that was developed specifically for this purpose, was used to compare the performance of the mixel camera to traditional cameras that correct keystone in hardware. The mixel camera can collect at least four times more light than most current high-resolution hyperspectral cameras, and simulations have shown that the mixel camera will be photon-noise limited – even in bright light – with a significantly improved signal-to-noise ratio compared to traditional cameras. A prototype has been built and is being tested.

© 2013 OSA

## 1. Introduction

1. P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt. **39**(13), 2210–2220 (2000). [CrossRef] [PubMed]

2. P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE **8032**, 80320U, 80320U-12 (2011). [CrossRef]

4. P. Mouroulis, R. O. Green, and D. W. Wilson, “Optical design of a coastal ocean imaging spectrometer,” Opt. Express **16**(12), 9087–9096 (2008). [CrossRef] [PubMed]

## 2. The data restoring method

*E*

_{1}= 10,

*E*

_{2}= 30,

*E*

_{3}= 100, and

*E*

_{4}= 50. This gives the following values

*E*for the recorded sensor pixels:when we assume that the intensity distribution over each pixel in the scene is uniform. In a real scene this will not be the case, but we will show how this can be handled in Section 3.

^{R}*E*

_{1},

*E*

_{2},

*E*

_{3}, and

*E*

_{4}, giving the following values:

*E*

_{1}= 10,

*E*

_{2}= 30,

*E*

_{3}= 100, and

*E*

_{4}= 50, which are identical to the actual values in the scene pixels as given in Fig. 4(a). We have now managed to restore the true values of the 4 pixels in the scene, based only on the information about the values of the 5 recorded sensor pixels and the amount of keystone (1 pixel). In addition, we have assumed that the light distribution is uniform within each scene pixel, but as we will see below, it is sufficient that the light distribution is known (not necessarily uniform).

*N*pixels in the scene from

*M*recorded sensor pixels, where

*M*>

*N*. This situation is shown in Fig. 5.

*E*is the pixel value (energy) for scene pixel #

_{n}*n*,

*E*is the pixel value (energy) recorded in sensor pixel #

^{R}_{m}*m*,

*q*is the fraction of the energy contained within scene pixel #

_{mn}*n*that contributes to the value (energy) recorded in sensor pixel #

*m*,

*N*is the total number of pixels in the scene, and

*M*is the total number of pixels recorded on the sensor.

*q*depend on the keystone and point-spread function (see Section 6.4) of the system, and are measured during camera calibration/characterization. Typically, only two scene pixels contribute to each recorded sensor pixel, therefore most of the coefficients

_{mn}*q*are equal to zero. Equation (3) can then be written in matrix form:where the coefficients

_{mn}*q*are nonzero only along the diagonals and zero everywhere else.

_{mn}*E*. Note that the system has more equations than unknowns (

_{n}*M*>

*N*). In fact, each extra pixel of keystone gives one extra equation. For the ideal case when there is no noise in the system, the matrix system is compatible, i.e., can be solved. However, for a real system with noise, the system is overdetermined and an optimization method, such as for instance the least squares method, could be used to obtain the solution.

*M*/

*N*~1.1. In this case, the noise will typically be amplified by a factor ~1.3. However, since the mixel camera can collect considerably more light than traditional cameras, in addition to being free of keystone errors, a significant improvement in signal-to-noise ratio will still be obtained (see Section 6).

## 3. The light mixing chambers

*q*can be determined. This can be obtained by inserting an array of light mixing chambers into the camera slit. The purpose of the chambers is to mix the light that goes through each chamber as evenly as possible, so that the light distribution at the output of the chamber is independent of the light distribution at the input. The light distribution at the output of each chamber will then always be the same and therefore always known. Each mixing chamber is a miniaturized version of a light-pipe homogenizer [9

_{mn}9. M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE **6104**, 61040Q, 61040Q-10 (2006). [CrossRef]

10. H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE **86**(8), 1586–1593 (1998). [CrossRef]

*L*) of the mixing chamber can be written as:where

*w*is the width of the chamber,

*F*is the F-number of the foreoptics, and

*k*is a constant that is chosen in such a way that the back face of each chamber has as uniform illumination as possible. For a given

*k*, the mixing result at the backface of a chamber with length

*L*will be the same for any choice of F-number and width (

*w*). We have used the value

*k*= 2 in our simulations, which gives a very uniform light distribution while at the same time keeping the number of reflections as low as possible (half of the rays are reflected once, while the rest of the rays pass through the chamber without being reflected).

## 4. Optical design

11. P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE **4829**, 1048–1049 (2003). [CrossRef]

*M*/

*N*= 1.1, see Section 2), i.e., the mixels will be approximately 21.5 μm in size. Second, the F-number for the foreoptics will be much higher than the F1.25 from the system's specifications. The foreoptics for this relay should have F-number F3.8 ( = F1.25/0.33), which makes the optimum length of the mixing chambers equal to 163.4 μm, see Eq. (5). This is great news: larger mixing chambers are probably easier to manufacture, and the F3.8 foreoptics is definitely much easier to design, manufacture and align than an F1.25 one.

*after*having passed through the mixing chambers in the slit. Any misregistration errors that are introduced

*before*the slit will not be corrected for. It is therefore important to minimize the misregistration errors in the foreoptics. We have obtained this by using only reflective optics. In this case, the rays of all wavelengths follow precisely the same optical path and keystone cannot occur. In the relay optics, however, this is not possible since rays of different wavelengths are supposed to end up on different parts of the sensor. The rays must therefore follow different optical paths through the relay system (this is obtained by use of a dispersive element), inevitably introducing a certain amount of keystone.

## 5. Virtual camera simulations

15. Technical specifications on CIS 2521F (last accessed 20.04.2013), http://www.fairchildimaging.com/catalog/focal-plane-arrays/scmos/cis-2521f

*E*and standard deviation √

*E*[16]. Here

*E*is the number of photons in the noise-free signal. The resulting relative error in the signal due to photon noise has zero mean value and standard deviation:The relative error due to photon noise decreases when the signal increases. When the signal increases by a factor 2 the relative error decreases with a factor √2. Figure 12 shows the relative error (1σ) as a function of number of photons in the signal.

*dE*, is given by:where

*E*is the scene pixel value (number of photons) and

_{init}*E*is the calculated value of the same scene pixel after the signal has been processed by the camera. We can then find the standard deviation of

_{final}*dE*over the 320 pixels and we can also determine the maximum relative error. Both are important parameters when evaluating the performance of the cameras.

## 6. Camera performance

### 6.1 Misregistration errors

### 6.2 Errors when photon and readout noise are included

### 6.3 Bright light

### 6.4 Transitions between mixels

*actual*transitions and the

*assumed*transitions in the system.

*x*=

*x*

_{1}and ends at

*x*=

*x*

_{2}. The ‘sharp’ value of mixel #1 (i.e., the signal level of the part of the mixel that is not affected by the transition) is equal to

*E*

_{1}. The ‘sharp’ value of mixel #2 is equal to

*E*

_{2}.

*a*and

*c*are two constants:The transition has odd symmetry about its center (

*x*

_{0},

*E*

_{0}), with

*x*

_{0}= (

*x*

_{1}+

*x*

_{2})/2 and

*E*

_{0}= (

*E*

_{1}+

*E*

_{2})/2. In order to calculate the transition between the two mixels, we must know the width and position of the transition zone, i.e.,

*x*

_{1}and

*x*

_{2}, and the ‘sharp’ mixel values

*E*

_{1}and

*E*

_{2}. A transition that extends 30% into each mixel is here referred to as a 30% transition.

*assuming*35% transitions in each case. Figure 17 shows the resulting errors when the transitions are 50% and 40% (cases with 20% and 30% transitions have similar errors with opposite sign). We see that we get the largest errors when the deviation between the assumed transitions (35%) and the true transitions (50%) is the largest, see Fig. 17(a). The standard deviation is small (0.7%) but the peaks are quite large (up to about 6%). When the deviation is smaller (when the true transitions are 40%), the standard deviation decreases to about 0.2% and the largest peaks are only about 2%, see Fig. 17(b).

### 6.5 Misalignment in the relative position between the mixel array and the sensor pixels

## 7. Camera calibration

*q*are correct. These coefficients describe the geometry of the mixel array image on the sensor, as well as the PSF of the relay optics, and must be determined precisely.

_{mn}## 8. Conclusion

*removed*from the hyperspectral data.

*not*be able to benefit from a camera with practically perfect keystone and PSF corrections, high signal-to-noise ratio and very high spatial resolution.

## References and links

1. | P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt. |

2. | P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE |

3. | P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE |

4. | P. Mouroulis, R. O. Green, and D. W. Wilson, “Optical design of a coastal ocean imaging spectrometer,” Opt. Express |

5. | G. Høye and A. Fridman, “Hyperspektralt kamera og metode for å ta opp hyperspektrale data,” Norwegian patent application number 20111001. |

6. | G. Høye and A. Fridman, “Hyperspectral camera and method for acquiring hyperspectral data,” PCT international patent application number PCT/NO2012/050132. |

7. | G. Høye and A. Fridman, “A method for restoring data in a hyperspectral imaging system with large keystone without loss of spatial resolution,” FFI-rapport 2009/01351 (2009), declassified on January 28th 2013. |

8. | A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published). |

9. | M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE |

10. | H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE |

11. | P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE |

12. | |

13. | R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE |

14. | G. Høye and A. Fridman, “Performance analysis of the proposed new restoring camera for hyperspectral imaging,” FFI-rapport 2010/02383 (2010), to be declassified. |

15. | Technical specifications on CIS 2521F (last accessed 20.04.2013), http://www.fairchildimaging.com/catalog/focal-plane-arrays/scmos/cis-2521f |

16. | B. E. A. Salech and M. C. Teich, |

**OCIS Codes**

(100.3190) Image processing : Inverse problems

(120.0280) Instrumentation, measurement, and metrology : Remote sensing and sensors

(120.6200) Instrumentation, measurement, and metrology : Spectrometers and spectroscopic instrumentation

(220.4830) Optical design and fabrication : Systems design

(300.6190) Spectroscopy : Spectrometers

(110.4234) Imaging systems : Multispectral and hyperspectral imaging

**ToC Category:**

Imaging Systems

**History**

Original Manuscript: February 21, 2013

Revised Manuscript: April 22, 2013

Manuscript Accepted: April 23, 2013

Published: April 29, 2013

**Citation**

Gudrun Høye and Andrei Fridman, "Mixel camera––a new push-broom camera concept for high spatial resolution keystone-free hyperspectral imaging," Opt. Express **21**, 11057-11077 (2013)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-21-9-11057

Sort: Year | Journal | Reset

### References

- P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt.39(13), 2210–2220 (2000). [CrossRef] [PubMed]
- P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011). [CrossRef]
- P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012). [CrossRef]
- P. Mouroulis, R. O. Green, and D. W. Wilson, “Optical design of a coastal ocean imaging spectrometer,” Opt. Express16(12), 9087–9096 (2008). [CrossRef] [PubMed]
- G. Høye and A. Fridman, “Hyperspektralt kamera og metode for å ta opp hyperspektrale data,” Norwegian patent application number 20111001.
- G. Høye and A. Fridman, “Hyperspectral camera and method for acquiring hyperspectral data,” PCT international patent application number PCT/NO2012/050132.
- G. Høye and A. Fridman, “A method for restoring data in a hyperspectral imaging system with large keystone without loss of spatial resolution,” FFI-rapport 2009/01351 (2009), declassified on January 28th 2013.
- A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published).
- M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006). [CrossRef]
- H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE86(8), 1586–1593 (1998). [CrossRef]
- P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE4829, 1048–1049 (2003). [CrossRef]
- http://www.hyspex.no/products/hyspex/vnir1600.php
- R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE7812, 78120M, 78120M-13 (2010). [CrossRef]
- G. Høye and A. Fridman, “Performance analysis of the proposed new restoring camera for hyperspectral imaging,” FFI-rapport 2010/02383 (2010), to be declassified.
- Technical specifications on CIS 2521F (last accessed 20.04.2013), http://www.fairchildimaging.com/catalog/focal-plane-arrays/scmos/cis-2521f
- B. E. A. Salech and M. C. Teich, Fundamentals of Photonics (John Wiley & Sons Inc., 1991).

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Figures

Fig. 1 |
Fig. 2 |
Fig. 3 |

Fig. 4 |
Fig. 5 |
Fig. 6 |

Fig. 7 |
Fig. 8 |
Fig. 9 |

Fig. 10 |
Fig. 11 |
Fig. 12 |

Fig. 13 |
Fig. 14 |
Fig. 15 |

Fig. 16 |
Fig. 17 |
Fig. 18 |

Fig. 19 |
Fig. 20 |
Fig. 21 |

« Previous Article | Next Article »

OSA is a member of CrossRef.