OSA's Digital Library

Optics Express

Optics Express

  • Editor: Andrew M. Weiner
  • Vol. 22, Iss. 10 — May. 19, 2014
  • pp: 12160–12176

Superresolution microscope image reconstruction by spatiotemporal object decomposition and association: application in resolving t-tubule structure in skeletal muscle

Mingzhai Sun, Jiaqing Huang, Filiz Bunyak, Kristyn Gumpper, Gejing De, Matthew Sermersheim, George Liu, Pei-Hui Lin, Kannappan Palaniappan, and Jianjie Ma  »View Author Affiliations

Optics Express, Vol. 22, Issue 10, pp. 12160-12176 (2014)

View Full Text Article

Enhanced HTML    Acrobat PDF (3148 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



One key factor that limits resolution of single-molecule superresolution microscopy relates to the localization accuracy of the activated emitters, which is usually deteriorated by two factors. One originates from the background noise due to out-of-focus signals, sample auto-fluorescence, and camera acquisition noise; and the other is due to the low photon count of emitters at a single frame. With fast acquisition rate, the activated emitters can last multiple frames before they transiently switch off or permanently bleach. Effectively incorporating the temporal information of these emitters is critical to improve the spatial resolution. However, majority of the existing reconstruction algorithms locate the emitters frame by frame, discarding or underusing the temporal information. Here we present a new image reconstruction algorithm based on tracklets, short trajectories of the same objects. We improve the localization accuracy by associating the same emitters from multiple frames to form tracklets and by aggregating signals to enhance the signal to noise ratio. We also introduce a weighted mean-shift algorithm (WMS) to automatically detect the number of modes (emitters) in overlapping regions of tracklets so that not only well-separated single emitters but also individual emitters within multi-emitter groups can be identified and tracked. In combination with a maximum likelihood estimator method (MLE), we are able to resolve low to medium density of overlapping emitters with improved localization accuracy. We evaluate the performance of our method with both synthetic and experimental data, and show that the tracklet-based reconstruction is superior in localization accuracy, particularly for weak signals embedded in a strong background. Using this method, for the first time, we resolve the transverse tubule structure of the mammalian skeletal muscle.

© 2014 Optical Society of America

OCIS Codes
(000.1430) General : Biology and medicine
(100.0100) Image processing : Image processing
(100.6640) Image processing : Superresolution
(180.0180) Microscopy : Microscopy
(110.4155) Imaging systems : Multiframe image processing

ToC Category:
Image Processing

Original Manuscript: December 26, 2013
Revised Manuscript: March 5, 2014
Manuscript Accepted: April 1, 2014
Published: May 13, 2014

Virtual Issues
Vol. 9, Iss. 7 Virtual Journal for Biomedical Optics

Mingzhai Sun, Jiaqing Huang, Filiz Bunyak, Kristyn Gumpper, Gejing De, Matthew Sermersheim, George Liu, Pei-Hui Lin, Kannappan Palaniappan, and Jianjie Ma, "Superresolution microscope image reconstruction by spatiotemporal object decomposition and association: application in resolving t-tubule structure in skeletal muscle," Opt. Express 22, 12160-12176 (2014)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light, 7th ed. (Cambridge University, 1975).
  2. S. W. Hell, “Far-field optical nanoscopy,” Science 316(5828), 1153–1158 (2007). [CrossRef] [PubMed]
  3. B. Huang, M. Bates, X. Zhuang, “Super-resolution fluorescence microscopy,” Annu. Rev. Biochem. 78(1), 993–1016 (2009). [CrossRef] [PubMed]
  4. G. Patterson, M. Davidson, S. Manley, J. Lippincott-Schwartz, “Superresolution imaging using single-molecule localization,” Annu. Rev. Phys. Chem. 61(1), 345–367 (2010). [CrossRef] [PubMed]
  5. K. R. Chi, “Microscopy: Ever-increasing resolution,” Nature 462(7273), 675–678 (2009). [CrossRef] [PubMed]
  6. S. W. Hell, J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19(11), 780–782 (1994). [CrossRef] [PubMed]
  7. M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198(2), 82–87 (2000). [CrossRef] [PubMed]
  8. J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008). [CrossRef] [PubMed]
  9. T. Dertinger, R. Colyer, G. Iyer, S. Weiss, J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009). [CrossRef] [PubMed]
  10. M. J. Rust, M. Bates, X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006). [CrossRef] [PubMed]
  11. M. Heilemann, S. van de Linde, M. Schuttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem-Ger. Edit. 47(33), 6172–6176 (2008). [CrossRef]
  12. M. Heilemann, S. van de Linde, A. Mukherjee, M. Sauer, “Super-resolution imaging with small organic fluorophores,” Angew. Chem. Int. Ed. Engl. 48(37), 6903–6908 (2009). [CrossRef] [PubMed]
  13. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006). [CrossRef] [PubMed]
  14. S. T. Hess, T. P. K. Girirajan, M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006). [CrossRef] [PubMed]
  15. J. C. Vaughan, S. Jia, X. Zhuang, “Ultrabright photoactivatable fluorophores created by reductive caging,” Nat. Methods 9(12), 1181–1184 (2012). [CrossRef] [PubMed]
  16. A. Pertsinidis, K. Mukherjee, M. Sharma, Z. P. Pang, S. R. Park, Y. Zhang, A. T. Brunger, T. C. Südhof, S. Chu, “Ultrahigh-resolution imaging reveals formation of neuronal SNARE/Munc18 complexes in situ,” Proc. Natl. Acad. Sci. U.S.A. 110(30), E2812–E2820 (2013). [CrossRef] [PubMed]
  17. A. Pertsinidis, Y. Zhang, S. Chu, “Subnanometre single-molecule localization, registration and distance measurements,” Nature 466(7306), 647–651 (2010). [CrossRef] [PubMed]
  18. H. Shroff, C. G. Galbraith, J. A. Galbraith, E. Betzig, “Live-cell photoactivated localization microscopy of nanoscale adhesion dynamics,” Nat. Methods 5(5), 417–423 (2008). [CrossRef] [PubMed]
  19. S. A. Jones, S. H. Shim, J. He, X. Zhuang, “Fast, three-dimensional super-resolution imaging of live cells,” Nat. Methods 8(6), 499–505 (2011). [CrossRef] [PubMed]
  20. B. Zhang, J. Zerubia, J. C. Olivo-Marin, “Gaussian approximations of fluorescence microscope point-spread function models,” Appl. Opt. 46(10), 1819–1829 (2007). [CrossRef] [PubMed]
  21. S. Stallinga, B. Rieger, “Accuracy of the gaussian point spread function model in 2D localization microscopy,” Opt. Express 18(24), 24461–24476 (2010). [CrossRef] [PubMed]
  22. F. Huang, S. L. Schwartz, J. M. Byars, K. A. Lidke, “Simultaneous multiple-emitter fitting for single molecule super-resolution imaging,” Biomed. Opt. Express 2(5), 1377–1393 (2011). [CrossRef] [PubMed]
  23. R. E. Thompson, D. R. Larson, W. W. Webb, “Precise nanometer localization analysis for individual fluorescent probes,” Biophys. J. 82(5), 2775–2783 (2002). [CrossRef] [PubMed]
  24. D. Sage, F. R. Neumann, F. Hediger, S. M. Gasser, M. Unser, “Automatic tracking of individual fluorescence particles: application to the study of chromosome dynamics,” IEEE Trans. Image Process. 14(9), 1372–1383 (2005). [CrossRef] [PubMed]
  25. J. L. Starck, F. Murtagh, A. Bijaoui, “Multiresolution support applied to image filtering and restoration,” Graph Model Im. Pro. C. 57(5), 420–431 (1995). [CrossRef]
  26. K. Fukunaga, L. Hostetler, “The estimation of the gradient of a density function, with applications in pattern recognition,” IEEE Trans. Inf. Theory 21(1), 32–40 (1975). [CrossRef]
  27. Y. Cheng, “Mean shift, mode seeking, and clustering,” IEEE T. Pattern Anal. 17(8), 790–799 (1995). [CrossRef]
  28. K. L. Wu, M. S. Yang, “Mean shift-based clustering,” Pattern Recognit. 40(11), 3035–3052 (2007). [CrossRef]
  29. E. Meijering, I. Smal, O. Dzyubachyk, and J.-C. Olivo-Marin, “Time-lapse imaging,” Microsc. Img. Proc. 401-440 (2008).
  30. E. Meijering, O. Dzyubachyk, I. Smal, “Methods for cell and particle tracking,” Methods Enzymol. 504, 183–200 (2012). [CrossRef] [PubMed]
  31. K. Jaqaman, D. Loerke, M. Mettlen, H. Kuwata, S. Grinstein, S. L. Schmid, G. Danuser, “Robust single-particle tracking in live-cell time-lapse sequences,” Nat. Methods 5(8), 695–702 (2008). [CrossRef] [PubMed]
  32. F. H. Li, X. B. Zhou, J. W. Ma, S. T. C. Wong, “Multiple nuclei tracking using integer programming for quantitative cancer cell cycle analysis,” IEEE Trans. Med. Imaging 29(1), 96–105 (2010). [CrossRef] [PubMed]
  33. T. Kanade, Z. Yin, R. Bise, S. Huh, and S. Eom, “Cell image analysis: algorithms, system and applications,” IEEE Work. App. Comp. (2011).
  34. F. Bunyak, K. Palaniappan, S. K. Nath, T. L. Baskin, and G. Dong, “Quantitative cell motility for in vitro wound healing using level set-based active contour tracking,” I. S. Biomed. Imaging 1040–1043 (2006).
  35. K. Palaniappan, F. Bunyak, S. Nath, and J. Goffeney, “Parallel Processing Strategies for Cell Motility and Shape Analysis,” High-Throughput Image Reconstruction and Analysis, Artech House Publisher, 39–87 (2009).
  36. B. Huang, W. Wang, M. Bates, X. Zhuang, “Three-Dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008). [CrossRef] [PubMed]
  37. M. Bates, S. A. Jones, X. Zhuang, “Stochastic optical reconstruction microscopy (STORM): a method for superresolution fluorescence imaging,” Cold Spring Harb Protoc 2013(6), 498–520 (2013). [CrossRef] [PubMed]
  38. H. Takeshima, M. Shimuta, S. Komazaki, K. Ohmi, M. Nishi, M. Iino, A. Miyata, K. Kangawa, “Mitsugumin29, a novel synaptophysin family member from the triad junction in skeletal muscle,” Biochem. J. 331(Pt 1), 317–322 (1998). [PubMed]
  39. S. J. Holden, S. Uphoff, A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011). [CrossRef] [PubMed]
  40. L. Zhu, W. Zhang, D. Elnatan, B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012). [CrossRef] [PubMed]
  41. T. Quan, H. Zhu, X. Liu, Y. Liu, J. Ding, S. Zeng, Z. L. Huang, “High-density localization of active molecules using Structured Sparse Model and Bayesian Information Criterion,” Opt. Express 19(18), 16963–16974 (2011). [CrossRef] [PubMed]
  42. Y. Wang, T. Quan, S. Zeng, Z. L. Huang, “PALMER: a method capable of parallel localization of multiple emitters for high-density localization microscopy,” Opt. Express 20(14), 16039–16049 (2012). [CrossRef] [PubMed]
  43. H. P. Babcock, J. R. Moffitt, Y. Cao, X. Zhuang, “Fast compressed sensing analysis for super-resolution imaging using L1-homotopy,” Opt. Express 21(23), 28583–28596 (2013). [CrossRef] [PubMed]
  44. E. A. Mukamel, H. Babcock, X. Zhuang, “Statistical deconvolution for superresolution fluorescence microscopy,” Biophys. J. 102(10), 2391–2400 (2012). [CrossRef] [PubMed]
  45. Y. Deng, P. Coen, M. Sun, J. W. Shaevitz, “Efficient multiple object tracking using mutually repulsive active membranes,” PLoS ONE 8(6), e65769 (2013). [CrossRef] [PubMed]
  46. P. Dedecker, S. Duwé, R. K. Neely, J. Zhang, “Localizer: fast, accurate, open-source, and modular software package for superresolution microscopy,” J. Biomed. Opt. 17(12), 126008 (2012). [CrossRef] [PubMed]
  47. C. Franzini-Armstrong, J. E. Heuser, T. S. Reese, A. P. Somlyo, A. V. Somlyo, “T-tubule swelling in hypertonic solutions: a freeze substitution study,” J. Physiol. 283, 133–140 (1978). [PubMed]
  48. E. Wagner, M. A. Lauterbach, T. Kohl, V. Westphal, G. S. B. Williams, J. H. Steinbrecher, J. H. Streich, B. Korff, H. T. M. Tuan, B. Hagen, S. Luther, G. Hasenfuss, U. Parlitz, M. S. Jafri, S. W. Hell, W. J. Lederer, S. E. Lehnart, “Stimulated emission depletion live-cell super-resolution imaging shows proliferative remodeling of T-tubule membrane structures after myocardial infarction,” Circ. Res. 111(4), 402–414 (2012). [CrossRef] [PubMed]
  49. S. Komazaki, M. Nishi, K. Kangawa, H. Takeshima, “Immunolocalization of mitsugumin29 in developing skeletal muscle and effects of the protein expressed in amphibian embryonic cells,” Dev. Dyn. 215(2), 87–95 (1999). [CrossRef] [PubMed]
  50. Z. Pan, D. Yang, R. Y. Nagaraj, T. A. Nosek, M. Nishi, H. Takeshima, H. Cheng, J. Ma, “Dysfunction of store-operated calcium channel in muscle cells lacking mg29,” Nat. Cell Biol. 4(5), 379–383 (2002). [CrossRef] [PubMed]
  51. S. Cox, E. Rosten, J. Monypenny, T. Jovanovic-Talisman, D. T. Burnette, J. Lippincott-Schwartz, G. E. Jones, R. Heintzmann, “Bayesian localization microscopy reveals nanoscale podosome dynamics,” Nat. Methods 9(2), 195–200 (2011). [CrossRef] [PubMed]
  52. E. Rosten, G. E. Jones, S. Cox, “ImageJ plug-in for Bayesian analysis of blinking and bleaching,” Nat. Methods 10(2), 97–98 (2013). [CrossRef] [PubMed]
  53. Y. S. Hu, X. Nan, P. Sengupta, J. Lippincott-Schwartz, H. Cang, “Accelerating 3B single-molecule super-resolution microscopy with cloud computing,” Nat. Methods 10(2), 96–97 (2013). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited