DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning

PLUTO / PLUTO-2 Spatial Light Modulators
Deep Learning / Neuronal Network Microscopy
Published on:
Authors: Nehme, Elias; Freedman, Daniel; Gordon, Racheli; Ferdman, Boris; Weiss, Lucien E.; Alalouf, Onit; Orange, Reut; Michaeli, Tomer & Shechtman, Yoav
Abstract:

“Localization microscopy is an imaging technique in which the positions of individual nanoscale point emitters (e.g. fluorescent molecules) are determined at high precision from their images. This is the key ingredient in single/multiple-particle-tracking and several super-resolution microscopy approaches. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF). The PSF is engineered using additional optical elements to vary distinctively with the depth of the point-source. However, localizing multiple adjacent emitters in 3D poses a significant algorithmic challenge, due to the lateral overlap of their PSFs. Here, we train a neural network to receive an image containing densely overlapping PSFs of multiple emitters over a large axial range and output a list of their 3D positions. Furthermore, we then use the network to design the optimal PSF for the multi-emitter case. We demonstrate our approach numerically as well as experimentally by 3D STORM imaging of mitochondria, and volumetric imaging of dozens of fluorescently-labeled telomeres occupying a mammalian nucleus in a single snapshot.”

Open Access

Publication: Nature Methods 17
Issue/Year: Nature Methods 17 (2020) 734-740, Volume 17; Number 7; Pages 734–740; 2019
DOI: 10.1038/s41592-020-0853-5
Link: https://doi.org/10.1038/s41592-020-0853-5

Related Papers

GAEA / GAEA-2 Spatial Light Modulators
Authors:Ekaterina A. Savchenkova, Andrey S. Ovchinnikov, Vladislav G. Rodin, Rostislav S. Starikov, Nikolay N. Evtikhiev, Pavel A. Cheremkhin

Adaptive non-iterative histogram-based hologram quantization

Applications: Deep Learning / Neuronal Network,Digital-/ Computer Holography/ CGH
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Jixiang Wang, Xin Xu, Hong Ye, Xin Zhang, and Guohua Shi

Efficient generation of high-contrast Bessel light sheets: a flexible method for improved imaging

Applications: Beam Shaping / Beam Steering,Bessel-/ Airy Beam Generation,Imaging/ Image Processing,Microscopy
LC 2012 PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Jiaosheng Li, Yifei Chen, Tianyun Liu, Bo Wu & Qinnan Zhang

Single-pixel Fresnel incoherent correlation holography compressed imaging using a Trumpet network

Applications: Deep Learning / Neuronal Network,Digital-/ Computer Holography/ CGH,Imaging/ Image Processing
LETO / LETO-3 Spatial Light Modulators
Authors:Yan Liu, Jianwei Qin, Yanbing Liu, Yuanan Liu, Xun Liu, Fangwei Ye, and Wei Li

Optical Fourier convolutional neural network with high efficiency in image classification

Applications: Deep Learning / Neuronal Network
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Juntao Hu, Wenjun Wei, Xinzhong Li, and Yixian Qian

Generation of polygonal non-diffracting beams via angular spectral phases

Applications: Beam Shaping / Beam Steering,Higher Order Modes / Optical Vortex / OAM,Microscopy
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Kyriakos Skarsoulis, Konstantinos Makris, Christophe Moser, and Demetri Psaltis

Ptychographic imaging with a fiber endoscope via wavelength scanning

Applications: Deep Learning / Neuronal Network,Imaging/ Image Processing
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Wangting Zhou, Hui Xie, Kezhou Li, Zhiyuan Sun, Jiangshan He, Zhen Yuan, Xunbin Wei, Xueli Chen

Flexible depth-of-focus, depth-invariant resolution photoacoustic microscopy with Airy beam

Applications: Bessel-/ Airy Beam Generation,Microscopy,Pulse Application / -Shaping
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Shile Yang, Yuecheng Shen, Jiawei Luo, Zhengyang Wang, Daixuan Wu, Jiaming Liang, Zhiling Zhang, Dalong Qi, Yunhua Yao, Lianzhong Deng, Bin Zhang, Zhenrong Sun, and Shian Zhang

Neural network enabled fringe projection through scattering media

Applications: Deep Learning / Neuronal Network,Turbid-/ Opaque Media /Multi Scattering
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Siyu Gao, Xiaoyun Liu, Yonghao Chen, Jinyang Jiang, Ying Liu, Tengfei Chai and Yueqiu Jiang

Oceanic turbulence parameters recognition based on convolutional neural network

Applications: Adaptive Optics / Wavefront Control,Deep Learning / Neuronal Network,Optical Communication
LETO / LETO-3 Spatial Light Modulators
Authors:Manuel Hüpfel and Gerd Ulrich Nienhaus

Beam shaping in light-sheet microscopy: an experimental analysis

Applications: Beam Shaping / Beam Steering,Microscopy
LETO / LETO-3 Spatial Light Modulators
Authors:Manu Gopakumar, Gun-Yeal Lee, Suyeon Choi, Brian Chao, Yifan Peng, Jonghyun Kim & Gordon Wetzstein

Full-colour 3D holographic augmented-reality displays with metasurface waveguides

Applications: AR/VR/MR / Holographic Display,Deep Learning / Neuronal Network
GAEA / GAEA-2 Spatial Light Modulators
Authors:Xiaofei Li, Xin Liu, Quanying Wu, Jun Zeng, Yangjian Cai, Sergey A. Ponomarenko, Chunhao Liang

Prime number factorization with light beams carrying orbital angular momentum

Applications: Deep Learning / Neuronal Network,Digital-/ Computer Holography/ CGH,Higher Order Modes / Optical Vortex / OAM
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Rui Xiong, Xiangchao Zhang, Xinyang Ma, Leheng Li, Zhanghao Ni, and Xiangqian Jiang

Adaptive coded phase mask design and high-quality image reconstruction for interference-less coded aperture correlation holography

Applications: Deep Learning / Neuronal Network,Digital-/ Computer Holography/ CGH,Imaging/ Image Processing
HES / HEO 6001 PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Omri Wengrowicz, Alex Bronstein, and Oren Cohen

Unsupervised physics-informed deep learning-based reconstruction for time-resolved imaging by multiplexed ptychography

Applications: Deep Learning / Neuronal Network
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Pan, Tuqiang / Ye, Jianwei / Liu, Haotian / Zhang, Fan / Xu, Pengbai / Xu, Ou / Xu, Yi / Qin, Yuwen

Non-orthogonal optical multiplexing empowered by deep learning

Applications: Complex Modulation,Deep Learning / Neuronal Network,Optical Communication
GAEA / GAEA-2 PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Fang, Xinyuan / Hu, Xiaonan / Li, Baoli / Su, Hang / Cheng, Ke / Luan, Haitao / Gu, Min

Orbital angular momentum-mediated machine learning for high-accuracy mode-feature encoding

Applications: Deep Learning / Neuronal Network,Higher Order Modes / Optical Vortex / OAM
PLUTO / PLUTO-2 Spatial Light Modulators
Authors: Yongqiang Chen, Chenggui Luo, Shiqi Wang, Yanping Li, Binglin Shen, Rui Hu, Junle Qu, Liwei Liu

Rapid, high-contrast, and steady volumetric imaging with Bessel-beam-based two-photon fluorescence microscopy

Applications: Microscopy
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Zejian Ren, Xu Yan, Kai Wen, Huijin Chen, Elnur Hajiyev, Chengdong He, Gyu-Boong Jo

Creation of a tweezer array for cold atoms utilizing a generative neural network

Applications: Deep Learning / Neuronal Network,Optical Trapping /-Tweezers
LETO / LETO-3 Spatial Light Modulators
Authors: Siqing Dai, Andrei Kobitski, Amirhossein Barati Sedeh, Süheyla Eroğlu-Kayıkçı, Lennart Hilbert, and G. Ulrich Nienhaus

Photon-Efficient Aberration Correction for 3D-STED Imaging of Thick Biological Specimens Using Sensorless Adaptive Optics

Applications: Adaptive Optics / Wavefront Control,Beam Shaping / Beam Steering,Microscopy
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Jawahar Prabhakar Desai, Joseph Rosen

Resolution-enhanced imaging by interferenceless coded aperture correlation holography with two competing methods of phase mask synthesis

Applications: Microscopy
HES / HEO 6001 PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Zicheng Huang, Ziyu Gu, Mengyang Shi, Yesheng Gao, and Xingzhao Liu

OP-FCNN: an optronic fully convolutional neural network for imaging through scattering media

Applications: Deep Learning / Neuronal Network,Turbid-/ Opaque Media /Multi Scattering
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Krishnendu Samanta, Anupriya Tiwari, Samsheerali P.T., Joby Joseph

Improving resolution in two orthogonal orientations from a single-shot digital holographic microscopy

Applications: Microscopy
PLUTO / PLUTO-2 Spatial Light Modulators
Authors: Grace Kuo , Florian Schiffers , Douglas Lanman , Oliver Cossairt , Nathan Matsuda

Multisource Holography

Applications: AR/VR/MR / Holographic Display,Deep Learning / Neuronal Network,Digital-/ Computer Holography/ CGH
PLUTO / PLUTO-2 Spatial Light Modulators
Authors:Zheyu Wu, Ran Gao, Sitong Zhou, Fei Wang, Zhipei Li, Huan Chang, Dong Guo, Xiangjun Xin, Qi Zhang, Feng Tian, Qiang Wu

Robust Super-Resolution Image Transmission Based on a Ring Core Fiber with Orbital Angular Momentum

Applications: Deep Learning / Neuronal Network,Higher Order Modes / Optical Vortex / OAM,Imaging/ Image Processing