Comparison of state-of-the-art Computer Generated Holography algorithms and a machine learning approach

Author(s):

Madsen, Andreas Erik Gejl; Eriksen, René Lynge & Glückstad, Jesper

Abstract:

“This work studies the use of machine learning and, in particular, a Convolutional Neural Network (CNN) to generate digital holograms and how such a network compares to state-of-the-art iterative methods, both in terms of reconstruction quality and computation time. Since CNNs only require a single pass through the network by a target image to generate a result, and not tens or hundreds of expensive iterations as in the iterative methods, they may be able to accomplish real-time hologram generation; an ability that could open the technology to proper commercial use.

In this work, a CNN built on the UNet architecture, capable of hologram generation, is presented. The network is trained on 4096 images of varying spatial frequencies, both user-generated and from the DIV2K dataset. It is compared to the most common iterative method for hologram generation, namely the Gerchberg–Saxton(GS) algorithm and its modern and improved implementations. In reconstruction quality, the neural network outperforms the original implementation of GS when evaluating Mean Square Error (MSE), geometric error (GE), Structural Similarity Index Measurement (SSIM), and Peak Signal-Noise Ratio (PSNR) of 64 unseen test images. However, on the same test images, the network lacks behind the modern, optimized GS implementations in all error and accuracy measurements. The network does, however, achieve these results at a rate 70–280 times faster than the iterative methods, depending on the particular implementation of the GS algorithm, which corresponds to a possible generation rate of the network of 32 FPS on average.”

Link to Publications Page

Publication: Optics Communications
Issue/Year: Optics Communications, Pages 127590; 2021
DOI: 10.1016/j.optcom.2021.127590

All-optical image identification with programmable matrix transformation

Author(s):

Li, Shikang; Ni, Baohua; Feng, Xue; Cui, Kaiyu; Liu, Fang; Zhang, Wei & Huang, Yidong

Abstract:

“An optical neural network is proposed and demonstrated with programmable matrix transformation and nonlinear activation function of photodetection (square-law detection). Based on discrete phase-coherent spatial modes, the dimensionality of programmable optical matrix operations is 30∼37, which is implemented by spatial light modulators. With this architecture, all-optical classification tasks of handwritten digits, objects and depth images are performed. The accuracy values of 85.0% and 81.0% are experimentally evaluated for MNIST (Modified National Institute of Standards and Technology) digit and MNIST fashion tasks, respectively. Due to the parallel nature of matrix multiplication, the processing speed of our proposed architecture is potentially as high as 7.4∼74 T FLOPs per second (with 10∼100 GHz detector).”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 29; Number 17; Pages 26474; 2021
DOI: 10.1364/oe.430281

Fourier horizontal parallax only computer and digital holography of large size

Author(s):

Kozacki, Tomasz; Martinez-Carranza, Juan; kuko{l}owicz, Rafa{l} & Chlipa{l}a, Maksymilian

Abstract:

“Registration and reconstruction of high-quality digital holograms with a large view angle are intensive computer tasks since they require the space-bandwidth product (SBP) of the order of tens of gigapixels or more. This massive use of SBP severely affects the storing and manipulation of digital holograms. In order to reduce the computer burden, this work focuses on the generation and reconstruction of very large horizontal parallax only digital holograms (HPO-DHs). It is shown that these types of holograms can preserve high quality and large view angle in x direction while keeping a low use of SBP. This work first proposes a numerical technique that allows calculating very large HPO-DHs with large pixel size by merging the Fourier holography and phase added stereogram algorithm. The generated Fourier HPO-DHs enable accurate storing of holographic data from 3D objects. To decode the information contained in these Fourier HPO-DHs (FHPO-DHs), a novel angular spectrum (AS) technique that provides an efficient use of the SBP for reconstruction is proposed. Our reconstruction technique, which is called compact space bandwidth AS (CSW-AS), makes use of cylindrical parabolic waves that solve sampling issues of FHPO-DHs and AS. Moreover, the CSW-AS allows for implementing zero-padding for accurate wavefield reconstructions. Hence, suppression of aliased components and high spatial resolution is possible. Notably, the imaging chain of Fourier HPO-DH enables efficient calculation, reconstruction and storing of HPO holograms of large size. Finally, the accuracy and utility of the developed technique is proved by both numerical and optical reconstructions.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 29; Number 12; Pages 18173; 2021
DOI: 10.1364/oe.421186

Comparative study on resolution enhancements in fluorescence-structured illumination Fresnel incoherent correlation holography

Author(s):

Jeon, Philjun; Kim, Jongwu; Lee, Heejung; Kwon, Hyuk-Sang & young Kim, Dug

Abstract:

“Fresnel incoherent correlation holography (FINCH) is a new approach for incoherent holography, which also has enhancement in the transverse resolution. Structured illumination microscopy (SIM) is another promising super-resolution technique. SI-FINCH, the combination of SIM and FINCH, has been demonstrated lately for scattering objects. In this study, we extended the application of SI-FINCH toward fluorescent microscopy. We have built a versatile multimodal microscopy system that can obtain images of four different imaging schemes: conventional fluorescence microscopy, FINCH, SIM, and SI-FINCH. Resolution enhancements were demonstrated by comparing the point spread functions (PSFs) of the four different imaging systems by using fluorescence beads of 1-μm diameter.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 29; Number 6; Pages 9231; 2021
DOI: 10.1364/oe.417206

Holographic contact lens display that provides focusable images for eyes

Author(s):

Sano, Junpei & Takaki, Yasuhiro

Abstract:

“In this paper, we propose a holographic image generation technique for contact lens displays. The proposed technique employs a phase-only spatial light modulator (SLM), a holographic optical element (HOE) backlight, and a polarizer. The proposed holographic technique can generate 3D images apart from the contact lens displays. Therefore, the eyes can focus on the 3D images while simultaneously observing the real scene through the phase-only SLM and the HOE backlight, which provides see-through capability. A bench-top experimental system was constructed to verify the far-distance image generation capability and see-through function.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 29; Number 7; Pages 10568; 2021
DOI: 10.1364/oe.419604

Towards real-time photorealistic 3D holography with deep neural networks

Author(s):

Shi, Liang; Li, Beichen; Kim, Changil; Kellnhofer, Petr & Matusik, Wojciech

Abstract:

“The ability to present three-dimensional (3D) scenes with continuous depth sensation has a profound impact on virtual and augmented reality, human–computer interaction, education and training. Computer-generated holography (CGH) enables high-spatio-angular-resolution 3D projection via numerical simulation of diffraction and interference1. Yet, existing physically based methods fail to produce holograms with both per-pixel focal control and accurate occlusion. The computationally taxing Fresnel diffraction simulation further places an explicit trade-off between image quality and runtime, making dynamic holography impractical. Here we demonstrate a deep-learning-based CGH pipeline capable of synthesizing a photorealistic colour 3D hologram from a single RGB-depth image in real time. Our convolutional neural network (CNN) is extremely memory efficient (below 620 kilobytes) and runs at 60 hertz for a resolution of 1,920 × 1,080 pixels on a single consumer-grade graphics processing unit. Leveraging low-power on-device artificial intelligence acceleration chips, our CNN also runs interactively on mobile (iPhone 11 Pro at 1.1 hertz) and edge (Google Edge TPU at 2.0 hertz) devices, promising real-time performance in future-generation virtual and augmented-reality mobile headsets. We enable this pipeline by introducing a large-scale CGH dataset (MIT-CGH-4K) with 4,000 pairs of RGB-depth images and corresponding 3D holograms. Our CNN is trained with differentiable wave-based loss functions and physically approximates Fresnel diffraction. With an anti-aliasing phase-only encoding method, we experimentally demonstrate speckle-free, natural-looking, high-resolution 3D holograms. Our learning-based approach and the Fresnel hologram dataset will help to unlock the full potential of holography and enable applications in metasurface design, optical and acoustic tweezer-based microscopic manipulation, holographic microscopy and single-exposure volumetric 3D printing.”

Link to Publications Page

Publication: Nature
Issue/Year: Nature, Volume 591; Number 7849; Pages 234–239; 2021
DOI: 10.1038/s41586-020-03152-0

Optimizing image quality for holographic near-eye displays with Michelson Holography

Author(s):

Choi, Suyeon; Kim, Jonghyun; Peng, Yifan & Wetzstein, Gordon

Abstract:

“We introduce Michelson holography (MH), a holographic display technology that optimizes image quality for emerging holographic near-eye displays. Using two spatial light modulators (SLMs), MH is capable of leveraging destructive interference to optically cancel out undiffracted light corrupting the observed image. We calibrate this system using emerging camera-in-the-loop holography techniques and demonstrate state-of-the-art 2D and multi-plane holographic image quality.”

Link to Publications Page

Publication: Optica
Issue/Year: Optica, Volume 8; Number 2; Pages 143; 2021
DOI: 10.1364/optica.410622

High-dimensional orbital angular momentum multiplexing nonlinear holography

Author(s):

Fang, Xinyuan; Yang, Haocheng; Yao, Wenzhe; Wang, Tianxin; Zhang, Yong; Gu, Min & Xiao, Min

Abstract:

“Nonlinear holography has been identified as a vital platform for optical multiplexing holographybecause of the appearance of new optical frequencies. However, due to nonlinear wave coupling innonlinear optical processes, the nonlinear harmonic field is coupled with the input field, laying a fundamentalbarrier to independent control of the interacting fields for holography. We propose and experimentallydemonstrate high-dimensional orbital angular momentum (OAM) multiplexing nonlinear holography toovercome this problem. By dividing the wavefront of the fundamental wave into different orthogonal OAMchannels, multiple OAM and polarization-dependent holographic images in both the fundamental wave andsecond-harmonic wave have been reconstructed independently in the spatial frequency domain through atype-II second harmonic generation process. Moreover, this method can be easily extended to cascaded χ2 nonlinear optical processes for multiplexing in more wavelength channels, leading to potential applicationsin multicasting in optical communications, multiwavelength display, multidimensional optical storage, anti-counterfeiting, and optical encryption.”

Link to Publications Page

Publication: Advanced Photonics
Issue/Year: Advanced Photonics, Volume 3; Number 01; 2021
DOI: 10.1117/1.ap.3.1.015001

Dynamic 2D implementation of 3D diffractive optics

Author(s):

Haiyan Wang and Rafael Piestun

Abstract:

“Volumetric computer-generated diffractive optics offer advantages over planar 2D implementations, including the generation of space-variant functions and the multiplexing of information in space or frequency domains. Unfortunately, despite remarkable progress, fabrication of high volumetric space-bandwidth micro- and nanostructures
is still in its infancy. Furthermore, existing 3D diffractive optics implementations are static while programmable volumetric spatial light modulators (SLMs) are still years or decades away. In order to address these shortcomings, we propose the implementation of volumetric diffractive optics equivalent functionality via cascaded
planar elements. To illustrate the principle, we design 3D diffractive optics and implement a two-layer continuous phase-only design on a single SLM with a folded setup. The system provides dynamic and efficient multiplexing capability. Numerical and experimental results show this approach improves system performance such as diffraction
efficiency, spatial/spectral selectivity, and number of multiplexing functions relative to 2D devices while providing dynamic large space-bandwidth relative to current static volume diffractive optics. The limitations and capabilities of dynamic 3D diffractive optics are discussed.”

Link to Publications Page

Publication: Optica
Issue/Year: Optica Volume 5, Issues 10
DOI: 10.1364/OPTICA.5.001220

Compact design for optical-see-through holographic displays employing holographic optical elements

Author(s):

Pengcheng Zhou and Yan Li and Shuxin Liu and Yikai Su

Abstract:

“Holographic AR display is a promising technology for head-mounted display devices. However, it usually has a complicated optical system and a large form factor, preventing it from widespread applications. In this work, we propose a flat-panel design to produce a compact holographic AR display, where traditional optical elements are replaced by two holographic optical elements (HOEs). Here, these two thin HOEs together perform the optical functions of a beam expander, an ocular lens, and an optical combiner. Without any bulky traditional optics, our design could achieve a compact form factor that is similar to a pair of glasses. We also implemented a proof-of-concept prototype to verify its feasibility. Being compact, lightweight and free from accommodation-convergence discrepancy, our design is promising for fatigue-free AR displays.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express Volume 26, Issue 18
DOI: 10.1364/OE.26.022866
1 2 3