Holographic Glasses for Virtual Reality

Author(s):

JONGHYUN KIM, NVIDIA, USA and Stanford University, USA
MANU GOPAKUMAR, SUYEON CHOI, AND YIFAN PENG, Stanford University, USA
WARD LOPES, NVIDIA, USA
GORDON WETZSTEIN, Stanford University, USA

Abstract:

“We resent Holographic Glasses, a holographic near-eye display system with an eyeglasses-like form factor for virtual reality. Holographic Glasses are composed of a pupil-replicating waveguide,
a spatial light modulator, and a geometric phase lens to create holographic images in a lightweight
and thin form factor. The proposed design can deliver full-color 3D holographic images using an optical stack of 2.5 mm thickness. A novel pupil-high-order gradient descent algorithm is presented for the correct phase calculation with the user’s varying pupil size. We implement benchtop and wearable prototypes for testing. Our binocular wearable prototype supports 3D focus cues and provides a diagonal field of view of 22.8? with a 2.3 mm static eye box and additional capabilities of dynamic eye box with beam steering, while weighing only 60 g excluding the driving board.”

Link to Publications Page

Publication: Association for Computing Machinery
Issue/Year: Association for Computing Machinery, 2022

Accommodative holography: improving accommodation response for perceptually realistic holographic displays

Author(s):

Kim, Dongyeon; Nam, Seung-Woo; Lee, Byounghyo; Seo, Jong-Mo & Lee, Byoungho

Abstract:

“Holographic displays have gained unprecedented attention as next-generation virtual and augmented reality applications with recent achievements in the realization of a high-contrast image through computer-generated holograms (CGHs). However, these holograms show a high energy concentration in a limited angular spectrum, whereas the holograms with uniformly distributed angular spectrum suffer from a severe speckle noise in the reconstructed images. In this study, we claim that these two physical phenomena attributed to the existing CGHs significantly limit the support of accommodation cues, which is known as one of the biggest advantages of holographic displays. To support the statement, we analyze and evaluate various CGH algorithms with contrast gradients – a change of contrast over the change of the focal diopter of the eye – simulated based on the optical configuration of the display system and human visual perception models. We first introduce two approaches to improve monocular accommodation response in holographic viewing experience; optical and computational approaches to provide holographic images with sufficient contrast gradients. We design and conduct user experiments with our prototype of holographic near-eye displays, validating the deficient support of accommodation cues in the existing CGH algorithms and demonstrating the feasibility of the proposed solutions with significant improvements on accommodative gains.”

Link to Publications Page

Publication: ACM Transactions on Graphics
Issue/Year: ACM Transactions on Graphics, Volume 41; Number 4; Pages 1–15; 2022
DOI: 10.1145/3528223.3530147

Neural 3D holography

Author(s):

Choi, Suyeon; Gopakumar, Manu; Peng, Yifan; Kim, Jonghyun & Wetzstein, Gordon

Abstract:

“Holographic near-eye displays promise unprecedented capabilities for virtual and augmented reality (VR/AR) systems. The image quality achieved by current holographic displays, however, is limited by the wave propagation models used to simulate the physical optics. We propose a neural network-parameterized plane-to-multiplane wave propagation model that closes the gap between physics and simulation. Our model is automatically trained using camera feedback and it outperforms related techniques in 2D plane-to-plane settings by a large margin. Moreover, it is the first network-parameterized model to naturally extend to 3D settings, enabling high-quality 3D computer-generated holography using a novel phase regularization strategy of the complex-valued wave field. The efficacy of our approach is demonstrated through extensive experimental evaluation with both VR and optical see-through AR display prototypes.”

Link to Publications Page

Publication: ACM Transactions on Graphics
Issue/Year: ACM Transactions on Graphics, Volume 40; Number 6; Pages 1–12; 2021
DOI: 10.1145/3478513.3480542

Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration

Author(s):

Peng, Yifan; Choi, Suyeon; Kim, Jonghyun & Wetzstein, Gordon

Abstract:

“Computer-generated holography (CGH) holds transformative potential for a wide range of applications, including direct-view, virtual and augmented reality, and automotive display systems. While research on holographic displays has recently made impressive progress, image quality and eye safety of holographic displays are fundamentally limited by the speckle introduced by coherent light sources. Here, we develop an approach to CGH using partially coherent sources. For this purpose, we devise a wave propagation model for partially coherent light that is demonstrated in conjunction with a camera-in-the-loop calibration strategy. We evaluate this algorithm using light-emitting diodes (LEDs) and superluminescent LEDs (SLEDs) and demonstrate improved speckle characteristics of the resulting holograms compared with coherent lasers. SLEDs in particular are demonstrated to be promising light sources for holographic display applications, because of their potential to generate sharp and high-contrast two-dimensional (2D) and 3D images that are bright, eye safe, and almost free of speckle.”

Link to Publications Page

Publication: Science Advances
Issue/Year: Science Advances, Volume 7; Number 46; 2021
DOI: 10.1126/sciadv.abg5040

Hologram computation using the radial point spread function

Author(s):

Yasuki, Daiki; Shimobaba, Tomoyoshi; Makowski, Michal; Suszek, Jaroslaw; Kakue, Takashi & Ito, Tomoyoshi

Abstract:

“Holograms are computed by superimposing point spread functions (PSFs), which represent the distribution of light on the hologram plane. The computational cost and the spatial bandwidth product required to generate holograms are significant; therefore, it is challenging to compute high-resolution holograms at the rates required for videos. Among the possible displays, fixed-eye-position holographic displays, such as holographic head-mounted displays, reduce the spatial bandwidth product by fixing eye positions while satisfying almost all human depth cues. In eye-fixed holograms, by calculating a part distribution of the entire PSF, we observe reconstructed images that maintain the image quality and the depth of focus almost as high as those generated by the entire PSF. In this study, we accelerate the calculation of eye-fixed holograms by engineering the PSFs. We propose cross and radial PSFs, and we determine that, out of the two, the radial PSFs have a better image quality. By combining the look-up table method and the wavefront-recording plane method with radial PSFs, we show that the proposed method can rapidly compute holograms.”

Link to Publications Page

Publication: Applied Optics
Issue/Year: Applied Optics, Volume 60; Number 28; Pages 8829; 2021
DOI: 10.1364/ao.437777

Holographic near-eye display based on complex amplitude modulation with band-limited zone plates

Author(s):

Chen, Yun; Hua, Minjie; Zhang, Tianshun; Zhou, Mingxin; Wu, Jianhong & Zou, Wenlong

Abstract:

“A holographic near-eye display (NED) system based on complex amplitude modulation (CAM) with band-limited zone plates is proposed. The whole system mainly consists of a phase-only spatial light modulator (SLM), an Abbe-Porter filter system, an eyepiece, and an image combiner. The point source method based on band limited zone plates is used to accurately control the bandwidth of the target complex amplitude. The effects of intensity modulation coefficient γ in the frequency-filtering method on the intensity and the quality of reconstructed images are analyzed, which provide a judgment basis for selecting the appropriate value of γ. We also derive the expressions of the field of view (FOV) and exit pupil of the NED system. Since the holographic image is magnified in two steps in this system, the large FOV can be obtained. The optical experimental results show that the proposed system can provide a dynamic holographic three-dimensional (3D) augmented reality (AR) display with a 23.5°horizontal FOV.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 29; Number 14; Pages 22749; 2021
DOI: 10.1364/oe.431032

Vision-correcting Holographic Display: Evaluation of Aberration Correcting Hologram

Author(s):

Kim, Dongyeon; Nam, Seung-Woo; Bang, Kiseung; Lee, Byounghyo; Lee, Seungjae; Jeong, Youngmo; Seo, Jongmo & Lee, Byoungho

Abstract:

“Vision-correcting displays are key to achieving physical and physiological comforts to the users with refractive errors. Among such displays are holographic displays, which can provide a high-resolution vision-adaptive solution with complex wavefront modulation. However, none of the existing hologram rendering techniques have considered the optical properties of the human eye nor evaluated the significance of vision correction. Here, we introduce vision-correcting holographic display and hologram acquisition that integrates user-dependent prescriptions and a physical model of the optics, enabling the correction of on-axis and off-axis aberrations. Experimental and empirical evaluations of the vision-correcting holographic displays show the competence of holographic corrections over the conventional vision correction solutions.”

Link to Publications Page

Publication: Biomedical Optics Express
Issue/Year: Biomedical Optics Express,Vol. 12, Issue 8, pp. 5179-5195, 2021
DOI: 10.1364/boe.433919

Holographic contact lens display that provides focusable images for eyes

Author(s):

Sano, Junpei & Takaki, Yasuhiro

Abstract:

“In this paper, we propose a holographic image generation technique for contact lens displays. The proposed technique employs a phase-only spatial light modulator (SLM), a holographic optical element (HOE) backlight, and a polarizer. The proposed holographic technique can generate 3D images apart from the contact lens displays. Therefore, the eyes can focus on the 3D images while simultaneously observing the real scene through the phase-only SLM and the HOE backlight, which provides see-through capability. A bench-top experimental system was constructed to verify the far-distance image generation capability and see-through function.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 29; Number 7; Pages 10568; 2021
DOI: 10.1364/oe.419604

Foveated near-eye display using computational holography

Author(s):

Cem, Ali; Hedili, M. Kivanc; Ulusoy, Erdem & Urey, Hakan

Abstract:

“Holographic display is the only technology that can offer true 3D with all the required depth cues. Holographic head-worn displays (HWD) can provide continuous depth planes with the correct stereoscopic disparity for a comfortable 3D experience. Existing HWD approaches have small field-of-view (FOV) and small exit pupil size, which are limited by the spatial light modulator (SLM). Conventional holographic HWDs are limited to about 20° × 11° FOV using a 4 K SLM panel and have fixed FOV. We present a new optical architecture that can overcome those limitations and substantially extend the FOV supported by the SLM. Our architecture, which does not contain any moving parts, automatically follows the gaze of the viewer’s pupil. Moreover, it mimics human vision by providing varying resolution across the FOV resulting in better utilization of the available space-bandwidth product of the SLM. We propose a system that can provide 28° × 28° instantaneous FOV within an extended FOV (the field of view that is covered by steering the instantaneous FOV in space) of 60° × 40° using a 4 K SLM, effectively providing a total enhancement of > 3 × in instantaneous FOV area, > 10 × in extended FOV area and the space-bandwidth product. We demonstrated 20° × 20° instantaneous FOV and 40° × 20° extended FOV in the experiments.”

Link to Publications Page

Publication: Scientific Reports
Issue/Year: Scientific Reports, Volume 10; Number 1; 2020
DOI: 10.1038/s41598-020-71986-9

Fast calculation of computer-generated hologram of line-drawn objects without FFT

Author(s):

Nishitsuji, Takashi; Shimobaba, Tomoyoshi; Kakue, Takashi & Ito, Tomoyoshi

Abstract:

“Although holographic display technology is one of the most promising three-dimensional (3D) display technologies for virtual and augmented reality, the enormous computational effort required to produce computer-generated holograms (CGHs) to digitally record and display 3D images presents a significant roadblock to the implementation of this technology. One of the most effective methods to implement fast CGH calculations is a diffraction calculation (e.g., angular spectrum diffraction) based on the fast-Fourier transform (FFT). Unfortunately, the computational complexity increases with increasing CGH resolution, which is what determines the size of a 3D image. Therefore, enormous calculations are still required to display a reasonably sized 3D image, even for a simple 3D image. To address this issue, we propose herein a fast CGH algorithm for 3D objects comprised of line-drawn objects at layers of different depths. An aperture formed from a continuous line at a single depth can be regarded as a series of aligned point sources of light, and the wavefront converges for a sufficiently long line. Thus, a CGH of a line-drawn object can be calculated by synthesizing converged wavefronts along the line. Numerical experiments indicate that, compared with the FFT-based method, the proposed method offers a factor-56 gain in speed for calculating 16-k-resolution CGHs from 3D objects composed of twelve line-drawn objects at different depths.”

Link to Publications Page

Publication: Optics Express
Issue/Year: Optics Express, Volume 28; Number 11; Pages 15907; 2020
DOI: 10.1364/oe.389778
1 2