The Rise and Stall of Retinal Projection in Virtual Reality: An Unexplored Story

Quick Links

Retinal projection is a technology that uses the retina of your eye as a projection screen, beaming light directly into your eye so that you see a “display” floating in space. It feels like it should be the future of VR, so what happened?

Understanding Virtual Retinal Displays

All of the flat panel displays you look at every day have a pixel grid and then either shine a backlight through that pixel grid, or the pixels themselves will emit light in the case of OLED displays.

Virtual Retinal Displays work more like the CRT (Cathode Ray Tube) monitors and TVs of days displayed, where an image raster is drawn on the back of a phosphorescent screen. Except, in this case, the image is drawn directly onto the retina of the eye.

The end result is what appears to be a screen floating in space, or an image that seems to be part of the scene.

Why Do We Want Retinal Projection?

Retinal projection has a number of advantages over current display technologies. While early retinal projection systems were bulky and heavy, modern systems use lightweight laser systems or modern LED technology to shoot photons into your eyes.

Current VR headsets use one or more flat panel displays viewed through special lenses that untwist the precisely distorted image on the LCD or OLED display. This results in an image shaped in such a way that it provides an immersive experience. Unfortunately, this design often results in visible pixel grids (the “chicken wire” or “screen door effect” ) and somewhat fuzzy visuals.

In contrast, the resolution and crispness of retinal projection images are exceptional. They don’t cause the same amount of eye strain as OLEDs or LCD screens an inch from your eye, thanks to the small amount of light needed.

Retinal projection systems also have optical advantages. This technology allows for optical correction on the fly, so you don’t need to worry about wearing glasses . It can also refocus to display near or far objects.

For VR or Mixed Reality (MR) headsets, retinal projection systems have the potential to make much smaller, less power-hungry sets. The holy grail for any sort of VR or MR headset is to eventually shrink them down to the size of sunglasses.

VSDC Pro Video Editor is a light professional non-linear video editing suite for creating a movie of any complexity. It supports the most popular video/audio formats and codecs, including 4K, HD and GoPro videos. Preconfigured profiles make the creation of videos for various multimedia and mobile devices absolutely hassle-free.

Key features:

• Import from any devices and cams, including GoPro and drones. All formats supported. Сurrently the only free video editor that allows users to export in a new H265/HEVC codec, something essential for those working with 4K and HD.
• Everything for hassle-free basic editing: cut, crop and merge files, add titles and favorite music
• Visual effects, advanced color correction and trendy Instagram-like filters
• All multimedia processing done from one app: video editing capabilities reinforced by a video converter, a screen capture, a video capture, a disc burner and a YouTube uploader
• Non-linear editing: edit several files with simultaneously
• Easy export to social networks: special profiles for YouTube, Facebook, Vimeo, Twitter and Instagram
• High quality export – no conversion quality loss, double export speed even of HD files due to hardware acceleration
• Stabilization tool will turn shaky or jittery footage into a more stable video automatically.
• Essential toolset for professional video editing: blending modes, Mask tool, advanced multiple-color Chroma Key

The Limitations of Retinal Displays

There are a few limitations of retinal projection that make it less than ideal as a replacement for the VR systems of today. First of all, the field of view possible with current retinal projection is too narrow for VR. This means it’s not immersive enough to meet the standards for modern VR experiences.

Retinal displays use various approaches to project images into the eye, including sophisticated micro-mirror arrays or precisely angled lasers. Tiny moving parts such as these are inevitably harder to make than solid-state systems such as an OLED screen. That complex under-the-hood system creates a lot of hurdles in the development process.

What Happened to Avegant?

Avegant

You may or may not know it, but there is actually a retinal projection technology headset you can buy and own. In 2016 a company called Avegant released the Avegant Glyph . The Glyph looks like a pair of standard headphones , but you can flip the headband down over your eyes and enjoy a video feed projected onto your retinas. It didn’t look like VR, but it was a 720p home theater system that you could take anywhere with you.

You can still buy a Glyph on Amazon, although it’s not likely to be new. On Avegant’s site, however, you’ll find no mention of the Glyph as a product you can buy. Instead, Avegant sells “light engines “ as components for other companies that want to develop wearable headsets. Even when the Glyph came out, reviewers were a little lukewarm on it and it suffered from typical first-generation technology syndrome. If you read contemporary reviews of the Glyph, reviewers noted that the headset was only 720p, heavy, tricky to initially set up, and much too expensive for what it offers.

That said, Avegant is very much still around and working on pushing its technology forward, perhaps so that a partner company (such as Facebook perhaps) could one day create a successful mainstream system. And while Avegant is the only company we know of that has brought out a commercial VRD product, many different players are investing in research and development to make VRD technology a reality.

In 2020, Bosch showed off smartglasses that use lasers to project images into your retinas. QD Laser’s Viserium used retinal projection to help people with low vision see more clearly. Magic Leap is working on next-generation augmented reality, and the list of companies working involved with VRD technology has at least a half-dozen more names to add.

Retinal Projection Might Be the Future of Mixed Reality

While current retinal projection might not be the best first for VR, it may have a future in MR applications. Devices like the Microsoft Hololens 2 incorporate laser-based retinal projection and don’t require large fields of view to be useful.

If retinal projection technology ever manages to achieve the same horizontal field of view as consumer VR headsets like the Quest 2 it might still end up being the most realistic and sharp VR solution short of jacking a computer directly into your visual cortex .

Some Approaches

The Rise and Stall of Retinal Projection in Virtual Reality: An Unexplored Story