The Reflections In Your Eyes Can Reconstruct 3D Objects And Environments

Scientists from the University of Maryland have made a remarkable stride into turning reflections in human eyes into 3D scenes. The experiment builds upon a technology known as Neural Radiance Fields (NeRF) while using artificial intelligence to reconstruct environments from 2D photos.
 
In their experiment, which was first reported by TechXplore, the team used reflections of light captured in human eyes to try and visualize what the person was looking at. By analyzing high-resolution photos of individuals looking towards the camera, the scientists zoomed in on their irises, extracting the object and determining the direction of the individual’s gaze.

Video via University of Maryland / GitHub
 
In one example, they caught an image of the subject looking at the video game character Kirby standing next to a dog. In another instance, the system was able to extract the views of a barbershop. However, its blurry quality made it seem like something out of a dream. 
 
The team took these samples, zoomed into each individual’s corneas, flipped them, removed the color and other details the irises might have distorted, and removed the curved-mirror effect to develop 3D images.

Video via University of Maryland / GitHub
 
Outside of these controlled experiments, where subjects were tasked to focus on an object or a specific space, the team also experimented with real-world scenarios. In two other illustrations, they studied Miley Cyrus’ and Lady Gaga’s eyes in their music videos. The results from this weren’t clear images. Instead, it looked like Cyrus was staring at a lighting grid while Lady Gaga could have been looking at a director or some other person on set. 
 
This all sounds like a convenient plot device in a spy movie where the good guys can zoom into someone’s eye to catch the reflection of the villain.

Video via University of Maryland / GitHub
 
Either way, it’s incredibly cool. And while the study is still in its nascent stages, the researchers believe the study could lend itself to expanding the possibilities of 3D scene reconstruction.

You can view the full set of renders here on GitHub.
 
 
 
[via New Atlas and Engadget, images via University of Maryland/GitHub]