Have you ever thought that your eyes might silently reveal the world you see? The latest scientific and technological research has revealed an amazing fact: we can achieve a 3D panoramic reconstruction of the surrounding environment by analyzing the reflection of the pupil of the eye.

Uncovering the Unknown: The Secret of the Eye Reflex

This seemingly sci-fi technology has already been proven in a laboratory setting. When we gaze into the eyes of others, we are actually catching the light reflected in their eyes.

It's like seeing another person's eyes as a mirror through which we can glimpse the world as they see it. The emergence of this technology has given us a new perspective, as if it is a black technology that came out of science fiction movies.

Experiments and Challenges: Experimental Verification and Practical Application of 3D Panoramic Reconstruction

The reflective nature of the human eye is an underrated source of information. Recently, a team of students at the University of Maryland performed an experiment in which they imaged the eyes of moving characters and found that eye reflections can be used to restore 3D scenes. They published the experimental results in the paper "Seeing the World through Your Eyes".

Since our eyes capture information from multiple viewpoints when the head naturally moves, the experimental team combined classical imaging methods such as cameras with Neural Radiation Field (NeRF) technology, using a still camera to Extracting multi-view cues from eye images under moving eyes. However, this alone is not sufficient for direct modeling. Therefore, they proposed a method based on deep learning, by predicting the viewing angle in the eye image, building the shape model of the eyeball, and extracting the reflection information of the eyeball surface through the principles of reflection and refraction.

Finally, they use multiple eye images from different viewing angles to restore the 3D structure of the scene, and estimate the brightness of each point in the scene through NeRF technology. With this method, they succeeded in recovering the full 3D world seen by the observer from the eye images.

The experimental team conducted experiments in a specific scene, which included a seated test subject, a professional camera, a large Kirby doll, and two extremely bright spotlights.

In this particular setting, they successfully achieved 3D modeling using the pupillary reflex. They also tried subjects randomly picked from the internet, such as Miley Cyrus' music video, but were unable to reconstruct the scene due to video quality issues.

History and Future: From Early Research to Future Prospects

However, this isn't the first time someone has tried to pick up information from the reflexes of the eyes. As early as 2004, Columbia University published a paper called "The World in an Eye", which discussed how to obtain the wide-angle view and fixation point around the character from the eye image by analyzing the geometric model of the cornea, and calculate The three-dimensional structure of the object is used to restore the image in the eyeball.

Rob Jenkins, a professor of psychology at New York University, also published a paper in 2013, showing that more information can be obtained from the pupils of people in photos, such as images of the photographer and bystanders, which can be used to detect and lock suspects.

This experiment is different from previous studies. It does not only focus on how to restore the entire panoramic image from the eye image, but through some methods can well restore the panoramic 3D world seen by the observer, not limited to reconstruction A single panorama environment.

Although there are still some bottlenecks in this technology, it is undeniable that these experiments have achieved good results in restoring the geometry and surface details of the scene, and are also valuable in applications such as face recognition and VR. The emergence of this technology undoubtedly provides us with a new perspective, allowing us to understand and observe the world in a new way.

Overall, this technique is an exciting new discovery that opens up a whole new field of research. Although it still has some kinks to work out, we believe that as the technology improves, these kinks will be fixed. We look forward to seeing the future applications of this technology and how it will change our lives.