VR Eye Tracking using Deflectometry

VR Eye Tracking using Deflectometry

Exploiting deflectometric information for high precision eye tracking

Although the task has been studied for several decades, robust, accurate, and fast eye tracking remains a challenging problem. To estimate the gazing direction of the human eye, current approaches utilize 2D features detected from eye images, or exploit sparse reflections of a few point light sources at the eye surface (“corneal/scleral reflections”). Our approach significantly increases the information content provided from corneal or scleral reflections by using teachings from single-shot Deflectometry to acquire a dense and precise 3D model of the eye surface. To estimate the gazing direction, we exploit the retrieved surface normals and dense 3D features extracted from the measurement. Additionally, we utilize learning-based and an inverse rendering-based approaches to estimate the gazing direction from the retrieved deflectometric information.

This project is current work in progress. More information will be available soon.


VR Eye-Tracking using Deflectometry
J. Wang*, B. Xu* (* joint first authors), T. Wang, W. Lee, M. Walton, N. Matsuda, O. Cossairt, F. Willomitzer.
OSA Computational Optical Sensing and Imaging

Publications on related topics:

Hand-Guided Qualitative Deflectometry with a Mobile Device
F. Willomitzer, C-K. Yeh, V. Gupta, W. Spies, F. Schiffers, A. Katsaggelos, M. Walton, O. Cossairt.
Optics Express 28(7), 9027-9038

Low-budget 3D scanning and material estimation using PyTorch3D
O. Cossairt, F. Willomitzer, C.-K. Yeh, M. Walton.
54th IEEE Asilomar Conference on Signals, Systems and Computers

A Mitsuba-based Study on Trade-offs Between Projectionand Reflection Based Systems in Structured-Light 3D Imaging
T. Wang, F. Schiffers, F. Willomitzer, O. Cossairt.
OSA Computational Optical Sensing and Imaging

Florian Willomitzer
Research Assistant Professor