High-quality 3D imaging with Commodity Devices

High-quality 3D imaging with Commodity Devices

3D imaging methods accessible to a broad audience

This body of work introduces a series of systems that only require commodity devices such as screens, (web-) cameras, low-end tablets or mobile phones to capture high-quality 3D data: The developed “Mobile Multiview Deflectometry” system exploits screen and front camera of mobile devices for deflectometry-based measurements. It works without the need for a calibration and is optimized for specular surfaces such as stained glass artworks. To compensate for the small screen, a multi-view registration technique is applied so that large surfaces can be densely reconstructed in their entirety. The “SkinScan” sensor principle uses the same hardware components but exploits photometric stereo algorithms for the measurement of matte object surfaces such as human skin.

Applications: The project is a first step towards a universal self-calibrating measurement procedure usable by a broad audience with little to no technical imaging experience. The introduced sensor principles have great potential for telemedicine or medical treatment/diagnosis in developing countries.

Jump to: [Related Talk Recording] , [Publications] , [Software] , [News and Press]



Related Talk Recording

F. Willomitzer
Fundamental Limits in Computational 3D Imaging – From Novel 3D Cameras to Looking around Corners
Beyond the Patterns Talk Series, University Erlangen-Nuremberg, Germany, 2021
[Full Video] , [Jump to Project (13:10)]

Publications

Hand-Guided Qualitative Deflectometry with a Mobile Device
F. Willomitzer, C-K. Yeh, V. Gupta, W. Spies, F. Schiffers, A. Katsaggelos, M. Walton, O. Cossairt.
Optics Express 28(7), 9027-9038

SkinScan: Low-Cost 3D-Scanning for Dermatologic Diagnosis and Documentation
M. A. Nau, F. Schiffers, Y. Li, B. Xu, A. Maier, J. Tumblin, M. Walton, A. K. Katsaggelos, F. Willomitzer, O. Cossairt
2021 IEEE International Conference on Image Processing (ICIP), 2021

Low-budget 3D scanning and material estimation using PyTorch3D
O. Cossairt, F. Willomitzer, C.-K. Yeh, M. Walton.
54th IEEE Asilomar Conference on Signals, Systems and Computers

3D Surface Measurement and Analysis of Works of Art
B. Xu, F. Willomitzer, C.-K. Yeh, F. Li, V. Gupta, J.E. Tumblin, M. Walton, O. Cossairt
53rd IEEE Asilomar Conference on Signals, Systems, and Computers

A Mitsuba-based Study on Trade-offs Between Projectionand Reflection Based Systems in Structured-Light 3D Imaging
T. Wang, F. Schiffers, F. Willomitzer, O. Cossairt.
OSA Computational Optical Sensing and Imaging

A Low-Cost Solution for 3D Reconstruction of Large-Scale Specular Objects
Y. Li, C. Yeh, B. Xu, F. Schiffers, M. Walton, J. Tumblin, A. Katsaggelos, F. Willomitzer, O. Cossairt.
OSA Computational Optical Sensing and Imaging


Software

SkinScan: An open-source framework for structured light 3D imaging, which includes the possibility to perform Deflectometry and Photometric Stereo measurements at home using solely commodity devices.


News and Press

Science Mag
Daily Northwestern
Northwestern Now
Smithsonian Magazine
Science Friday
Apollo
Popular Science
Medill News
CBC ‘Quirks and Quarks’
Santa Fe New Mexican
Chemistry World
la Repubblica
KRQE-TV
WBEZ Chicago
Telegraph (UK)
Spiegel
Kronen Zeitung

Avatar
Florian Willomitzer
Research Assistant Professor