High-quality 3D imaging with Commodity Devices

3D imaging methods accessible to a broad audience

This body of work introduces a series of systems that only require commodity devices such as screens, (web-) cameras, low-end tablets or mobile phones to capture high-quality 3D data: The developed “Mobile Multiview Deflectometry” system exploits screen and front camera of mobile devices for deflectometry-based measurements. It works without the need for a calibration and is optimized for specular surfaces such as stained glass artworks. To compensate for the small screen, a multi-view registration technique is applied so that large surfaces can be densely reconstructed in their entirety. The “SkinScan” sensor principle uses the same hardware components but exploits photometric stereo algorithms for the measurement of matte object surfaces such as human skin.

Applications: The project is a first step towards a universal self-calibrating measurement procedure usable by a broad audience with little to no technical imaging experience. The introduced sensor principles have great potential for cultural heritage analysis and for telemedicine or medical treatment/diagnosis in developing countries.

Related Talk Recording

	F. Willomitzer Utilizing Nature’s Limits for Better Computational 3D Imaging Center For Robotics and Biosystems Seminar, Northwestern University, USA, 2022 [Full Video] , [Jump to Project (29:40)]
	F. Willomitzer Fundamental Limits in Computational 3D Imaging – From Novel 3D Cameras to Looking around Corners Beyond the Patterns Talk Series, University Erlangen-Nuremberg, Germany, 2021 [Full Video] , [Jump to Project (13:10)]

Publications

Hand-Guided Qualitative Deflectometry with a Mobile Device
F. Willomitzer, C-K. Yeh, V. Gupta, W. Spies, F. Schiffers, A. Katsaggelos, M. Walton, O. Cossairt.
Optics Express 28(7), 9027-9038, 2020

3D Imaging of Complex Specular Surfaces by Fusing Polarimetric and Deflectometric Information
J. Wang, O. Cossairt, F. Willomitzer
ArXiv Preprint, 2024

SkinScan: Low-Cost 3D-Scanning for Dermatologic Diagnosis and Documentation
M. A. Nau, F. Schiffers, Y. Li, B. Xu, A. Maier, J. Tumblin, M. Walton, A. K. Katsaggelos, F. Willomitzer, O. Cossairt
2021 IEEE International Conference on Image Processing (ICIP), 2021

Low-budget 3D scanning and material estimation using PyTorch3D
O. Cossairt, F. Willomitzer, C.-K. Yeh, M. Walton.
54th IEEE Asilomar Conference on Signals, Systems and Computers, 2020

3D Surface Measurement and Analysis of Works of Art
B. Xu, F. Willomitzer, C.-K. Yeh, F. Li, V. Gupta, J.E. Tumblin, M. Walton, O. Cossairt
53rd IEEE Asilomar Conference on Signals, Systems, and Computers, 2019

A Mitsuba-based Study on Trade-offs Between Projectionand Reflection Based Systems in Structured-Light 3D Imaging
T. Wang, F. Schiffers, F. Willomitzer, O. Cossairt.
OSA Computational Optical Sensing and Imaging, 2021

A Low-Cost Solution for 3D Reconstruction of Large-Scale Specular Objects
Y. Li, C. Yeh, B. Xu, F. Schiffers, M. Walton, J. Tumblin, A. Katsaggelos, F. Willomitzer, O. Cossairt.
OSA Computational Optical Sensing and Imaging, 2021

SkinScan: 3D Dermatologic Diagnosis and Documentation with Commodity Devices
M. A. Nau, F. Schiffers, Y. Li, B. Xu, A. Maier, J. Tumblin, M. Walton, A. K. Katsaggelos, F. Willomitzer, O. Cossairt
55th Asilomar Conference on Signals, Systems, and Computers, 2021

How to archive the visual contents of aging analog film holograms?
J. Choi, A. Dashpute, B. Xu, O. Cossairt, M. Walton, J. Tumblin, F.Willomitzer
Proceedings of the 124th DGaO Conference, Berlin, B6, 2023

Software

SkinScan: An open-source framework for structured light 3D imaging, which includes the possibility to perform Deflectometry and Photometric Stereo measurements at home using solely commodity devices.

News and Press

Science Mag
Daily Northwestern
Northwestern Now
Smithsonian Magazine
Science Friday
Apollo
Popular Science
Medill News
CBC ‘Quirks and Quarks’
Santa Fe New Mexican
Chemistry World
la Repubblica
KRQE-TV
WBEZ Chicago
Telegraph (UK)
Spiegel
Kronen Zeitung

High-quality 3D imaging with Commodity Devices