DI4D’s Head Mounted Camera System Captures True Facial Performance

Facial performance capture company Dimensional Imaging (DI4D) is launching its new Head Mounted Camera system at the 2019 Game Developers Conference, 18-22 March in San Francisco. The new DI4D HMC outputs extremely high fidelity facial performance data when used together with the DI4D post processing pipeline.  

The DI4D HMC is designed to combine image quality, camera stability and comfort of fit for the actor. The new system uses a calibrated stereo pair of 4MP cameras and synchronised on-board strobed LED lighting to sharpen the cameras’ focus and make sure illumination is even across the face while minimising the apparent brightness to the subject. A custom head-band and boom design avoids pinch points and ensures camera stability. The system is also completely wireless and can operate at mainstream production frame rates up to 60fps.

Markers, make-up or structured light projection are not necessary – the system relies on high quality image data instead. DI4D facial performance capture is based on proprietary passive stereo photogrammetry software that automatically processes the captured stereo pair of images to derive a dense 3D model, with a 3D point calculated for every foreground pixel.

Multiple stereo pairs of cameras can be used to capture multiple overlapping 3D models, which are then merged automatically into a single 3D model per frame. If colour cameras are used then a per frame colour texture map can be applied to the 3D model sequence.

For the DI4D HMC, Dr Colin Urquhart, DI4D's CEO said, "The development of the HMC draws on the experience we have gained in five years of using stereo camera HMCs for many different visual effects projects."

Although the DI4D HMC can improve the output of most third-party facial processing pipelines, it has been designed to be especially effective when used with DI4D's post processing systems. As well as calculating per frame 3D models, the DI4D Processing Software also calculates a dense optical flow map for each image to the previous and subsequent image in the sequence. This dense optical flow data is used in the DI4D Tracking Software to track (follow) landmarks or the vertices of a fixed topology mesh through a sequence.  www.di4d.com