Interra Detects and Verifies Lip Sync Errors with Machine Learning

Interra Systems has developed BATON LipSync, an automated tool for lip sync detection and verification, that uses machine learning and deep neural networks to detect audio and video sync errors. Using BATON LipSync, broadcasters and service providers can accurately detect audio lead and lag issues in media content, and use the information to improve the quality of experience for viewers.

“Lip sync errors are annoying and distracting and negatively impact viewing overall,” said Anupama Anantharaman, vice president of product management at Interra Systems. “BATON LipSync addresses audio-video sync issues with an approach based on image processing and machine learning. As well overcoming the challenges of detection, operators receive debug information and detailed reporting to use as a head start when fine-tuning audio-video sync accuracy."

LipSync software performs facial detection, facial tracking, lip detection and lip activity detection. Sync errors can be debugged further in the BATON Media Player through an interface with tools that can plot out-of-sync audio and video errors on a skewed timeline for clearer visualisation. After errors are detected, BATON LipSync presents a comprehensive report of all the lip sync issues.

Timeline plot

Available content can be checked in any language independent of the region and area. Supporting most industry formats, BATON LipSync is able meet complex sets of professional requirements for delivering high-quality video to multiple screens. The software can be integrated into the user’s existing content processing workflow, or used as a stand-alone application.

In effect, replacing a time-consuming, expensive manual processes for lip sync detection and verification with faster, more accurate machine-assisted detection gives broadcasters a chance to increase efficiency and augment their existing workflows with an ML-powered system.

Interra Systems will present a 1-hour BATON LipSync online demo as part of the company’s digital series of events taking place from now through to 24 April.   www.interrasystems.com