Real-time camera tracking with Mo-sys Star Tracker
80six are video specialists that supply equipment and technical services for virtual productions – film and TV drama, live entertainment, broadcast, esports and corporate events. Their support ranges from LED screens to cameras, video control systems and real-time video content creation.
The company recently launched its virtual production studios in Slough in England, opening them to external customers. The facility has a preconfigured curved LED stage for shooting in-camera VFX.
Virtual R&D
80six also invests time and resources into R&D for new virtual filmmaking techniques. The studio has been a primary developer in the construction and implementation of xR (extended reality) stages, and has now directed its attention to live LED in-camera virtual production, which uses similar approaches. The company has spent close to two years carrying out R&D with LED walls and cameras for film and TV productions with partners such as Epic Games, Dimension Studio, DNEG and OSF.
“Throughout the last couple of years, we have carried out various tests that have given us a solid understanding of the principles of VP. We can now confidently advise our clients on the benefits and limitations they can expect when shooting in-camera VFX or when using LED video panels as light sources,” said Jack James, Director at 80six.
Set in an overall area of approximately 335m2, the Virtual Production Stage encompasses an 18m x 4.5m high-resolution LED volume, built with ROE Visual Diamond DM 2.6mm for the rear wall and ROE Carbon 5.7mm for the LED ceiling, with movable lighting dollies.
The ROE screens run on Brompton Tessera SX40 4K Version 3.1 LED processors. “Brompton’s 3.1 software has high frame rate functionality that puts out smoother, faster play-back of content, which is especially useful for slow-motion for film, TV and esports,” Jack said.
Dynamic Calibration
80six virtual production stage LED volume, © 80six.
Jack regards the experience 80six gained by supporting a major VFX and animation company on the production of a short film, shot on 80six's virtual production stage at the start of 2021, to be invaluable.
"It was necessary to calibrate our ROE Diamond 2.6mm for HDR,” Jack said. “We used Brompton's Hydra calibration system to help us produce the true-to-life colours that were required for the photorealistic background content, and achieved excellent results. The request for HDR was the result of several months of testing involving colour scientists and VFX teams. We're fortunate that the majority of our LED inventory uses Brompton Tessera R2 receiver cards, which means we can recalibrate them using Hydra whenever necessary."
The Hydra system uses Brompton’s Dynamic Calibration to calibrate each area of the panel. Traditionally, a fixed, factory-specified calibration is applied to panels that is used for all content over their lifetime, which may prevent them from performing at the brightness they are capable of. Dynamic Calibration is a more flexible approach that works with the R2 receiver cards and processes image information in real time, without adding latency. It intelligently determines the optimum way to drive each LED for the given images.
Building an LED volume for Virtual Production tests.
The dynamic approach means the panels can adapt, for instance, to images with extremely bright areas by making full use of the LED’s brightness. It adapts to vivid areas by accessing the maximum available gamut of the LEDs. Smooth areas of content are displayed with precise uniformity for greater clarity, and sensitive regions of content, such as skin tones, are balanced for a natural look.
Interactive LEDs
The result is brighter whites, higher contrast ratios and more saturated, accurate colours that increase the depth and realism of images. Throughout, Dynamic Calibration’s colour processing keeps the video content looking uniform, out to the extremes of brightness and colour gamut.
The dynamic nature of the system makes calibration more interactive as well, and means the desired brightness, primary colours and white point are user-adjustable. Jack noted that, in fact, the request for HDR was the result of several months of testing involving colour scientists and VFX teams. “Because the majority of our LED inventory is equipped with Brompton R2 cards, we could recalibrate them using Hydra whenever necessary," he said. Instead of a lengthy, expensive process, recalibration can now be done at any time from a user interface on the Tessera processor.
Gunslinger
Due in part to the pandemic, filmmakers, DOPs and VFX studios are thinking about VP now in more practical, immediate terms than earlier on when the advantages of lower costs and greater control didn’t feel as urgent. Another element of virutal production is the opportunity it opens to shift the creative process further into the pre-production rather than the post phase.
Last year, 80six worked with DNEG, Dimension Studio, Unreal Engine, ARRI, Mo-Sys, ROE and Brompton Technologies to explore these emerging techniues shaping the future of film and virtual production. Their test used LED walls and real-time engines, and was one of the first in the UK trialling the same techniques previously used in Disney’s The Mandalorian.
This world has depth and texture that can be taken advantage of through precise camera tracking throughout the shoot. Viewers see two gunslingers out in Dimension’s wide, sweeping desert landscape. Tracking of the camera moves was performed via a Mo-Sys Star Tracker device mounted to the camera.
The device continuously calculated and re-calculated how far away and at what angle the camera was from a series of stickers fixed to the floor and set. From these distance values the production calculated the exact perspective at which the viewer would expect to see the background at every moment of the movie. As long as the CG model has been completed to accommodate those perspectives, it can be rendered to the screen and captured in-camera along with the live action in front of it.
More Potential, Less Limitation
© Wilder Films, photographer Tom Oxley Virtual Production
Brompton Technology processing devices and software are part of the 80six studio's basic technical equipment. "Companies like Brompton are constantly developing new software features that open up more potential for virtual production,” Jack said. “Brompton’s Frame Remapping is another interesting concept we are looking forward to exploring as it means you can now have multiple cameras shooting the same physical screen, but with different layers of content represented on each individual frame for each camera.
“As a filmmaker, this means you are no longer limited to a particular shot. You can effectively have two scenes running at the same time, or a green screen frame as the second scene that you can then use for keying. This gives filmmakers the flexibility to edit their work in post-production as if they were still on-set." www.bromptontech.com
