Synapse works with Brompton’s LED video processors to carry out virtual production with creative flair, pushing to overcome multi-cam and image quality challenges on the LED stage.
Synapse Virtual Production is an immersive entertainment collective in Los Angeles whose members are working in film and television production, advertising, music videos, live events, broadcast, esports and other media. Blending artistic with technical skills, the Synapse team specialises in virtual production with a creative edge, using their collected experience to give clients a wider sense of creative possibilities on set.
The Synapse team constructed a temporary pop-up LED volume in early 2023 for hands-on R&D and production work while planning and building out their permanent facility, set to open in October 2023 at Los Angeles Center Studios. The pop-up space made use of nine Brompton Tessera SX40 processors, running the 60-foot wide LED volume at 4K and 60 fps.
Converging Talent
From the company’s start, Synapse has worked with Brompton Technology’s LED video processors to carry out virtual production. Synapse’s Chief Technology Officer Geoff Knight said, “Virtual production is so exciting right now because the talent pool is converging from so many different areas, and there’s a robust shared knowledge base that extends to the equipment we use. Coming from a background in live events, I’ve been very familiar with Brompton for years, and know first-hand how the processors perform on set and how reliable they are.”
One of the projects that Synapse has learned from the most while working at the temporary site is a music video for Jackson Wang titled ‘Cruel’, directed by Rich Lee. As well as a music video director and VFX artist, Rich is Synapse’s Chief Creative Officer and known for his work with artists such as Billie Eilish, Lana Del Rey, Rihanna and Eminem. He is very familiar with high drama, high concept kinds of productions like the Jackson Wang project, and pushing looks with visual effects.
'Cruel', Jackson Wang
Everything In-Camera
However, using virtual production for ‘Cruel’ shifted nearly all of their attention to what was happening on set, by allowing them to create and capture a digital world all in-camera.
“The concept for ‘Cruel’ was incredibly ambitious and it turned out to be a very impressive effort in terms of execution and scope, especially with tight timelines for prep and shooting,” said Christopher Probst, ASC, Chief Innovation Officer at Synapse. “Rich and I had worked on a music video a few years ago with a similar post-apocalyptic look, which we shot on location with a lot of practical elements, augmented later in post with VFX. It was a huge challenge.
“For ‘Cruel,’ we were able to capture everything in the LED volume over just two 10-hour days of shooting – a timeframe that’s virtually unheard of for a music video. The shoot went very smoothly and it was such an eye-opening experience for Rich and I – we couldn’t believe the quality of the visuals we were getting in-camera.”
To set the scene for the video shoot, the Synapse team created virtual environments in Unreal Engine, projected them onto the LED screens and integrated them with practical foreground elements on the LED stage, ready for in-camera capture. The Brompton processors were especially helpful in two ways – multi-camera shooting, and colour rendition of low-light images.
Brompton Frame Remapping
Multi-Cam on the LED Stage
The Tessera’s Frame Remapping feature facilitates multi-camera shooting by manipulating the refresh rate of the screen and the images it displays, with the shutter speed of the cameras. Consider that virtual production workflows typically rely on camera tracking, which updates the perspective of the digital content on the LED backdrop to match the camera’s POV as it follows the live action in front of it, creating the illusion of actually filming in that virtual location.
However, this kind of tracking means the scene only looks correct for one camera, since the 3D environment is only rendered for one specific camera perspective. Any others would record the background from the wrong angle, and break the illusion.
Frame Remapping overcomes this limitation, enabling multiple cameras to each record a version of the background rendered with the correct perspective for its POV. Tessera can do this, even though the cameras are located at different positions and are all viewing the same LED screen.
Frame Remapping increases the screen’s refresh rate to a multiple of the shutter speed. It alternates the different renders as the screen refreshes, and offsets the shutter for each camera accordingly, so that it only captures the render intended for it.
LED at Low Brightness
The dark, moody look of the virtual background scenes challenged the project in another way by making it necessary to operate the screens at less than maximum brightness – which is usually the case when they are used in a studio for in-camera work. Such situations can result in a reduction in effective bit depth, and lead to visible artefacts like banding and loss of detail, especially in dark areas of the image.
Tessera’s Dark Magic processing improves image quality and colour rendering on LED panels that are operating at minimal brightness by processing the video signal at up to 26 bits per channel within the Tessera R2 and R2+ receiver cards. The effective bit depth of the output can then be increased, resulting in a clearer, smoother image with preserved detail in darker areas.
To improve further on this enhanced processing, users with compatible LED panels can choose the processor’s Studio Mode, which allows LED panels to run at a lower maximum brightness without a loss of bit-depth. Dark Magic and Studio Mode combine to achieve the ‘best possible low-brightness performance’ – that is, image quality that is high enough to support smooth gradients and colour transitions.
Christopher commented, “This 3D environment in particular was at the lower end of the lumen value, so having Brompton’s video processing be able to articulate all those nuances in the grayscale and capture that level of detail was impressive. What Brompton is doing to improve low light performance is really exciting and expands the possibilities for what can be filmed using virtual production.”
Brompton R2 receiver card.
What’s Ahead
As Synapse prepares to open the doors to its permanent home in Los Angeles later this year, the team is looking forward to experimenting with the next generation of virtual production tools and systems that are about to reach the market.
Geoff remarked, “It’s incredible what we’ve been able to achieve with the existing gear, so once we get our hands on Brompton’s new G1 receiver cards and the upcoming LED panels, I think it will blow everything we’ve done so far out of the water.”
A single G1 card can support one-million-pixel capacity for the new extremely fine pixel pitch panels. The G1 is also capable of driving panels at refresh rates up to 1,000 fps and controlling extra emitters beyond RGB with per-pixel calibration. It is among the first receiver cards to support 10Gb fibre Ethernet connections direct to the panel, and it can connect directly to the fibre outputs on the Tessera SX40 processor. www.bromptontech.com
