Sony, Plateau Virtuel and Studios de France created the first virtual production studio in Europe built with Sony Crystal LED screens, aligned and ready to use with Sony VENICE cameras.
Sony, virtual production studio Plateau Virtuel and broadcast set provider Studios de France have been working together to create the first virtual production studio in Europe built with Sony Crystal LED screens. Installed in a 700m2 set in the Studios de Lendit at Seine Saint Denis, north of Paris, the 90m2 curved screen measures 18m wide and 5m high. The full screen is composed of 450 borderless screen units, each holding eight LED modules, tiled together to achieve the final size. The teams are using the set in combination with Sony VENICE cinema cameras.
Crystal LED Screens
Sony's Crystal LED screens are developed to achieve a high contrast ratio and accurate reproduction of detail and motion, with a fine pixel pitch of 1.5mm. Pixel pitch measures the spaces between the LED elements, or pixels, in the video wall, and is one of the most important specifications to choose when building a virtual set.
Because it defines the pixel density (pixels per screen area), the pitch directly determines how close viewers or the camera can be positioned from the wall before individual pixels become visible – the finer the pitch, the closer the acceptable camera distance. About 2.6mm is considered standard. Pixel density also influences cost. As pixel pitches become finer and viewing distances decrease, materials and production costs increase because more LED clusters are needed.
While flat LED screens work in smaller studios, choosing a concave, curved screen for a bigger studio like this one allows wider screen shots.
Assembly and Installation
"The screen took 15 days to assemble and install, working with the Sony teams. Typically, studio screens are placed on the ground but for us, it was important to have a suspended structure in order to be able to slide different floors underneath, either LED or another kind of floor, to suit different productions. We also have an LED ceiling that helps us integrate the different elements of the set,” said Bruno Corsini, technical director of Plateau Virtuel.
This integration refers to lighting – the ceiling and wall panels are not joined physically. The primary use of the LED ceiling is to bring realistic light to a set by allowing video playback of the environment overhead. For example, if a production needs a car placed on set to appear to be moving through a landscape, playing back video of the sky or trees or clouds from the LED ceiling creates realistic reflections on its roof. The camera captures real reflections, created in real time as it records, instead of artists adding these effects in post.
"The current studio project was initiated after filming a campaign for the European Space Agency using virtual production with the VENICE camera. The production was a success but Plateau Virtuel wanted to take the technique further. They started discussions with Sony about improving their virtual production in terms of both reproduction quality and their on-set operations,” said Fabien Pisano, Sales Director, Media Solutions, Sony Europe.
Winning Combination
Having developed their panels and VENICE camera together, Sony now has an effective combination for image quality, and is one of the few companies in the world that supplies both LED panels and cinema cameras. In order to maximise their potential, the engineering teams developed and designed them in collaboration. Filmmakers can bring the camera very close to the screen without moiré effects, and the consistency between the colour spaces of the capture device – the VENICE camera – and display – Crystal LED – is very good.
The three main components of a virtual production set are the LED wall, the stage and talent in front of it and the camera. In a desert scene, for instance, set dressers might place a couple of meters of sand in front of the LED wall where the actors perform, and on which the CG content is played out – all shot at once by the camera. These three elements need to match in specific ways, and If the panels and the camera are already aligned, as the Sony VENICE and Crystal LED walls are, producers can save a lot of time in post production.
Colour must be consistent between the camera and the LED. The Sony cameras and the LEDs use the same colour gamut and, with the same light shining on both devices, both are ‘seeing’ in the same universe. This leads to fewer artefacts and less post production, which is the primary goal of virtual production. The wide-gamut colour reproduction and consistency of the LED display will help support the camera’s colour sensitivity, giving more latitude for the DP to create a specific ‘look’ for a scene.
Screen and Camera Synchronisation
The refresh rate of the LED screen also needs to be synchronised to the camera’s shutter speed, and both need to be refreshing at the same time, so that the sensor will capture the complete image on the display and not the gap in between. Since LED systems generally use scanning to illuminate pixels sequentially, instead of all at once, it’s important to be aware that the scanning frequency and sequence pattern may combine with the camera shutter in a way that introduces artefacts such as scanning bars or lines.
Developing the screen and camera together makes it possible to synchronise these factors by default. The super-fast rolling shutter on the VENICE and VENICE 2 means it performs almost like a global shutter. Further to that, the GENLOCK SHIFT feature added with a recent firmware update means the cameras and the panels can be synchronised during set-up, in order to avoid black lines or other artefacts.
During testing, engineers found that actually, many variables affected the presence of scanline artefacts, which were sometimes difficult to identify using on-set monitoring. Variables like camera position and angle, movement, movement speed, render settings – on top of shutter speed and angle and display refresh – all affected the presence of artefacts.
The final look of the virtual production is impacted by the interaction between the LED display performance, the cinema camera, especially lens and sensor characteristics, along with the real-time rendering engine, motion tracking systems and stage lighting. For instance, the relationship between the pixel pitch, density of camera sensor photosites, shooting distance, lens focal length and depth of field all influence whether moiré patterns become visible in a shot.
Upcoming Research
Sony and the other teams will continue to research how systems can be designed to limit undesirable results like poor black level performance and off-axis colour shift affecting colour and brightness, as well as moiré pattern visibility.
As work got underway with Plateau Virtuel, Sony supplied them with their own demo test system so they could assess performance, usability, picture quality and other factors, supported by sessions with the Sony teams who shared their experience and knowledge. "Throughout exchanges with Sony, we gave them very precise specifications, and they accompanied us on many tests,” said Bruno. “During the testing, we worked directly with Sony technicians both on the hardware and the software. As a result, we have developed a complete synchronisation system with the Sony VENICE camera.”
This new studio, intended for premium film and audiovisual production, is also intended to support television production, opening up new possibilities for Studios de France. It will have further potential as a ‘laboratory’ studio, giving them the ability to respond to all types of requests.
A launch event was held at the studio on 12 January 2023, and will be followed by technical workshops on 16 February and 30 March. Find more details here. pro.sony
