Sony’s new Virtual Production Tool Set improves pre-production, colour calibration and on-set workflows for crews and producers using VENICE cameras, Unreal Engine and LED walls.
Sony has developed a new Virtual Production Tool Set to help improve pre-production and on-set workflows. The tools, the Camera and Display Plugin and the Color Calibrator, work with the Sony VENICE cameras and Sony’s Crystal LED and other HDR-enabled LED walls. By giving producers a clearer view in pre-production of how the virtual elements will look in post, the tools are intended to improve performance and help solve common virtual production challenges such as colour matching and moiré, to enhance in-camera visual effects.
The Camera and Display Plugin is software that allows Sony VENICE cameras to be operated virtually inside of Unreal Engine. The DP can run it on a computer and work with the VENICE digitally to determine and set up the desired cinema look on screen, as if looking through a physcal camera. Being able to dial the look in ahead of time, even while still in the software engine, represents a leap forward in terms of precision and opportunity to experiment.
VENICE and Unreal Engine
When the plugin is installed in Unreal Engine, you can determine your exposure index, check the lens and so on, and see the result in real time during a virtual previs shoot in-Engine. The virtualised VENICE in the plugin allows DPs to reproduce the settings of the VENICE, VENICE 2 and Crystal LED display. This allows the Virtual Art Department (VAD) to create assets using the VENICE's own colour pipeline during pre-production, before crews are paid to go on set.
The Virtual VENICE feature in the Camera and Display Plugin can also simulate the camera's exposure index and Neutral Density (ND) filters to recreate shallow depth of field and help productions choose lenses during pre-production.
The settings from this shoot can be saved, exported from the plug-in and transferred later to the physical camera when on set. This means that DPs do their own tech vis, ahead of the shoot, and know that they will capture what was determined, regardless of what they see when on set. Before, the viewport could be used to line elements up and have an idea of the final composition, but light and colour were likely to be different.
The plugin also alerts users to potential issues that may not be visible in-camera. For example, as individual cameras and LED walls have different colour characteristics, a common workflow problem for the VAD is the inability to anticipate the actual colour performance of specific cameras used in virtual production. Productions can use the software to identify and solve this and various other virtual production workflow issues.
This plug-in also displays a custom moiré alert during the shoot, depending on the pixel pitch and other specifications of the wall, to help productions make changes to the camera position and camera movement – a helpful tool to identify artefacts that may not be obvious with on-set monitoring but could be visible in post-production.
Color Calibrator
The Color Calibrator tool is an application for Windows 10, used to ensure proper colour reproduction when shooting LED walls with the VENICE camera. It helps overcome the fact that white point temperature tracking may not be totally accurate as you shoot LED displays on the VENICE. Different brightness levels can produce different temperatures. The Color Calibrator ensures the LED screen displays the same temperature at different brightness levels, and that the white point is precisely tracked.
Formerly, this calibration task could take two hours or more but now takes around 15 minutes and can be completed by a general technician. Through the Calibrator’s interface, users can calibrate the look of LED walls, including HDR-enabled LEDs, to the intended look of the VENICE camera, instead of relying on guesswork, reshoots or complex LUTs on set or in post.
A test pattern is recorded on the production LED wall with the VENICE camera and the Color Calibrator application automatically analyses the result. A 3D LUT is then generated which can be applied to the LED controller, colour management tool or Unreal Engine. Simple and repeatable, the goal of the process is to make sure that, once 3D content is created, taken to set and shot, the original look will come back on screen.
The initial Virtual Production Tool Set will be available at no cost in the second half of 2023. sonycine.com
