Production VFX Supervisor Matt Johnson spent almost two years working on ‘Mary Poppins Returns’. Work started in early 2017 with tests and animation concepts and previs. “Pre-production was a lengthy process. The director Rob Marshall likes to shoot in long takes and tends to put performance ahead of other elements of a production, including looks.
Matt had worked with the director Rob Marshall on other films, and with both Rob and cinematographer Dion Beebe together on 'Into the Woods', and commented that Rob typically used story and visuals together to emphasise the characters and their interactions. “The VFX needed to take the same approach,” he said.
“Rob avoids relying on replacing people with digital doubles, simply to work more easily with a CG background, for example. He also preferred to capture actors performing together naturally in camera instead of putting the shots together later after separate green screen shoots, even in fantasy sequences like the undersea world in the bathtub.
Stunts, VFX and Previs
“Achieving that required detailed animatics and storyboards before committing the action to previs, which was extremely important for the whole film to make sure that stunts and SFX would combine well with VFX. A previs team from The Third Floor worked with us for several weeks at the studios in London to plan exactly how to capture the sequences in a way that worked for both the director and the VFX teams.”
Framestore worked as lead VFX studio on the film, and their Art Department also got an early start in pre-production, helping to carve out concepts for the underwater world inside the bathtub, and collaborating with the 2D animators at Duncan Studio on the shooting approach for the animated world within the Royal Doulton China Bowl sequences.
Two Framestore VFX supervisors, Rob Duncan and Christian Kaestner split on-set supervision responsibilities between the various locations and studios around London, and their CaptureLab Supervisor Richard Graham and his team contributed to texture photography as well as planning for technically challenging aspects of the digital character work.
Framestore had team members on set whenever possible to support Matt Johnson. Understanding the requirements and complexity of the work in production was an important part of the artists’ work in post. Being on set helped them understand Rob Marshall’s creative vision as well. Given that the film is a musical, getting the preparation for visual effects right on the day was even more important than for a regular visual effects shoot. Also, working with children meant shooting time was limited - they had to avoid becoming a bottleneck on set.
Getting Wet
In the underwater scenes, Mary and the children were on wires suspended from a custom overhead tracking system that allowed pitch and yaw motion. A Technocrane was used for the camera work. To give the characters a 'floating' feeling at choreographed moments, a higher frame rate was used on some shots to allow re-speeds in post.
Framestore put a lot of planning and creativity into the underwater world. Considering that, in the story, this was Mary's first big opportunity to show off her magical skills to the Banks children, and that this environment was in effect an extension of a bathtub, it couldn't be entirely realistic.
Even so, the artists wanted to take advantage of the natural beauty of an underwater scene. finding the right look for the light was especially challenging, and standardising it across all types of elements in the scene. They created schools of fish interacting with coral, and realistic light effects. They also built digital doubles of the characters, not to use in the final shots but as drivers for the particle simulations they created for bubbles and other underwater particles.
When Mary and the children jump into the painted world of the Royal Doulton China Bowl, combining real and 2D animated characters in the sequences was always going to be a major challenge for the VFX artists, requiring specially constructed environments to accommodate 2D and 3D elements. This section of the film was also intended to recall the original ‘Mary Poppins’ film of 1964 in which 2D animation is integrated with live action footage using manual techniques.
Sodium Vapour Light
As a youngster, Matt said he had been fascinated by 'Mary Poppins' and learned details of the analogue sodium vapour light technique used to composite together live action actors with 2D characters and environments. The actors were filmed performing in front of a white screen lit with sodium vapour lights, which output yellow colours within a range that falls between the red, green and blue colour sensitivity layers of regular cinema film. Therefore, the lights' colours were not recorded on the film, which means the production and costume design could include a full range of conventional colours.
Capturing the scene was a camera with a beam-splitter prism, exposing two separate film elements. The main element was the regular colour negative film that is not very sensitive to sodium vapour light, and the other is black-and-white film with a fine grain and extreme sensitivity to sodium light wavelengths. The black-and-white element was used to create a matte, plus a hold-out matte, for use during compositing on an optical printer. These mattes are complementary and allow the artists to isolate the various image elements, either captured on the film or created separately by animators, very precisely so that, as they are re-exposed onto a single new piece of negative, one at a time, those images don't interfere with each other or leave gaps.
The principle is simple and it worked well, but was a painstaking manual task and had to be repeated all over again any time the camera position changed. Therefore, moving cameras, or more than a few camera angles, weren't feasible. Today, digital processing and modern compositing software have substantially changed the workflows on set and in post for this kind of 2D/3D compositing. But although the scope for creativity has widened, it still isn't easy and audience expectations have continued to rise.
Modern Cameras
For the new movie, the 2D characters were hand drawn in the traditional way, but the production knew they wouldn't be able to satisfy viewers with essentially static cameras as the 1964 production had done. Those 2D elements had been confined to just a few planes positioned at different depths or distances from the camera, to make the cartoon characters feel as if they occupied 3D space with the actors.
The modern crew, on the other hand, used steadicams, cranes and other sources of camera motion that introduced parallax to many shots, and the VFX team was able to assemble and re-assemble the shots in post. The China Bowl sequences were some of Christian Kaestner’s main responsibilities. He said, “We established our core approach to the animated world some time before the shoot started. It was crucial that everyone understood the requirements for these sequences in advance. We even went through the process of a test shot as a dry run with Duncan Studio to check that we could actually achieve the look and feel required to bring approximately 12 pages of the script to life.
“Because of the children's limited working hours we couldn't always just go back and try something different during the shoot, so we planned and rehearsed far as possible before they arrived. For example, the special effects team would work with Rob Marshall and Duncan Studio to meticulously plan the movements and motions of anything Mary, Jack or the children had to interact with, such as the carriage, cart or truck – all physically built as green screen props .”
The characters' escapades through this hybrid world are divided between a fast-action carriage ride and an on-stage Music Hall performance - with penguins. The carriage ride is all about edge-of-your-seat action, while the Music Hall features a choreographed musical number confined to a small stage. That contrast challenged the VFX team in different ways.
Real Music Hall
For the musical number - 'A Cover is Not the Book' - moving the action onto a stage introduced an interesting level of complexity. In a real music hall, a viewer might expect to see flat stage props and sets but would know that they are still real objects. Therefore, the artists 'built' the digital stage sets, textured to feel like hand drawn artwork, and placed them into the environment as if they were pieces of a stage set.
The Royal Doulton Music Hall was initially created based on concept artwork, but prior to the actual shoot, the choreography was rehearsed and the size was revised until it could be accurately built as a full green-screen set.
Rob Marshall was able to direct all aspects of the stage performance to his liking in advance. The stage lights could be programmed to the music and the positions of the stand-in penguins could be altered during rehearsals. synchronisation between cameras, music and special effects was of the essence to achieve a performance for a concept that is quite abstract. Framestore’s main challenges became handling the lighting and shadows of the 2D characters, projecting, re-projecting and re-scaling them depending on how far the were from camera.
“Once principal photography was complete, we handed Duncan Studio all the relevant camera information and a scan of the various stages of the set. The animators blocked out the action of their 2D penguins based on the stand-ins and started refining the look of the books. At that point in the process, we started replicating all of the beautifully hand crafted book textures in CG.
“The desire here was to create a visual that complimented the musical number and stays true to the hand drawn look. Creatively, the visual effects challenge was to perfect the integration of the actors into a magical world of hand drawn animation, yet leave room for us to lift the visual perception into the modern day magic of VFX work.”
When Rob Marshall made it clear early on that a purely hand-drawn look was not what he had in mind, Framestore internally referenced some of the musical numbers in Chicago. Aspects like gold leafing on the books would add real world specularity, for example, and the use of atmosphere and dust motes helped to lift the look and feel to what Rob envisioned, something that would have been impossible to draw entirely by hand.
Impossible Speed
The carriage chase, on the other hand, didn’t have the limitations of a small stage and didn’t require any synchronisation to music. The space of the forest was undefined and the story point of the sequence was to emphasize the vastness and openness of the environment - just like in a child's nightmare.
“We wanted to keep the audience on the edge of their seat while revealing what was going on in the minds of the children. Given the fact we were in the hand drawn world of a china bowl, we didn’t really have any physical restrictions on space or speed. From the photography we had a solid construct of the action beats and what needed to happen, while the perception of the forest and the travel speed was something we explored after the fact.
“Technically speaking, the carriage and truck are racing through the forest at an impossible speed of over 100 km/h, but that was the speed we often needed to create the desired impression - trees and grass are rushing past camera, emphasising the heightened danger of the children’s attempt to save their belongings.”
As in the Music Hall, Rob wanted to elevate the hand drawn animations to match the expectations of today’s audiences, but at the same time the artists wanted to stay true to the traditional 2D techniques of forced perspective and screen space animation. Essentially the characters’ motion through the 3D set – including front to back - was down to Rob’s direction. He directed both teams with whatever was required to get the shot that he wanted.
“We experimented a lot with the sharpness of the shadows on and off the trees as well as the level of realism for the atmosphere. Additionally, we selectively introduced motion blur for certain shots to find the right balance of speed and strobiness.”
Post Workflow – 2D, 3D and Choreography
When designing the post workflow, cameras, layers and 3D space were the dominant factors. They had based and planned everything on real world motion and physics to begin with, as each shot could contain up to five actors. Any sort of stylistic changes like forced perspective or 2D movements, were deliberately done in post. Furthermore, everything was timed to music, which allowed the choreography to play an important role.
“Rob’s background as a Broadway director really helped us to understand his vision while shooting. Once everything was captured in camera, we had more creative freedom to move elements around in the screen space, although the actors' performances were not altered at all. Rob was very keen to stay true to what he had directed on the day.
“Modern production techniques gave us a lot of flexibility and opportunities for creative freedom when shooting such a complex scene. For instance, we were able to sync and, most importantly, repeat the motions of the special effects rigs exactly as we needed which, in turn, allowed us to limit how much we would have to change in post - or at least to control how and what should change if desired. Ultimately, it gave Rob the ability to focus on directing his actors and camera crew.”
The process was relatively straightforward on one hand, and massively complex on the other. Initially, as described above, everything was approached like a regular visual effects shot. Standard green screen and tracking markers were used during the shoot and, other than detailed planning and programming for the motion of the rigs and animation of the lighting for the musical numbers, nothing was so out of the ordinary during production.
Once they were in post production, however, the digital workflow had to accommodate the 2D animators over at Duncan Studio working on physical media. They camera tracked and undistorted everything traditionally, but added a step in which they would prepare every plate with registration marks, so that the team at Duncan Studios could print every frame and ensure registration for each frame was correct and consistent.
Nuke Set-up Script
“We would also provide them with a Nuke script to set up the scene with very rough keys. This was done to speed up the process because the hand animation was going to be very time consuming and we needed to get them working as soon as possible.
“Once their side of the work was complete, they sent back our Nuke script with their work incorporated, allowing us to merge their work with whatever progress we had made in the meantime to key and de-spill the plates. Finally, we gave meticulous attention to the integration and shadows in the shots once we had the final backgrounds and animation.
Compared to the original Mary Poppins 2D sequence, the artists had the freedom to position everything in 3D space, even if the hand drawn characters were essentially cards. “Some aspects of the work were new to us as we normally don’t work with screen space animation and forced perspective. It was all about choreography and performance - we were there to ensure that everything looked seamless in the final picture.
“Obviously, we were after a certain style, and the forced perspective, as well as moving characters and objects around in screen space, made this process rather challenging at times. Since our environments sometimes ended up entirely in 2D, and sometimes a hybrid of 2D and 3D assets, it was essential that we re-conformed everything into a world with 3D cameras, even if it meant that some of the 2D animation in screen space required us to animate our 3D cameras.”
Words: Adriene Hurst
Images: © Walt Disney Studios Motion Pictures
