THE THREE MUSKETEERS

Previs and Post-vis
Mr X, based in Toronto, has completed several projects with director Paul Anderson, including the ‘Resident Evil’ films, and with Constantin Films, and has developed a sound relationship with Paul and the producers. Thus, they were involved with this new project from the early script reading stages and completed the movie’s previs. From there they moved to on-set supervision and started working with the editors on critical sequences, providing post-vis animatics to help stage the action correctly.
“Previs served two roles,” Eric said. “One was artistic giving the director a chance to plot out his vision and visualize the script beyond storyboards, but the other was technical. Some of the film’s more dramatic camera moves, for example, could be planned in previs. In a shot where the hero d’Artagnan is entering Paris along a bridge, from a location shoot in Bavaria, the crew was using a 50ft Technocrane, which Mr X built in Maya to determine how it could move to give desired coverage within a very limited space on the bridge. Paul would sometimes work with the animators designing cameras moves, which might later be enhanced with post moves in the composite.”

City Views
No production shooting was done in Paris, only research and texture shoots. Historical maps and illustrations were used to re-create the 17th century city, with modern maps and Google to fill in the gaps. “Initially, the majority of our Paris backgrounds were intended to be 2.5D digital matte paintings, but during production the camera moves grew increasingly aggressive. At one point during spring 2011, we realized that we needed to re-think our approach. Eventually, most of Paris ended up as matte painting assets baked down onto 3D geometry and lit in our sequence lighting rigs - or as true 3D matte paintings,” Eric said.

The magnificent airships, which according to the story were based on Leonardo da Vinci’s own blueprints, were among the team’s top CG challenges. As they sailed overhead, they were almost always viewed from below. The larger ship’s crow’s nest area is only viewed from above once. The team’s first task was to sell the height and scale of the ships, 300 to 400 feet above ground at any time. “This became especially challenging in the climactic battle sequence when the ships are temporarily lost in cloud during a storm and the ground is not visible as a visual reference. As the sequence progressed and the cloud cover opened to show more land, we could prepare viewers for shots later on over Paris where the airships are much closer to the ground.”

Airship Design
The airship design was extremely complex and originated from three sources. Art Department concepts were the first step, from which scale-model maquettes were made, at the same time as partial sets were under construction of the decks for the live action shoot. This sometimes gave the team slightly conflicting references to work with, because a set extension based on the maquette, for example, might not accurately match the set build. They had to walk a fine line between these two references.
Furthermore, a fair amount of detail was not covered by any of the three sources. The scale model wasn’t able to represent the 80m airship in fine enough detail, and interior details were often not represented anywhere, so Mr X had to pitch some interior designs to the director based on looks in the concepts. They also had to invent some of the mechanicals such as the operation of the sails, rigging or rudder. Because adding detail contributed to the sense of scale of the airships, they modelled individual cannonballs, boxes, coils of rope, tables, barrels and rivets. In one scene when the Duke of Buckingham’s airship lands outside the Louvre, the camera moves so close to the rudder that individual scratches are revealed.

Scaling in Stereo
Working in stereo made conveying scale more complex. The convergence and interocular distance on their Maya cameras had to be set to make the airships feel large whether they were close to the viewer, 100m away or up in the sky. Their experience from a previous project - shooting stereo footage from a helicopter using RED cameras on a rig with a 6in interocular distance – helped them understand how much stereo is in a scene and build an effective CG camera set up accordingly. Nevertheless, a compromise always exists between realism and drama for this kind of work.

Showing ripples of wind affecting the fabric of the sails and the ships balloons further helped convey scale. A cloth team built tools into the balloons so the animators could adjust how much effect the wind would have on the sails and balloons whether they were in stormy or calm skies, or landing and stopping.

The environment, weather and lighting was always important aspects of storytelling in this film, for a few reasons. The clouds afforded attractive compositions and expressed dramatic cues for different levels of tension or humour in a scene. The Musketeers use the cloud as a place to hide during the stormy battle. The matte painting department developed several concept paintings for the storm and from these, the digital FX team projected layers of clouds onto geometry.

Dramatic Skies
Using layers to take advantage of the 3D not only made the shots more dynamic but allowed them to produce a more convincing stereo effect. They tried to place objects in the foreground whenever possible. The aerial camera view, as it looked at the ships in the mid-ground, included wisps of cloud passing very close to camera as the foreground, with the background surrounding all of this. This set up produced the immersive quality in the 3D, drawing the audience into the picture, especially effective for CG green screen shots and set extensions.

Scene by scene, the skies would change from dark and overcast, to blown-out to beaming with sunlight. The airships’ journey to Paris needs to reach a climax as they pass through the storm and the team enhanced the drama through the lighting effects in the sky. When D’Artagnan triumphs at the end, the sky glows at the dawn of a new day with god rays shining down from the sun.
Consequently, a custom cloud system was developed for the project, starting with a large matte painted hemisphere as the background layer. The darker part of this sky indicated south, the light part was north, in line with the story. Over this background and mid-ground, clouds rendered in Houdini from the effects department were added as required.

At the heart of the storm these clouds and mist would move in around the airships, using a more complex simulation to allow foreground mist to curve around the CG objects. Meanwhile the team had to ensure everything was correctly layered to work in stereo. When the airships flew into a cloud bank, all elements had to be lined up perfectly to hold the correct illusion of disappearing into the clouds.

Photo Survey
The build of Notre Dame Cathedral started with sending a team to Paris for a photo survey, supplemented with historical documents. A matching Notre Dame cathedral is also located in Rheims, where another team was sent to complete a photo survey with RED cameras for an accurate sense of scale. They had to take some license with the design, partly because the set build wasn’t an exact replica of the roofline, and certain camera angles had to be adjusted to create a suitable back drop.
When one of the airships collides with the cathedral, a dramatic dynamic simulation had to be set up for the roof tiles breaking off and the damage to the roofline. To prepare the asset for this, all affected interiors had to be completed and the structure made, in effect, ‘water tight’. While it was modelled in Maya, the effects simulations were done in Houdini, so the models had to work properly across both applications, which required attention to all details and continuous checking for compatibility down the pipeline.

For example, if a CG cannon shot a ball into an airship, the wood needed to splinter and fly off following realistic physical behaviour from all shooting angles. These simulations were computationally expensive and often tied up the render farm as they worked out the best process.

Rank and File
The film’s version of the Louvre was based on a shoot at an historic Bavarian location, Würzburg Palace. One of the camera’s approaches to it was a long cable cam shot using a cable that passed over the building starting at ground level, rising over a statue up to a balcony where the king and queen were standing. The avenue leading up to it and the forecourt was filled with thousands of digital soldiers in rows, standing at attention.

“Fifty live action soldiers were included at the shoot but 24,000 more were added digitally using Mr X’s own crowd system and animation cycles, resulting in ranks of living, breathing soldiers. Each was made to subtly shift weight or slightly sway, while the ranks were interspersed with isolated actions to add further signs of life.

The Palace building standing in as the Louvre was a photographic element, but the grounds around it were CG, replacing the location’s current surroundings of a modern city. The element was isolated and a forest was added behind with the Seine River along one side to form the aerial view. The gravel forecourt where the soldiers stand is now a car park that the team made about five times larger. The environment was created from textured assets and projected or conventional matte paintings, always varied according to the proximity of the camera and the need to address parallax.

Duelling Doubles
The fight between D’Artagnan and Rochefort on a precarious rooftop at Notre Dame required a hybrid approach to CG and digital double replacement. The shot of D’Artagnan swinging into view, Tarzan-style, onto the roofline was full CG with both actors as digital doubles. “Almost all wide shots used doubles, combined with a few plate elements. Others needed almost everything replaced - except the actors. Chunks of the set build, such as the peak of the roof, could be included for them to stand on. A different solution was required almost anytime the camera angle changed.

“Another fight at the Tower of London features an explosion bursting through the Tower wall to the exterior. A flaming Beefeater falling from a burning window is actually a photo element while the hero D'Artagnan, leaping to safety onto the airship above them is replaced digitally. The Tower itself is also CG, and the small portion of the airship was in the plate and extended.

“When we researched the Tower for this shot, we did the usual book research and photo survey on foot, but also surveyed from a helicopter to give us reference at the same aerial camera angles the director wanted in the CG shots. An action movie needs shots with dynamism,” Eric said.

Global Illumination Pipeline
This was one if the first movies Mr X has worked on using a full global illumination pipeline. “Earlier, GI was too expensive on the render farm to be practical”, Trey Harrell explained. “But since adopting V-Ray as the primary lighting renderer, the GI is virtually free. It adds a pleasing aesthetic to the shots and was used on almost all the shots for ‘The three Musketeers’.

“Traditionally in VFX, particularly with Renderman style pipelines such as those based on PRMan, Mantra or 3Dlight, artists would light with spotlights and try to mimic light bouncing around the room, or sunlight or fill light from the sky bouncing up off the ground. But now with physically based lighting tools artists could add for example, a Kino Flo 4 Bank to one side of a scene and have it render as if that lamp had been on set. They can achieve multiple light bounces intuitively without resorting to traditional ‘cheats’.”
Using GI also meant the digital doubles, airships and other assets could be surfaced and textured and calibrated to behave consistently under any lighting set up. Eric had spent a week in Germany on set to carry out texture scanning of the actors for digital doubles. They built a digital version of this scanning stage, taking all of their texture and photo references for doubles and props, recreating the process in Maya and V-Ray, including the exact same lighting set up used during photography, to be able to make objective comparisons with the reference photography. The approval process could thus literally be looking at the real photos right next to the CG, sometimes head to waist.

Aerial Views
Because of the airships, ‘The Three Musketeers’ involved aerial views and approaches to various locations, like the Louvre and the Tower, and the team had to assess several factors when deciding how much photo imagery should be included to enhance the photorealism and how much to rely on CG. It depended on how close to camera they worked, how much detail was needed, and how much parallax was in the shot. CG trees were a particular challenge when trying to introduce enough natural looking motion into them to prevent a static look without whipping them into a storm. Too subtle motion at a distance often looked like noise.

During the aerial battles, some of the full CG shots showed cannons firing into the airships, fire erupting on impact, smoke flying past the sails and dissipating. Each hit ended up being custom-made. They had intended to build up a library of cannon fire effects but the individual camera angles in one shot meant creating hero simulations for each one.

Compositing was done in Nuke, in which the stereo tools were especially useful for subtly changing the stereo position of objects in a scene to improve the balance and focus attention. For example, when the airship crew begins throwing gear overboard to lighten it and the objects fall into camera on the ground, the director wanted to give the shot more impact. They were able to use Nuke tools to give more offset to objects to make them fall further into the audience. These tools also helped correct anomalies in the plate photography such as locational offset and keystoning.

The light was also vital for the VFX as they involved so many CG assets. Lighting for realism is one thing but they also needed to communicate scale and mood and to give the audience clues for tension, optimism and humour. The best approach was to start each lighting scenario with the onset light data as usual and they proceed based on emotional cues. Few sequences ended up sticking strictly to HDR references.

The amount of texturing and detail on the doubles, airships and Paris itself demanded an upgrade to their entire surfacing and texturing pipeline. Mr X was one of the first effects houses to purchase licences of Mari from The Foundry, which was adopted at the very beginning of the project. Without it, Trey doesn’t think they could have completed all the work, with as few staff as they had, in time.

Balloon Textures
An example of the volume of texturing is the balloons on the airships. “One of these balloons contains 64 4K texture tiles with very high texture resolution, because in some shots the camera is just inches from the surface. All digital doubles and the hulls of the airships again represented very high texture resolution due to camera proximity combined with the demands of stereo – scratches and bolts all had to be visible.

Many approaches were applied to lighting a given scene, to set up interactive lighting for cannon blasts, for example, the ships might be rendered in V-Ray, explosions and smoke in Houdini while the debris was simulated in Houdini but rendered in V-Ray so the y would cast accurate shadows and bounce lighting back into the shot and mesh with everything else. At other times, the traditional methods worked, like throwing in an orange spotlight to light up an explosion in the side of a ship.
When lightning storms hit, they might render out lightning flash passes but they also now have a re-lighting pipeline. Trey said, “The compositing department has begun using 2.5D and true 3D compositing a lot more in the last few film projects. In one situation on ‘Three Musketeers’, compositing artists could interactively add lightning flashes from arbitrary directions, for example on top of Notre Dame, to match with set lighting and CG lighting as well.

Hybrid Escape
On another occasion, a giant airship passing over the Louvre was handled with projections from shadow passes rendered out of their daylight system – or they might find traditional roto mattes worked best. All such decisions were made on a per shot basis, choosing the most efficient approach that would make sense and work for the show.”

In one layer, interactive light from the explosion affecting the double and the side of the ship was handled both as a V-Ray render pass and another Houdini render pass for fill light, then the re-lighting pipeline was used, where the compositors can place lights and flashes into a scene for detail, plus extensive moving roto mask work.

Twice the Time
Trey talked about timing in a stereo 3D project like ‘The Three Musketeers’ on which their teams were working from the very start. “Despite, their frequency these days, 3D movies can still easily take twice as much time in post production as conventional 2D films, especially from a rotoscoping and compositing perspective. Stereo rotoscoping required on true stereo productions is specialised and painstaking. From a lighting point of view, it can likewise demand twice the intensity of effort - much of this due to render farm demands.

“On ‘The Three Musketeers’, asset artists and animators were onto the shots from previs to help design them and once they were approved and shot, the rotoscoping department started immediately on clean up for hair and mattes, pulls keys, and makes decisions on where to seam up live action with CG material.” Then models are surfaced and textured and built while the asset and lighting teams work out look development to balance the final shot, and may have to pitch some looks to the VFX supervisor and director.

Getting to the stage when effects animation is approved might take three months, after which tasks tend to split between the effects and lighting teams. After several more weeks of work, shots are presented in dailies and revised until they are right. “Planning definitely pays off even on shots that seem easy because you always end up having to think on your feet on the ones you least expect,” Trey said.

The god rays and flares were achieved in comp. Flares in particular are very tricky to pull off well in stereo, when they are not equally visible in both eyes. It’s preferable to achieve these types of looks in comp where they're much quicker to dial in.

The wide shots, among the most distinctive and memorable in this film, were a new direction for Mr X due to their complexity. “The airships with their millions of polygons plus multiple CG characters on board represented huge amounts of data to push through the pipeline. Wide shots of Paris actually required writing propriety tools to handle so much data at one time. At times, rendering in stages is preferable for the compositors because it give them more flexibility, But in most cases when we needed to we managed to optimize the data enough to render entire scenes at once.”

Parting Shot
The final shot is a view from over the open sea, populated with a fleet of graceful sailing ships was very complex to put together, starting with a matte painted environment so that the water would reflect and refract it correctly. The cameras were positioned to show a bright area of the sun creating a strong highlight on the water that shows off its movement, achieved through several layers of animation. Large breakers 10m long were combined with secondary and tertiary ripples travelling with the wind with enough detail to indicate the camera was about 100m high at the end of the shot.

Meanwhile a dynamic simulation of the ships’ hulls moving through the water was required. This level of extra detail was needed to fix the relationship between the plate element – onboard the hero ship where Buckingham stands – and the team’s virtual camera as it pulls up and back. Instead of moving the ships through the water, establishing this allowed them to simply move the water past the ships. They also had to be careful about adding motion to the ocean, as distorting the plate would affect the photo element of one of the film’s stars.
The fleet of ships was laid out in a carefully distributed wedge pattern, balancing out a fleet of airships overhead in the sky. The final composition of 46 airships and 54 sea ships had to look impressively full and pleasing as the camera pulls away. The ranks of ships also needed enough variation in sail animation. Generic cloth simulations or caches could be baked into the rigging. The duration of these were prolonged to allow the offset to be changed between the various sails and avoid repetitive motion.
Lighting Supervisor Trey Harrell and the lighting team had a special involvement on this shot. “This was a huge collaboration across teams for us,” said Trey. “The water was rendered by Jeremy Dineen in the effects department, comprising hundreds of 4K water tiles fully simulated inside Mr X’s proprietary water system inside Houdini. The sky was based on a matte painted sky dome to create a very stylized sunrise scene, including the composition and colour palette the compositing department added sun and sun flares. The sun’s reflection was rendered in both the effects and lighting departments.

“All air and sea ships balloons, sails and flags were sent through the cloth department and then back to lighting. All of Buckingham’s airships – with 48 million polygons in each - were rendered simultaneously, taking about 12 minutes per pass. Careful planning kept this sequence from becoming a problem, as everyone almost expected.” Lighting of the water was in the end relatively straightforward. A proprietary crowd system was used for the ships, rendering several hundred hold out mattes for the compositing team to affect each ship individually down to the colour of the metal on them, and give different dirt maps to each ship, to balloons to weathering maps and to sails.

“Knowing in advance how challenging the water would be and showing this shot’s epic nature without revealing repetition, we decided to push the revision into the compositors’ arena. It was a 700 to 800 frame shot and this much control was very useful because to have to re-render would have been too expensive,” said Trey. Following such a huge planning effort, they succeeded on just version three.

Words: Adriene Hurst
Images: Courtesy of Constantin Film