Juanma Nogales and Javier M Pacin, VFX Supervisors at Twin Pines talk about recreating full-CG underwater environments, an historic naval battle and a road trip through the American South.
Twin Pines post production studio in Madrid was sole VFX vendor for ‘La Fortuna’, a series streaming on Movistar+ and directed by Alejandro Amenábar. The project presented an exciting challenge for the studio, whose team spent a year on close to 1,000 shots, recreating a full-CG underwater environment, an historic naval battle and a road trip through the American South, among many other sequences.
The director Alejandro Amenábar and the studio had worked together for the first time on the movie ‘While At War’ (2019), earning Twin Pines a Best Special Effects nomination in the Goya Awards. They have collaborated again on ‘La Fortuna’, a televison adaption of the graphic novel ‘The Treasure of the Black Swan’ by Paco Roca and Guillermo Corral.
Spanning multiple periods in history, the story features a devious treasure hunter and his opponent, a young diplomat who becomes the leader of a collective effort to recover an underwater shipwreck.
To manage the complexity of the project, especially in the historical period scenes, the team began on the project in summer 2019 after scanning the physical ships that were to be used in filming. Pre-vis of the sequences followed, using the scanned material, with a study of possible camera angles carried out with the director. Juanma Nogales and Javier M Pacin, Twin Pines’ VFX supervisors on the show, talked to Digital Media World about the work.
Pre-Production and Previz
“We were involved with Alejandro early in pre-production, contributing our ideas and solutions. Fortunately, as a director, he is very clear about every last detail and communicates his vision of the story precisely, which helped us develop the concept art and previz,” said Juanma.
Previz was important for both the underwater scenes and the battle sequence. They based the previz on LIDAR scans, to make sure they were working with the actual measurements of the sets. They had scans of not just one port, but of all the ports that the production considered at that stage as possible shoot locations.
This meant making several versions of the animatics, always taking into account the position of the sun in each of the ports to prevent the green screens from casting shadows on the ships and also to determine the order of the shoot based on the sun.
Shipwreck
Javier said, “Without a doubt, the underwater sequences were the most complex because, with no existing footage to base the shots on, we had to create everything from scratch – art, environment, lighting and physical simulations, basing it all on the storyboard and the director’s instructions.”
As one of the main narrative elements in ‘La Fortuna’, the underwater environment around the shipwreck had to be both realistic and cinematic. Twin Pines created a collection of 3D elements including sand, plants, coral, rocks, vessels and cannons, which were then aged.
“We started by collecting documentation like underwater photos and videos of other wrecks, and also of underwater remotely operated vehicles (ROVs) like the one in the movie,” said Javier. “We found the best documentation in Spain’s national museums. In 2015, for instance, the National Archaeological Museum had organised an exhibition on the shipwreck of La Mercedes, the ship whose history had inspired the story of ‘La Fortuna’. Excellent material also exists in the Naval Museum in Madrid and the National Museum of Underwater Archaeology in Cartagena.
“Whenever we build up 3D environments for projects, our starting point is always narrative needs and documentation. To this, we add a strong design element to adapt the set to the story without losing realism. The underwater world is a fascinating environment with great possibilities.”
The design of the wreck was based on plans of the original ship. They decided to keep the frames and part of the masts. “Actually, it was a pretty optimistic estimate of what the ship’s state of conservation would be! Most likely, under such circumstances, hardly any traces of wood would remain. But it was vital to the story to communicate to the viewer the ROV's entry into the wreck. It is an emotional and special moment. The remains of the ship, the algae, the suspended particles... we used all the elements to give it a magical sensation of travelling to the past,” Javier said.
Illumination Underwater
Lighting was also critical to creating this feeling, taking into account both art direction and the real nature of light in water in shots at different depths, ranging from a few meters below the surface down to the seabed at more than 1,000m. At 15m, water absorbs almost all the colours from the surroundings, leaving only a small range of blue light. At 1,000m, almost no sunlight remains, leaving only the illumination of the ROV.
As well as absorbing wavelengths, the physical medium of water disperses and refracts light. The lighters’ task was to imitate the real physical conditions and combine them with the show’s aesthetics, while adapting to the suspense as it builds through these sequences.
“The ageing of underwater remains is also distinctive, and different to the wear and tear that occurs on land,” Juanma noted. “It is a mixture of erosion, chemical degradation, micro-organisms, molluscs or algae. The real secret was the work of our digital artists, meticulously texturing and scattering objects, algae, shells, sand and so on.”
They also had to create two looks for two POVs – one for the shots from outside a remotely operated underwater vehicle (ROV) and other from inside the ROV, which would be seen on the screens of the American and Spanish ships.
“From outside the ROV we could make the shot interesting by moving our camera and playing with the lighting,” said Juanma. “However, from the inside we had to stick to the realistic movement of the ROV and also to a more frontal and less cinematic lighting. To compensate, we played more with the elements suspended in the water like particles, sand and bubbles.”
Naval Battle
The impressive naval battle posed one of the biggest technical challenges for Twin Pines. As a key sequence in the story, the VFX were crucial – explosions, cannon fire, gun shots, sinking ships and a combination of real and 3D-generated ships.
Javier said, “On-set supervision is always a core part of our work, but for ‘La Fortuna’, we faced one of the most complicated shoots of our careers, to the point that, when shooting the naval battle, we had two supervisors on board the ships. It was an unusual, demanding set-up – we had a ship anchored in the port of Pasajes in the Basque region of Spain, and guided it with a tug into the ideal position relative to the positions of the camera and sun.
“Behind the ship in the port, we had gigantic green screens that we manoeuvred with large cranes. Another part of the shoot was held outside the port on the open sea, where we had to coordinate between the two ships as well as with the drone team.”
Shipbuilding
Recreating the ships was an interesting part of the project that involved a major scanning effort. Although viewers see two fleets of four ships each on screen, for the shoot the production only had one ship for each fleet and Twin Pines created the rest in post.
For the scan, Twin Pines travelled to Brittany in France where the actual historical ships were moored that the ships in the episodes needed to match. These are original ships that are still used for tourism or recreation. One of them is the corsair frigate l'Étoile du Roy, permanently moored in Saint-Malo.
The ships and the underwater robot were scanned with LIDAR to achieve high fidelity in the detail. Juanma remarked that working with the resulting volume of data wasn’t as challenging as you might expect. “After the raw scan, you have to align the point clouds. From there, the key is to work with the point cloud instead of with the geometry, and only generate the geometry that is needed, on demand,”he said.
Back in the archives of the Naval Museum in Madrid, the art department undertook an intense investigation, nearly on a level with the one carried out by the protagonists in the story of ‘La Fortuna’.
Figurehead
Scans and research in hand, the team then adapted and modified the practical ships shot on set, eliminating anachronisms and adding elements necessary to the story. “For example, for the figurehead of ‘La Fortuna’, we were inspired by the figurehead of ‘La Diana’, a twin frigate of the same type as the royal ship ‘La Mercedes’ that the story is based on,” said Juanma. “We created an original figurehead design by mixing different neoclassical sculptures from that era with the theme of Fortuna, the goddess of fortune. On the rest of the ships, we added a figurehead with the theme of a rampant lion, very common at the time.”
Finding reference for accurately depicting what happens to ships as they sink was unexpectedly easy. Javier said, “Strange as it seems, large numbers of ships are sunk on purpose for very different reasons – to create artificial reefs, as an attraction for scuba divers or simply to get rid of obsolete ships instead of dismantling them for scrap. A lot of video documentation of all this is available and it was essential for us to give those sinking effects realism.”
Battle FX
Beyond scanning of ships, places and characters, the battle sequence required hours of explosion and other simulations, rendering and motion capture, as well as detailed animatics and green screen work.
“When handling physical effects, our process always combines 2D and 3D work. We always start with the 3D layout and from there we incorporate modelling, texturing, environment design and physical simulations,” said Javier. “For a project like this, it’s not just explosions and gunshots – we’ve incorporated a lot of of water simulations, waves, smoke and other looks.
“For physical simulations such as waves, smoke, explosions and splashes, we use Houdini. When the task is more a question of environment, as in the case of bubbles or suspended elements with little interaction, we use Clarisse. Then in compositing with Nuke, we typically add and reinforce some effects with photography, either shot specifically for this project or from the library, always looking for realism and expressiveness in the shot.”
Building the seabed full of treasures, wood, corals and above all sand, was an on-going challenge that they were also able to solve with Clarisse, owing to its ability to handle millions of polygons. That ability allowed them to recreate the sand grain by grain in the closest shots and populate it with thousands of algae and rocks.
Life on Deck
Motion capture was used to bring to life the crews of the ships and the extras that fill US street scenes in another part of the film. “We recorded the motions at the studio of our colleagues from 23 lunes, and prepared screenshots of all the usual tasks aboard a frigate of the era, plus the actions necessary for the battle. For more common movements like walking and running, we could use the motion captures from our in-house library,” said Juanma.
“We approached all the 3D elements based on the pre-vis we had designed with Alejandro and we worked until the last second on improving these shots, creating dozens of versions of each part and working on different phases. It involved the greatest deployment of resources we had ever undertaken so far. The naval battle was the first thing we started on and the last thing we finished, but it was also the most satisfying sequence for Twin Pines as a VFX studio.”
Road Trip
Another major task for Twin Pines was simulating an automobile journey along roads in the US state of Georgia. Footage of Spanish roads was used for the exteriors, and a green screen shoot on the film set for the car interiors. Finalising the 3D composition included creating the road signage and the various elements of the landscape like bridges, buildings, speed barriers and cars driving in both directions.
In the most important of these driving sequences, in the final episode, the characters travel 300km from the outskirts of Atlanta to the Statesboro air base. But in fact, the whole sequence was shot in Spain. It needed to feel as though they were travelling from a less populated area toward the city. Working on the edited sequence, they added the elements needed to give each shot the right setting.
Interestingly, the shoot in Spain was done on a stretch of road that was unfinished and had never been used, which meant the compositing team even had to lay down the asphalt lines. Juanma said, “To give realism to the sequence we had to research the work thoroughly. For instance, for traffic signs we took care to follow regulations specific to Georgia.
“We investigated the types of construction materials – from retaining walls and elevated bridges to guardrails and also the highway environment with buildings, service areas, warehouses, workshops, dealerships, billboards. Furthermore, as cars on American roads are very different from the usual ones on a European road, that was another set of assets that we had to recreate.” twinpines.es
