The team at Cinesite opens ‘Pirates of the Caribbean: On Stranger Tides’ with a thrilling chase through the streets of old London, led by VFX Supervisor Simon Stanley-Clamp. At ILM Singapore, VFX Supervisor Mohen Leo’s team gave Blackbeard his ability to capture pirate ships in bottles, trapped forever on the xuất hiện seas.
Bạn đang xem: Pirates of the caribbean
|f the over 300 stereoscopic visual effects shots Cinesite created for ‘On Stranger Tides’, roughly 200 were required for the daring và comical carriage chase scene through London. In fact, according lớn the production’s original storyboards, the scene was intended lớn be even longer but a large chunk of it was cut out before the shoot as the director Rob Marshall continued lớn change his mind.|
First CutStarting their own plans with the original storyboards, Simon’s team took care to lớn ask for the first edit early on because the post schedule on the sequence was very tight, among the first shot and delivered for the London location. The production also needed a rough cut back in the meantime to kiểm tra how the edit was working. “That first edit was very long – over 10 minutes,” said VFX Supervisor Simon Stanley-Clamp. “Then it was cut back a few times before arriving at the 5 1/2 or so minutes you see in the final film.
“While awaiting the first edit, we went on set to gather what we needed khổng lồ start designing CG buildings & environments khổng lồ replace the very large blue screens in place at the three locations in & around London. We based the CG on existing architecture, often in side streets and mews. Well in advance of the plates arriving, we built 3 chiều assets và tested our photogrammetry techniques in csPhotoMesh, the facility’s new software, khổng lồ prove that we could capture stills of a building, model from the stills và create an effective environment.”Once they were awarded the sequence, they had spent a week surveying the film’s main London locations at Greenwich, Hampton Court Palace and Middle Temple, plus the sets at Pinewood Studios in the docklands. The team captured numerous lighting and texture stills, including before và during set construction at Greenwich where Simon stayed on mix each day of the three week shoot. They also managed to vày a full mix survey at Middle Temple, where the shoot had to be completed in a single day khổng lồ accommodate the lawyers working there. The entire London section of the film was shot over three months.
Blue Screen StreetsBecause the edit was uncertain for some time, the team wasn’t able lớn previs but used their stills and data capture lớn pre-build much of the architecture they would need for the different environments. For example, Greenwich had a distinctive look for which they built up a library of a dozen building variations, approved và ready to drop in, before they actually had any footage.
Street layout came from the stills, the storyboards and a few production designs, which Simon extrapolated. This helped them anticipate where they would shoot from. He would send back concept stills khổng lồ production, showing the buildings he suggested to lớn fill gaps & replace xanh screens, sometimes with options. Few changes were made lớn these looks. The main criterion was that the buildings simply be completely unnoticeable and look ‘right’ for the situation.The blue screens in some spots were enormous, in one case requiring that the team replace a full street of CG buildings. The justification for it was economic. The street was only needed for two or three shots, và an equivalent practical mix would have been huge. Another massive xanh screen was employed for the opening shot of London, passing through an arch with St Pauls cathedral in the distance, và others had lớn be replaced at either end of the Greenwich location khổng lồ extend into the distance.
Pre-CompositesThe first plates arrived in October & November. The team helped production trim the cut down và figure out was needed, shot by shot, by making rough pre-comps – taking still frames, putting them on cards & dropping them into position – so the editors could see the shots without blues screen and hone the looks as they liked. “It wasn’t true previs, but it wasn’t a finished comp either. The process also helped us decide what elements had khổng lồ be shot,” said Simon.
“People walking through frame alone or in groups, shot on xanh screen, were supplied abundantly for us to lớn randomise in the plates, all shot in stereo. Later on, fire và smoke elements were shot in stereo as well. One critical smoke element was commissioned và shot only days before the sequence delivery. The story needed extra smoke as a neat escape route for Jack Sparrow, allowing him khổng lồ swing undetected from a pub sign, but we didn’t have enough time for CG smoke development. We match moved the plate và layered up multiple bits of the practical smoke.”
“The first 10 or 12 minutes of the film’s opening London shots, coming across the water và leading into the sequence in King George’s palace before the carriage chase, were part of our award, the film essentially begins with our work,” said Simon. “We were even responsible for modelling and animating Jack’s CG cream puff for the palace dining room shots, which gets tossed around until it gets stuck on the chandelier. This chandelier was a massive, weighty practical prop controlled by giant pullies và motors that had to tư vấn Jack. But it didn’t swing naturally, so apart from Jack’s rig removal, we needed to carry out variable respeeds to give its motion the expected look.”
Variable RespeedsUsually variable respeeds are frame or vector based for the smoothest interpolation. But in stereo, the associated clean up must extremely thorough & precise. They built a tool in Nuke that would vì chưng a ‘nearest’, vector or blended respeed, or a combination of any of these as required. They could feed in a QuickTime reference, get an exact match & choose whatever process worked best. Then they made sure it was performed exactly the same way in each eye.
Simon feels that, at this stage in stereo development, respeeds are best kept khổng lồ a minimum, especially the variable type. “Sometimes they have to lớn come back & be redone to lớn make sure they look right & match the cut. They can even affect the audio because of the lipsync. The editors were usually quite helpful on this,” he said.
|The Stereo HurdleCreating CG elements for stereo 3 chiều footage created a significant learning & equipment hurdle for Cinesite. The production shot virtually in parallel, slightly toed-in, which allowed convergence during compositing. In other words, convergence is not baked into the footage but can be pulled forward or back. The team extrapolated convergence data from the plate and, using 2 chiều tracking, generated a stereo track and piped this into their Maya scenes.|
“This way, what we rendered is an exact match lớn each eye. Then we can use tools in Nuke, a nudge tool, for example, lớn adjust và fine-tune shots so that all elements sit at their true stereo depth. But generally, rendering & tracking tasks simply take twice as long, which made the whole project take substantially longer than it would have as a 2d project. However, this was something that the production và vendors were all aware of from the outset.” Tracking was usually done with 3 chiều Equalizer or a Nuke track to lớn finesse elements into place.Consequently, from shoot to lớn post, emphasis was on avoiding too much clean up work, making sure that what got into the plate was supposed to lớn be there. Stereo cleanup means painting things out in precisely the same way in each eye khổng lồ avoid artefacts và floating masks that attract the eye. Perhaps what gave us the most help was having their own stereographer on board, almost from the start of the shoot. As the plates turned over, he analysed them và picked up errors the team could correct themselves.
StereographerSimon said, “Convergence information wasn’t always immediately provided from production, causing delays, but the stereographer could advise us on the best convergence to lớn set lớn allow us khổng lồ go ahead. It was really helpful, especially combined with the in-house tools and using Nuke for all of the compositing. Being able khổng lồ produce stereo QuickTimes for client review, for example, made a lot of difference to lớn the workflow.”
Equipping the facility was a major but essential step. All compositors now have their own 3 chiều monitors & Cinesite’s main theatre has been converted to Dolby D. A dedicated suite with stereo viewing system was built for ‘Pirates’. “You can’t guess about the images. They all have to be checked in the same way that they will be viewed in cinemas. The stereographer needed his own suite with monitors as well. Virtually every project now has a stereo agenda or deliverable, và now we are set up for it,” said Simon.
Changing LightAs the chase sequence was shot during the northern autumn, October và November, light was not consistent from set khổng lồ set. Greenwich featured flat white-greyskies, which was a good match for the classic, gritty old London look specified in the early stages. Middle Temple was shot within the same period & didn’t cause many problems but over Pinewood’s exterior set, the skies were bright & sunny almost every day. This meant hanging huge diffusers over the sets lớn prevent sharp shadows & where the diffusers failed, the team had lớn remove shadows, re-grade some shots & replace a few backgrounds with murkier environments.
Slight rainfall at Greenwich wouldn’t have been an issue except for the stereo factor. The Pace rig the production were using has an exposed polarising mirror at the front. If rain falls on it, the drops appear in the footage as floating artefacts in the foreground. Cleaning these out in post is very time-consuming and expensive. At the shoot such problems were sometimes handled with eccentric measures like driving the rig backwards down the road, which gave them reversed shots but kept the rain off the mirror.
A very talented Jack Sparrow double performed many of the trickier stunts for actor Johnny Depp, và other stunt men stood in for him on specific manoeuvres. “But whenever he is recognisable as himself, it really is him – there were no face replacements for him. Instead stunt rigs were often used that required substantial digital removal from the stereo footage. In one case they had lớn replace a digital building to make sure the cable was totally cleaned out.
Frog EngineeringThe pirates encounter poisonous frogs in the jungle, which Captain Barbossa captures in a jar. For the scene, Cinesite created and animated a full CG frog complete in every detail – a complete rig, wet eyes, in red, yellow, green & blue. For its size, it’s highly over-engineered,” admitted Simon. “You might even miss it. It was one of the first shots we completed, based on plates from the Hawaii location. I’m always more confident when I can survey the mix myself for tracking và lighting data but since we didn’t have anyone working over there, we had to rely on the set information supplied lớn us.
|“The animation in particular took a long time to lớn lock down. The frog drops onto the actor’s shoulder, falling from overhead, jumps from one side of the frame khổng lồ the other & was such a tiny nuance of animation, in seven tricky shots. Although we kept the glass jar in the plate for the composite, we still had to mã sản phẩm a 3d jar, building the glass with depth so we could create passes khổng lồ reproduce the refraction and reflection that would occur with a real glass jar.“When you look through the jar at the frogs, they ripple & change shape accordingly. We spent months on it, starting before the shoot with about eight different designs in myriad variations but the director wanted a simple, realistic frog. We completed four iterations, some with exaggerated limbs or other parts but the result is virtually true lớn life, a real poison-dart frog with a puffed throat và lens movements in the eyes.”|
Walk the WalkBarbossa’s peg leg was an effect that appeared in most vendors’ shots. They shared out the work on it within their awarded shots. A physical peg leg was built as a looks reference and, again, stills of it were captured on location, plus texture reference shots, measurements và a cyber scan for an exact build & texture. Simon had shots taken of it just after each shoot – sometimes it appeared close-up – with the RED cameras to record it exactly as it had looked in the rest of the sequence under the same lighting with all the actors và gear still in place.The rig was straightforward. They had built a mã sản phẩm of Barbossa, & used it for shots when he was walking around, match moving it into the shot and animating only his lower torso performing the required walk. Not surprisingly, the way the actor walks and the way he is meant khổng lồ appear on screen are different. The leg was meant to lớn be rigid from the hip and unable lớn bend at the knee but as Geoffrey Rush walks around in his blue sock with the tracking markers, he does bend his knee.
“The cleanup was the tricky part, often needing a bit more than only the peg portion. The animation required a few trials khổng lồ lock down while making sure that leg was perfectly straight! We weren’t match moving what the actor is doing but what it would look lượt thích if he weren’t bending his knee.
No Magic“The hardest shot was Barbossa’s first, when the camera follows him into King George’s palace hall & he is seen virtually in silhouette. This made tracking và clean up harder, and we needed lớn make a replacement marble floor to reveal reflections and shadows from the leg. Invisibility và realism was crucial in our shots. Our team was not dealing with magic, lasers và collapsing buildings.
“We had all lighting scenarios to giảm giá khuyến mãi with as well. When Jack clashes with Angelica, posing as himself, in the sword fight in the Captain’s Daughter pub, the fire-lit interior is nearly dark. As the pair fight in the rafters overhead, they are wearing complicated rigs against the smoky ceiling behind them. To lớn clean out the wires, we replaced the roof in CG and we also built the CG barrels you see in the background.”
Frozen in TimeThe ILM Singapore team headed by VFX Supervisor Mohen Leo mainly took on a supporting role for ILM’s artists in San Francisco but two intriguing sequences were handled entirely in Singapore. The team needed to create miniature pirate ships, complete and realistic in every detail, inside a number of glass bottles. In the film’s story, the nefarious Blackbeard possesses magical powers that allow him lớn seize and shrink these ships khổng lồ capture them in bottles. Furthermore, he traps each ship with precisely the same ocean conditions of its capture, freezing that moment in time & in miniature.
Although Mohen’s team had no one on set, he spent a month in San Francisco working with ILM’s supervisor there, Ben Snow, before the shoot. He took the opportunity to discuss technical approaches for the bottle sequences và what data they would need from set to lớn complete the work in post.
Their main sequence involved Blackbeard’s full collection of bottles and takes place below decks on his ship. To lớn get started on looks và aesthetics, the team focused on live kích hoạt props in the plates – some model ships và a cabinet holding empty bottles, which the production crew had supplied as practical elements for the wider shots. In CG, the ILM team replaced the models, built ships and environments for the existing bottles, và built a large number of CG bottles with ships và environments inside.
Magic MomentThe team also needed lớn build a variety of ocean conditions, from calm seas to heavy storms. As the work got underway, the idea for the sequence evolved. The director Rob Marshall decided it would also be good lớn preserve the time of day and lighting conditions of their moment of capture, some in broad daylight, some at sunset, others at night.This was particularly challenging because ultimately, they all had khổng lồ go into the same overall environment of the background plate và tie into it. ILM’s concept artist John Bell prepared a few concepts and the Compositor Paolo Acri did some look development on one of the wider shots và started roughing in ideas for different conditions, light and other looks to lớn show the director.
In the end, the project went one step further when one compositor, Ben Warner, tried putting canon fire & small explosions on one of the ships as if it were trapped in an endless battle. The director really liked this idea – it wasn’t just capturing weather & time but also a narrative moment in the bottles, so that each ship possessed its own story.This resulted in one ship covered in ice caught in a snow flurry at night, for example, and Jack Sparrow’s own đen Pearl still had the parrot & monkey from the previous film, held in an eternal lightning storm with waves crashing against it. Within that scope, the team had tremendous creative freedom. The client gave them opportunities khổng lồ try out different ideas & Rob chose the ones he liked, even a completely sunken ship with fish swimming by.
Lighting InteractionFor the vessels themselves they took inspiration from the model ships the art department supplied, plus ships they had in their library after working on all three previous ‘Pirates’ movies. They started with these resources, though Rob eventually diverged from the set design slightly and requested a Chinese junk in one bottle. They constructed close to lớn a dozen different CG ships within CG bottles.
The representation of the bottles changed from shot lớn shot. For close ups, practical bottles in the cabinet were removed to lớn make way for the fully CG assets, & for the camera when taking reverse angle shots looking back at the actors as they discussed the ships. In these close shots, the lighting interaction between the ship & the bottle needed lớn be carefully designed and controlled, which was best achieved when both were built in CG.In wide shots, the crews’ live action bottles were left on the cabinet và could be retained as secondary bottles, only briefly glimpsed, with the team’s CG ships, oceans và weather placed inside & motion applied. The team really enjoyed this chance lớn devote their efforts lớn something purely ‘pretty’, with almost more detail than could be appreciated in one viewing.
Ocean LooksA second sequence occurs on the beach at the over of the movie, when Jack’s old friend Joshamee Gibbs returns the black Pearl in its bottle lớn Jack, who must then figure out how khổng lồ release his ship. The light in these shot presented a reversed lighting scenario, but the same challenge. Mohen said, “In order lớn let the ships keep their own time of day & still visually fit into the background plates we had lớn bridge the two lighting scenarios. In the dark sequence under deck we had light from the brighter bottles spill out onto the cabinet, while in the beach sequence we had the warm sunlight in the plate pick up some highlights on the black Pearl in its otherwise gloomy overcast environment in the bottle.”
In all, the team completed over 40 of these ship and bottle shots. The shots were composed almost entirely of the plate material and their CG, with some rotoscope work from the plates and a few xanh screen shots of Jack’s face shot later in the schedule. The CG pipeline at the facility comprises a combination of in-house và off-the shelf software. Modelling is done in Maya, creature simulations, effects và lighting are proprietary. Texturing, rotoscoping and paint are all done with a combination of tools. Compositing is in Nuke.Effects lead John Kilshaw based the water and fluid simulations on ILM’s water software used on the third ‘Pirates’ film & more recently on ‘The Last Airbender’. They began with stormy, choppy và calm water looks. From there, the spray, mist, foam và underwater bubbles from churned up water were developed. All these simulations were run & placed into the individual bottle environments with the ships. By mixing and matching the ocean and lighting conditions, they generated numerous different looks for the different bottles.
Unforgiving“The stereo 3d element of this project added a few special complications for our work,” said Mohen. “ILM had stereo experience from ‘Avatar’, of course, but this was the first time they were involved from the start và on such a large scale. Also, this was shot completely in stereo, with no conversion work. It was an opportunity lớn advance the stereo pipelines for compositing, for camera match move and layout, even for rotoscoping.
“One challenge was simply how unforgiving stereo really is for a VFX artist. The small cheats & inaccuracies are no longer viable. The viewer’s brain knows naturally how to perceive stereo vision, thus every detail và mistake is instantly recognized. Even a pixel or two out of place between left and right eyes will spoil the depth of a shot. It demands complete accuracy và the quality control is time-consuming.”The fact that the bottles were made of thick hand-crafted glass that refracted within the bottle added a separate challenge. Had they rendered all their CG assets with those refraction effects built into their looks, they would have been inflexible in the composite. Nothing could have been added, such as practical elements, or tweaked.
Independent RenderConsequently, lighting Lead Simeon Bassett and Compositing Lead Jon Bowen modified their shaders và Nuke mix ups in such a way that allowed them lớn render the bottle contents, especially the anh hùng shots, without refractions from the glass and then apply them in the composite at the very end in a way that still worked in stereo. The result looked correct while allowing enough flexibility khổng lồ adjust the looks exactly as required, independent of the refractions.
Compositing was all done in Nuke. Tracking was, again, done with proprietary tools but the stereo made it quite complex, demanding absolute precision. The layout team was finding the stereo plates substantially harder lớn manage but they have now advanced their pipeline & feel they can move onto other projects with more confidence.
“It’s not just the tools but the artists themselves who advance. We all learned how to judge stereo images with our own eyes – at first, we were all tempted to rely on 3d glasses for this, but it’s not always the best technique lớn identify errors. Once the eye and brain are trained, it can be much more effective khổng lồ just flip back and forth between left & right images – without glasses. By detecting shifts between them, you can determine exactly where và why errors are occurring.”
|Meshing và MatchingHead of visual effects technology, Michele Sciolette, led Cinesite’s efforts to build the stereo production pipeline & develop new tools to address specific challenges. These included csStereoColourMatcher, an automated tool lớn compensate for colour differences between stereoscopic image pairs. The environmental artists were using csPhotoMesh khổng lồ rapidly build up the large CG sets.|
“csPhotoMesh is photogrammetry và 3D scene reconstruction software, which was useful for building the carriage chase sequence environments,” said Michele. “It is a simple, flexible way lớn capture geometry. Given a set of digital images of a static scene, it produces a textured 3 chiều mesh accurately representing the scene geometry and 3D cameras matching the original photos’ positions. The function is automatic – you just drop all the images in a directory & run the command. This kicks off a reconstruction process on our render farm resulting in a 3 chiều mesh & camera positions ready for texturing.
“csStereoColourMatcher is also fully automated. Colour differences between stereoscopic image pairs can be caused, for example, by the beam splitter in the camera rig, or other factors that introduce significant colour shifts across different stereo views. Derived from vector-based analysis, we built it into the front kết thúc of all our compositing work for the film. It requires no user supervision & is completely integrated into the Nuke compositing system. We used it to colour balance more than 300 shots for the film.”
|Words: Adriene HurstImages: Courtesy of Disney Enterprises|
|MORE FEATURE ARTICLESFeatured in Digital truyền thông media World. Subscribe to lớn the print edition of the magazine and receive the full story with all the images delivered to lớn you.Only$79 per year.PDF version only FREEsubscribe|