About Graham Edwards

I'm senior staff writer at Cinefex magazine. I also write novels. In a former life, I produced animated films for theme park rides and science centres. If you offer me a cold beer, I won't say no.

Lucasfilm & ILM Launch “ILMxLAB”

ILMxLAB

Lucasfilm, Ltd. have just announced the formation of ILM Experience Lab. That’s ILMxLAB to you and me.

Drawing upon the talents of Lucasfilm, ILM and Skywalker Sound, this new division is on a mission to create immersive entertainment experiences at a fidelity never seen before. ILMxLAB’s ambitious plans include the development of virtual reality, augmented reality, real-time cinema, theme park entertainment and narrative-based experiences for a range of future platforms.

Watch the ILMxLAB launch video:

The official ILM press release includes comments from some of the key players, starting with Lucasfilm Executive Vice President and ILM President Lynwen Brennan, who stated:

“The combination of ILM, Skywalker Sound and Lucasfilm’s story group is unique and that creative collaboration will lead to captivating immersive experiences in the Star Wars universe and beyond. ILMxLAB brings together an incredible group of creatives and technologists together to push the boundaries and explore new ways to tell stories. We have a long history of collaborating with the most visionary filmmakers and storytellers and we look forward to continuing these partnerships in this exciting space.”

Vice President of New Media for Lucasfilm Rob Bredow added,

“The pioneering spirit that inspired storytellers and technical artists to improvise, innovate and help imagine a galaxy far, far away is in the DNA of ILMxLAB. We see xLAB as a laboratory for immersive entertainment. It’s amazing to be working in a new medium where we get to help invent how stories are told and experienced, connecting artists with their audiences like never before.”

Lucasfilm President Kathleen Kennedy stated:

“The people who work here have been investing in achieving the unachievable for more than 40 years. Creative storytelling was something that George Lucas instilled in each of the companies from their earliest days and out of that came the incredible innovation that continues to this day. We are currently exploring the fictional universes of Star Wars, and I think a lot of people would like to be immersed in them. The challenge of ILMxLAB will be to find out what storytelling looks like in this new space.”

Creative Director, ILMxLAB John Gaeta noted:

“Cinema is a master storyteller’s art form. Until recently, a “4th wall” has contained this form. Soon, however, we will break through this 4th wall and cinema will become a portal leading to new and immersive platforms for expression. ILMxLAB is a platform for this expansion. We want you to step inside our stories.”

ILMxLAB is currently collaborating on a number of projects, all of which are currently under wraps. But, given the steady build-up towards a certain cinematic event in December, it comes as no surprise to learn that the division has already promised to announce “exclusive Star Wars-based experiences later in the year”.

ILMxLAB

M is for Matte Painting

M is for Matte PaintingIn the VFX ABC, the letter “M” stands for “Matte Painting”.

Take any film aficionado’s top ten list of favourite movie tricks, and the chances are you’ll find the venerable art of matte painting near the top. But what actually is matte painting, and what makes it so special?

To put it in a nutshell, a matte painting is a piece of artwork used to fill in part of a scene that can’t otherwise be photographed. Take a cathedral interior, for example. Assuming you can’t find a real cathedral to shoot in, do you really want to shell out half your precious budget on constructing that mile-high vaulted ceiling? Wouldn’t you prefer to build your set up to a convenient height of, say, ten feet, then use a painting to patch in the rest?

Or, let’s say you want to photograph Count Dracula’s castle perched precipitously on top of a mountain. Are you prepared to ship a construction crew all the way out to the Bavarian Alps? Are you ready to face a mob of locals with torches and pitchforks protesting about how you’re defacing the landscape? Doesn’t it make more sense to photograph a suitably rugged portion of rocky terrain, then hire a skilled artist to paint in the vampire’s looming lair?

In short, isn’t the most straightforward solution to use a matte painting? Of course it is.

Unfortunately, matte painting isn’t quite as simple as that …

Top: a scene from "Dancing Pirate" (1936) showing original stage photography. Bottom: final shot composited with matte painting background. Chief technician on the film was Willis O'Brien. Frame enlargements first published in “Movie Makers”, November 1936.

Top: a scene from “Dancing Pirate” (1936) showing original stage photography. Bottom: final shot composited with matte painting background. Chief technician on the film was Willis O’Brien. Frame enlargements first published in “Movie Makers”, November 1936.

What’s a Matte

Matte painting has been around since the dawn of cinema. To understand its origins, we first have to understand the use of the word “matte”, which in visual effects terminology is really just another word for “mask”.

The original mattes were nothing more than pieces of black material, cut to shape and positioned in front of a camera in order to blank out part of a frame for later enhancement –the top half of a cathedral interior, for example. Thus the use of the term “matte painting” to describe the artwork created to fill in the blank.

As for combining the painting with the live-action, a common solution was to double-expose the artwork into the blank space left in the original footage, while masking the already-exposed portion of the frame with a counter-matte to protect it from further exposure. Another option was the “glass shot”, in which the vaulted ceiling of our notional cathedral would be painted in situ on a piece of glass positioned between the camera and the partial set.

Original photography for a shot in "The Last Days of Pompeii" (1935), showing the partial stage set before the addition of a foreground glass painting.

Original photography for a shot in “The Last Days of Pompeii” (1935), showing the partial stage set before the addition of a foreground glass painting.

Completed shot, with the background added via a glass painting positioned between camera and set. Chief technician on "The Last Days of Pompeii" was Willis O'Brien. Photographs originally published in “American Cinematographer”, January 1940.

Completed shot, with the background added via a glass painting positioned between camera and set. Chief technician on “The Last Days of Pompeii” was Willis O’Brien. Photographs originally published in “American Cinematographer”, January 1940.

Ah, but the artwork used in a glass shot isn’t strictly speaking a matte painting, because no masks are involved.

To split such hairs is to open a much wider debate on the subject of process photography – the craft of taking multiple elements and combining them into a seamless, composite shot. There are many ways this can be done. Indeed, in the early days of the movies, hardly a year went by without a legion of competing special effects technicians filing one patent after another, each trying to corner the process photography market.

Here’s just a small selection, as chronicled by Earl Theisen in the June 1934 edition of The International Photographer:

  • 1874 – C. M. Coolidge was granted patent No. 149,724 for a process of making composite prints by masking
  • 1912 – A. Engelsmann was granted patent No. 1,019,141 for a system of combining actors and artwork on a glass plate placed in front of a painted backdrop
  • 1917 – R. V. Stanbaugh was granted patent No. 1,226,135 for a process in which a traveling mask was threaded in the camera together with an unexposed film
  • 1918 – Norman Dawn was granted patent No. 1,269,061 for a process using photographs of the foreground as a cut-out mask, behind which a background was then added
  • 1918 – Frank D. Williams was granted patent No. 1,273,435 for a bi-pack matte process
  • 1923 – D. W. Griffith was granted patent No. 1,476,885 on a process using a painted screen with a hole cut in it, with actors performing behind
  • 1925 – Ralph Hammeras was granted patent No. 1,540,213 for a new glass shot technique
  • 1926 – Eugene Schufftan was granted patent No. 1,569,789 for a variant of the glass shot which involved photographing through a transmission mirror
  • 1927 – C. D. Dunning was granted patent No. 1,613,163 for a traveling matte process using colour separations

I’ll spare you the long version of the above list. It goes on for a very long time.

The matte photography room at Warner Bros. circa 1940. Photograph first published in “American Cinematographer”, January 1940.

The matte photography room at Warner Bros. circa 1940. Photograph first published in “American Cinematographer”, January 1940.

The Golden Era of Matte Painting

So how does a matte artist actually go about his business? Here’s an account from the July 1929 edition of American Cinematographer, written by Fred W. Sersen, who at the time was chief of the art department at William Fox Studios, and bearing the charming subtitle Some of the Intricacies of Making Things Seem What They Are Not Explained for the Amateur by an Expert with Years of Experience:

“It is best for the matte to be placed about thirty inches from the camera, even further if the glass or the material to be used for the matte is easy to obtain. When matting to the footage (especially when there is any wind blowing) or when dust is created by action in the scene, in a rain or snow storm, very soft blend is desirable and the matte should be placed four to six inches from the lens.”

Sersen goes on to explain various ways in which the painted part of the frame can be matched to the original photography. One of these involves projecting a test section of the original photography on to a coloured surface, and tracing the outlines of the scene:

“After the drawing is completed, it is laid in with oil paint in black and white, and on the artist’s ability and experiences depends the matching of the tones of the first exposure, which is ascertained by making the hand test and comparing the tones. He does this repeatedly by correcting the painting until the match is perfect.”

No sooner had artists perfected such techniques within the monochromatic world of early cinema, than along came colour. Now it wasn’t just a matter of matching light and shade, but also hue.

In the January 1940 edition of American Cinematographer, Byron Haskin (who went on to direct The War of the Worlds in 1953) made the following observations about the use of colour in matte paintings:

“It is obvious that the coloring of the actual set or landscape of the live-action portion of the shot must be precisely matched by the coloring of their respective continuations in the painting. This is by no means easy.

“It is entirely possible that the pigments used to paint a set may not photograph with the same Technicolor values as will visually identical pigments used in producing the matte painting. Therefore … the matte painter must not only know what colors were used in the set, but what paints were used to produce those colors. Where it is possible, he should have samples of the colors and paints used. The same, of course, is also true of fabrics and the like where they enter the matte painter’s problem.

“Equally important is the color of the lighting used in photographing the matte painting. Most Technicolor interiors are lit with special arc equipment which gives a light very closely matched to natural daylight.”

Because of these and other technical constraints, matte painters were forced to develop skills quite different to those of the average artist. They had to mix their colours and make their brushstrokes while all the time bearing in mind the myriad quirks and vagaries of the photochemical filmmaking process.

Peter Ellenshaw produced between 30–40 matte paintings for Richard Fleischer's fantasy adventure "20,000 Leagues Under the Sea".

Peter Ellenshaw produced between 30–40 matte paintings for Richard Fleischer’s fantasy adventure “20,000 Leagues Under the Sea”.

Once the colours were matched and the composition approved, the matte painting was finally ready to be combined with the original negative. Alignment of the two was critical, so as to avoid unwanted black lines at the point where the two images came together. Feathered edges were frequently used to soften the joins. Equally critical was the stability of both camera and projector, as the slightest movement would cause the painted part of the frame to judder against the original photography.

By now it should be evident that these pioneers of matte painting faced some stiff challenges. As Earl Thiesen remarked in the November 1936 edition of Movie Makers:

“Glass paintings are not simple, since the picture must be photographic in technique. The details, tone values and harmony of all parts of the painting must resemble a photograph and must match the perspective and photographic values of the setting. Very few artists have the ability necessary to do a glass painting.”

If you want to dig a little deeper into the history of the craft, check out Matte Shot, Peter Cook’s expansive repository of screenshots and commentary exploring the golden era of matte painting.

Mike Pangrazio of ILM works on a matte painting for "Return of the Jedi". Live-action stage photography was rear-projected into the areas of glass left blank.

Mike Pangrazio of ILM works on a matte painting for “Return of the Jedi”. Live-action stage photography was rear-projected into the area of glass left blank.

The Matte Painting Evolves

Throughout the 20th century, matte painting continued to thrive, enhancing the look of Biblical epics, Westerns, thrillers and period dramas alike. Got a scene in a lavish ballroom? Use the cathedral trick and just build the bottom half of the room – those matte boys just love painting chandeliers. Need to see a Spanish galleon anchored just off-shore? Forget shipbuilders – call the art department instead. And even if you’re forced to shoot on an overcast day, there’s no need to fret. It’s the work of a moment to paint a dramatic sky filled with ominous clouds.

As they developed their craft, matte painters were constantly trying out new techniques to stop their paintings looking like, well, paintings. They used layered artwork to create a greater illusion of depth. They devised cunning animated gags to simulate movement in the waves of a painted sea, or used backlighting to create convincing flares around a setting sun.

During the 1980s – that ambitious age when traditional photochemical effects techniques were being stretched to the limit – matte painters continued to push the envelope. For many of the spectacular wide shots in Return of the Jedi, for example, multiple sections of live-action were rear-projected into gaps left deliberately empty in gigantic glass paintings.

Top: A matte painting of Pankot Palace by Mike Pangrazio and Christopher Evans for a scene in "Indiana Jones and the Temple of Doom". The painting was ultimately rejected by the artists. Bottom: The final shot was created using photography of a cut-out silhouette positioned on a hilltop, enhanced by matte painted highlights and additional architectural details.

Top: A matte painting of Pankot Palace by Mike Pangrazio and Christopher Evans for a scene in “Indiana Jones and the Temple of Doom”. The painting was ultimately rejected by the artists. Bottom: The final shot was created using photography of a cut-out silhouette positioned on a hilltop, enhanced by matte painted highlights and additional architectural details.

At the same time, artists were also exploring more deeply the idea of combining “straight” matte paintings with other visual effects techniques. Disappointed by the look of their original matte painting of Pankot Palace for Indiana Jones and the Temple of Doom, ILM’s Mike Pangrazio and Christopher Evans prepared a cut-out silhouette of the building, which they photographed on a convenient hilltop at sunset. Matte painting techniques were then used to enhance the resulting image with highlights and other architectural details.

By the time ILM was working on the 1988 fantasy Willow, the line between matte painting and miniature had blurred still further. Speaking about Willow in Cinefex 35, matte department supervisor Christopher Evans remarked:

“There are real advantages to using miniatures with matte paintings. Miniatures have incredible perspective, but matte painting can create a sense of atmosphere and distance better than a model. Combining the two techniques is like making an alloy in metallurgy – the combination of the two ingredients is stronger than either by itself. The miniature is shot latent image and then we add the paint to it – so it is actually a blending and intermixing of the two.”

Yet, in this same decade, digital techniques were also on the rise. In this brave new world of photorealistic visual effects, would there still be a place for the traditional matte painting?

A Bigger Canvas

Astonishingly, despite all the changes that have swept through the effects business over the decades, matte painting is still a recognised – and thriving – discipline. Nowadays, of course, it’s called “digital matte painting” – or DMP – and on the face of it bears little resemblance to what was practised by those early paintbrush-wielding pathfinders.

In an era when the movie camera can (and frequently does) go anywhere, the new watchword for all visual effects artists is flexibility. It’s no longer good enough to lock down the camera and let the action play out across a static canvas – even a digital one.

What’s more, it’s now not unusual for a visual effects shot to be tweaked ten, twenty, even hundreds of times, with notes and comments flying endlessly back and forth between artist, supervisor and director. Faced with the need to generate multiple iterations of a shot, what artist would be crazy enough to use a paintbrush?

That’s why matte painters have had to enter not only the digital realm, but also the third dimension.

This Rodeo FX Game of Thrones reel contains many great examples of digital matte paintings and environment extensions:

A modern digital matte painting is created not by daubing paint on to a piece of glass, but by mapping artwork on to CG geometry. In a typical DMP environment, the models used might be relatively crude – perhaps just a series of flat planes or geometry layered up in virtual space. However, the devil is in the detail, and the detail is in the textures.

Textures for a DMP might be collages assembled from reference photography, perhaps taken on location or on the set, or artwork created from scratch. The digital matte artist refines these with retouching and hand-painting techniques, using software such as Adobe Photoshop or The Foundry’s MARI to create files that can subsequently be wrapped around the necessary geometry.

The result? A rich, dimensional environment that can be viewed from a number of key angles or, if necessary, from all sides.

Yet, even in this integrated digital world, the artistry of the matte painter still holds sway. And on the rare occasion a matte painter actually finds himself in the director’s chair – as Robert Stromberg did recently when he helmed Disney’s Maleficent – the possibilities are endless.

Talking about his directorial debut in Cinefex 138, Stromberg revealed:

“I did about 100 matte paintings [for Maleficent] … I’ve been lucky enough to make people feel a certain way through background art or matte painting. This was an opportunity to explore other ways to create emotion. As a movie director, it’s a bigger canvas.”

A composite shot from "Maleficent" integrating a physical clifftop set with a digital environment by MPC. The latter included matte paintings, terrain projections and a digital model of the distant castle.

A composite shot from “Maleficent” integrating a physical clifftop set with a digital environment by MPC. The latter included matte paintings, terrain projections and a digital model of the distant castle.

Matte Painting in a Nutshell

As we’ve learned, in the context of visual effects, “matte” means “mask”.

According to the Oxford English Dictionary, “paint” is “a coloured substance which is spread over a surface and dries to leave a thin decorative or protective coating”.

Taking those two definitions at face value, traditionalists might argue that modern DMP isn’t matte painting at all. I can see their point. Since it began, the craft has changed out of all recognition, hasn’t it?

Yes, and no.

Take another look at that list of patents from the first few decades of the 20th century. What do they prove if not that change has been with us since the very beginning? Each generation strives to better the last, and will eventually develop the tools to do so. Either every age is a golden age, or none of them is.

But it’s really only the tools that change, isn’t it? Everything else stays the same: the intent of the artists; the commitment they show to their task; the skill with which they wield their tools.

That’s why I don’t believe matte painting has changed at all. The skill-sets of the people doing the work may have altered, but their purpose remains what it has always been.

What is that purpose? To extend reality beyond its natural borders. To create the solid gold setting in which the jewel of performance can shine. To enhance what exists with what we can only imagine, and in doing so makes the mundane beautiful.

Now that’s matte painting in a nutshell. What do you know? It turned out to be simple after all.

Related articles:

“Return of the Jedi” photograph copyright © 1983 by Lucasfilm Ltd. “Indiana Jones and the Temple of Doom” photographs copyright © 1984 by Lucasfilm Ltd. “Maleficent” photograph copyright © 2014 by Walt Disney Pictures.

“Avengers: Age of Ultron” – Stereo Q&A

Marvel's Avengers: Age Of Ultron  L to R: Hulk (Mark Ruffalo), Captain America (Chris Evans), Iron Man (Robert Downey Jr.), Hawkeye (Jeremy Renner), Black Widow (Scarlett Johansson), and Thor (Chris Hemsworth)  Ph: Film Frame  ©Marvel 2015

When booking your tickets for the latest summer blockbuster, you’ll probably be faced with a choice: 2D or 3D? With Avengers: Age of Ultron, the question is boiled down to its most visceral form. Is it enough just to see Hulk smash? Or do I want to see Hulk smash in stereo!

According to the latest MPAA report, Theatrical Marketing Statistics, nine out of the top ten box office hits in US/Canada in 2014 boasted a 3D theatrical release. The same year saw the global proportion of 3D digital screens increase to 51% (70% in the Asia Pacific region). Pundits continue to debate the pros and cons of 3D, but as long as the major studios continue to pump out big stereo movies, the desire will remain to make the 3D experience as punchy as possible.

Satisfying this desire on Avengers: Age of Ultron were two stereo conversion facilities: Prime Focus World and Stereo-D. In total, Prime Focus World converted 830 shots for the movie, with production running for three months, and the number of team members peaking at 613 across their London, Mumbai and Vancouver offices.

Cinefex spoke to Richard Baker, senior stereo supervisor at Prime Focus World, about the state of the art in stereo conversion, and about the company’s work on Avengers: Age of Ultron.

Marvel's Avengers: Age of Ultron - Hulk

So how about Hulk? He’s a big guy – does that make him a natural subject for 3D?

The issue of Hulk’s size was an interesting one. We obviously wanted to use the stereo to emphasise his scale, and the natural tendency would have been to pump him up in the stereo conversion. But this actually has the opposite effect, and tends to minimise scale. To increase the feeling that Hulk is much bigger than the other characters, we actually flattened him off a little, slightly reducing his internal depth and ensuring that he was never too separated from the background.

Okay, before we get too deep into Avengers: Age of Ultron, let’s fill in the background. First, is the demand for stereo as big now as it was shortly after Avatar brought it back to the mainstream?

There still is a big demand, although the market has now consolidated, with four main service providers in the conversion industry. Marvel is a great example of a studio committed to 3D: all of their films are planned to release in 3D, and they have recently added more films to their slate. Disney, Warner Bros., Fox, Lionsgate … most of the studios have 3D releases in their line-ups.

Have the techniques changed much since those early days?

In the early days of stereo conversion, it was assumed that linear stereo was always preferable – that it was more natural. In fact the industry learned that linear was not always acceptable – that sometimes it just didn’t look right. Also, filmmakers wanted full control over depth, in order to use the stereo as another storytelling device, alongside the grade, edit and sound mix.

This is where stereo conversion comes in. We have developed techniques to incorporate linear and non-linear elements into the same scene, including the generation of virtual stereo camera rigs from our converted scenes and delivering them to VFX to allow them to render CG elements that will slot straight into the conversion.

Editor’s note: The terms ”linear” and “non-linear” refer to the way the stereo offset – in other words, the amount of “3D-ness” – reduces from the foreground to the background of an image. With linear stereo, the offset reduces at a constant rate. With non-linear stereo, the rate of decrease follows a curve, to more comfortably emulate what the human eye naturally perceives.

You use a number of proprietary tools that you’ve developed in-house. What are the advantages of doing this?

When we create tools, it is generally to cater to a specific requirement that off-the-shelf tools can’t handle, or don’t handle well. One of the benefits of maintaining an R&D department and developing tools in-house is that you can create them to be exactly what you want them to be. And sometimes we need do things that no-one else is doing, or has thought to do. We specify, build and rigorously test all our proprietary tools, then roll them out across our global network.

The conversion process itself is constantly evolving also. My background is in VFX, so it always seemed sensible to me to use geometry in the conversion process – for example, with characters’ heads, and with certain environments where it is important to maintain consistency and accuracy of depth across shots. We use cyberscans of the main actors to produce head geometry for all our shows, and we also request lidar for environments that we know would benefit from geometry.

Watch a video about DepthGen, one of Prime Focus World’s proprietory stereo conversion tools:

What kind of background or training do you need to become a stereo artist?

As with VFX, the best stereo supervisors and artists understand and embrace both the creative and the technical aspects of what they are doing. Conversion is very much a VFX process, and we have a range of levels of compositing artists in our stereo teams – seniors, mids and juniors. Being able to really see the subtleties in a stereo shot takes months of training, of looking at stereo, and developing an eye for what works and what doesn’t.

Stereo conversion can provide a great grounding for junior artists who may decide to move into VFX later, and with most of the big blockbuster movies being released in 3D, this training can prove invaluable, not just in learning to recognise good stereo but also in learning about pipelines, colour space, working in Nuke or Fusion and a whole host of other areas.

Watch a video about another of Prime Focus World’s stereo conversion tools – PFLive:

Let’s get back to Avengers: Age of Ultron. How did Prime Focus World get involved in the show?

We were awarded the work on Avengers: Age of Ultron on the back of our stereo conversion work for Marvel on Guardians of the Galaxy. That was a great project to be involved with creatively, and it went on to win an award for “Best 3D Live Action Feature” from the Advanced Imaging Society.

At what stage in the production did you actually start to work on the show?

I was invited out to view an early cut of Avengers: Age of Ultron at the end of 2014. Even though it was all still previs and greenscreen at that point, the edit was already in great shape. I sat with Evan Jacobs, Marvel’s stereographer, and Mike May, Marvel’s stereo producer, to talk through the creative aspects of the conversion and discuss production details. This allowed me to put together my depth summary for the show – an internal guide for our international teams that describes how we intend to approach the stereo, how it will play out across the sequences we are working on, and where we can use the depth to create “3D moments”.

Marvel's Avengers: Age of Ultron - Scarlet Witch

Was any of the film shot with a stereo camera rig?

No, the whole film was stereo converted, although there were some stereo renders – for example, the Iron Man HUD shots. Robert Downey Jr. was shot mono on greenscreen for these sequences, and this footage was subsequently stereo converted. The converted shots were then delivered to Cantina, who comped the stereo HUD elements into the shot. Stereo renders of graphics content and transparencies always look better.

Does it make for more work, doing the stereo conversion in post?

To shoot Avengers: Age of Ultron native would have been nigh-on impossible. There was so much greenscreen content that making depth choices for the live-action characters would have been tantamount to setting the stereo blind, because the backgrounds hadn’t even been created yet. Also, for the VFX houses to work in and render stereo is a big overhead. Adding this to a tight production schedule and hugely complicated creative work would have been an extra hit they didn’t need.

Do you get a superior end result this way?

Ultimately, being able to control the stereo in post is the best way to go. The quality of the stereo result is no longer in question. In fact, it is arguably superior to native, unless native has been through its own post process treatment. Perhaps most importantly, it gives studios and directors the flexibility to make changes to the depth as they see the film come together. With native shooting, you are locked in to the decisions you made on set, so you’d better hope that they were right!

Tell us more about how the stereo “look” of the film was planned.

Stereo decisions are generally made by the show stereo supervisor – Evan Jacobs in this case – in conjunction with the director. For Avengers: Age of Ultron, Evan and I discussed the stereo up-front, making a lot of references to the style and direction that we had set up for Guardians of the Galaxy. Once the brief was clear, we started working up the depth in individual shots. Specific comments or decisions would come up later during client review. As the stereo supervisor across the entire show, Evan was seeing the big picture, and thinking about how the depth was coming together for the whole movie, rather than just for any one sequence.

How closely do you liaise with the VFX vendors during the stereo conversion process?

Our production team has a great rapport with the Marvel production team, which is invaluable on a big, complicated show like this. We also have great relationships with the VFX vendors, which is crucial when we need to harvest VFX elements early. ILM couldn’t have been more helpful. Their incredible one minute-long opening shot for Avengers: Age of Ultron was one of the last renders to come through to us, and they were there 24 hours a day to assist us with breaking it out for conversion.

Marvel's Avengers: Age of Ultron - Church battle

Describe your workflow for stereo converting the film’s big VFX sequences.

The huge battle in the abandoned church towards the end of the movie is a good example of a massive, elements-based VFX sequence. We started working with ILM early in the conversion process to get element passes so that we could start to set up the scene in stereo, even though we knew the animation was likely to change. This benefited Marvel, as it gave them an early temp stereo version of the shot so they could see how it was going to play in the cut, and for DI. As newer versions of the shots came in, we incorporated the changes using our internal tools. This kind of collaboration is crucial to delivering shots like this, with hundreds of VFX layers, within the allocated production timeline.

In terms of the elements pipeline, the VFX work was made available to us in comp scripts containing all the constituent elements and layers that we needed for the conversion. We used our proprietary tool, AssistedBreakout, to render out smaller “minicomp” versions of the huge VFX files, giving us a more manageable script. Without this, the breakout process would have been manual, and taken skilled artists much longer to perform.

Watch a video explaing how Prime Focus World’s AssistedBreakout operates:

The depth generation was handled through our View-D conversion process, using depth mattes, Z-depths (where available), geometry and hand-sculpting to create a depth map which gave us the stereo offset. I reviewed every shot first here in London and delivered any internal notes. Once I was happy, the shot went forward to the client for approval, and any client notes were reviewed over TVIPs, our live stereo review system, with Evan Jacobs in LA and me in London. Once the shots were finaled, they were delivered as 2K log DPX files.

Is it easier to create effective stereo for an element-based VFX shot, rather than live-action straight out of the camera?

Let’s say you had two similar shots: one with digital characters and lots of CG dust, smoke and explosions, and the other shot live-action with practical effects. The benefits of having access to the VFX elements would be most apparent in the cleanliness of the image. The live-action shot would require lots of paint work to fill the occluded areas – which can absolutely be done, and which we have done many, many times – but with VFX elements available we would be able to approach the shot in a different way. And it would be quicker.

On VFX-heavy shows – the majority of the shows that we work on – having the VFX elements allows us to extract Z-depths for the CG characters, and to separate the various layers of smoke, dust, particles and explosions. This reduces the amount of paint required and allows us to work at a higher quality with even more detail and accuracy.

Which shot in Avengers: Age of Ultron gave you the biggest challenge?

I guess if I had to choose the most challenging shot it would be the opening sequence. It was over a minute long. It was a continuous shot. It was one of the last to deliver. ILM had split it into three parts, and we had to split it into eight parts to make it more manageable in the time-frame. It was challenging from an editorial point of view, from a production point of view and artistically too. But we’re used to working with long shots after our work on Gravity!

Which shot do you think gives the most stereo bang for buck?

I think probably the two slow-motion shots that we worked on – the iconic tableau of the Avengers flying across the screen in the opening forest sequence, and the big rotating battle shot in the abandoned church. These are both perfect 3D moments, giving the viewer time to really take a look at the stereo. In the abandoned church shot, you have characters flying in and out of the action, towards the screen, plus there’s a really nice camera move, lots of atmospherics and dust and lasers and lightning … it’s all going on!

Marvel's Avengers: Age Of Ultron  L to R: Hulk (Mark Ruffalo), Captain America (Chris Evans), Thor (Chris Hemsworth), Iron Man (Robert Downey Jr.), Black Widow (Scarlett Johansson), and Hawkeye (Jeremy Renner)  Ph: Film Frame  ©Marvel 2015

You talked about big VFX shots coming in very late in the day. Since stereo conversion is one of the last jobs to be done, how do you cope with the time pressures?

In terms of the VFX, yes, we’re one of the last to touch the film, which means there is huge pressure on the stereo team. By the time we receive turnover, many departments have overrun – through no fault of their own – either because of reshoots, or edit changes, or other delays. It’s an inevitable aspect of the process. We know that the biggest VFX shots are the ones that will deliver last because they are the most complicated. It’s just the way it is, and we anticipate this in our schedules. We know that the last month of a big project like Avengers: Age of Ultron is going to be super-intense!

And yet, working with Marvel, this push to the finish line is also enjoyable. We have weekly calls with the entire post team, and everyone is working towards the common goal of delivering the best quality work and hitting their delivery dates. This openness and transparency is so important. One of Prime Focus World’s great strengths is that we can mobilise our global resources very quickly to ensure that we can handle any situation.

"Marvel's "Ant-Man"

Prime Focus World is continuing its relationship with Marvel on Ant-Man. Does the film’s microscopic hero make it a good subject for stereo conversion?

Ant-Man is going to be a whole new challenge, with its own stereo style driven by the scale considerations of the main character. We’re talking to Marvel a lot about how we develop the stereo language for this show. How does our hero look when he is transformed into his tiny form? Does he feel small in a big environment? Or do you imagine the camera has shrunk with him so that he feels normal-sized? This movie is perfect for 3D, because the stereo can play such a huge part in creating a feel for the show, and we can use it to support the storyline. We’re really looking forward to this!

Finally, are there any new advances in stereo on the horizon?

Deep compositing is really exciting for us. Not all VFX houses are working with deep yet, but companies such as ILM, Double Negative and Weta Digital are. Receiving deep comps from VFX allows us to use the depth data to create the stereo offset, because the occluded areas are present within the deep information. Once we create the left eye/right eye offset, the “missing” information is automatically filled in. The development of our DeepGen tool to take full advantage of deep comps in the conversion process is probably one of the biggest areas of advancement on the tech side for us at the moment.

Also our merger with Double Negative last year was a big step forward for the company, and we’re already seeing the benefits of this relationship in our collaboration on Avengers: Age of Ultron, and on forthcoming projects such as Terminator: Genisys and Ant-Man. The close working relationship of our production teams, the ability to access VFX elements earlier in the process, the time and cost saving benefits to our clients – it’s a big step forward in terms of our efficiency.


Read the full story behind the visual effects of Avengers: Age of Ultron in Cinefex 142. For this latest Marvel spectacular, VFX supervisor Christopher Townsend oversaw a visual effects team that included ILM, Double Negative, Animal Logic, Luma Pictures and Framestore, with special effects supervisor Paul Corbould and the mechanical wizards at Legacy Effects lending practical effects support. Buy your copy of Cinefex 142 now!

Special thanks to Tony Bradley. “Avengers: Age of Ultron” and “Ant-Man” photographs copyright © 2015 by Marvel Entertainment.

Inspiring the Monster Makers

What drives people to work in the makeup and creature effects industry? The glamour? The technology? All those ravening monsters and exploding spaceships? Or is it just another job? In an ongoing series of articles, we ask a wide range of effects professionals the simple question: “Who or what inspired you to get into the effects business?”

Here are the responses from professionals working in the field of makeup and creature effects. Yes, it’s time to find out what inspires the people who make the monsters.

Frankenstein's Monster photograph by Universal Studios (Dr. Macro) [Public domain], via Wikimedia Commons

Boris Karloff as the Monster in “Bride of Frankenstein”.

A Universal Inspiration

It’s a given that, to be a successful monster maker, you have to love monsters. So it’s not surprising to learn that many of the makeup and creature effects experts working in the business today were inspired by all those classic movie monsters of old.

And what could be more classic than the iconic creatures that lumbered out of Universal Studios during the era of black-and-white?

“I used to watch the late-night double bill of Universal horror movies like Frankenstein and The Wolfman,” recalled Mark Coulier. “My dad would put me to bed at 8pm and then wake me up again later to watch with him – I was probably about ten years old, and loved it.”

Tom Woodruff Jr. also remembers relishing the antics of Dracula and his bloodthirsty buddies on the small screen, citing as one of his early inspirations “the monster movie craze of the mid ‘60s that put Universal Monsters on TV.”

Another lifelong fan is John Rosengrant: “The Universal Monsters had a huge impact on me when I was a kid, and sparked my interest in practical effects.”

Creature from the Black Lagoon photograph by Florida Memory, via Wikimedia Commons

The Creature rises out of the Black Lagoon.

For Howard Berger, there’s one Universal monster that stands head and shoulders above the rest. “What inspired me was seeing The Creature From the Black Lagoon when I was probably four years old,” Berger enthused. “The movie itself was amazing to me, but when I saw the Creature, I was blown away! My little brain could not process what I was looking at. It was truly a real monster! I still love this film, and the Creature suit is one of the best. The design, execution and performance still holds up. I love the Creature!”

As far as Mike McCarty is concerned, there’s only one person to blame for his childhood fascination with monsters. “It was all my mom’s fault. She loved classic horror, but wouldn’t watch it alone. When I was a kid, I was fascinated by monster movies, all the old classics: The Fly, The Blob, The Wolfman.”

One of the most consistently inspiring figures for many effects artists is filmmaker and animator Ray Harryhausen.

One of the most consistently inspiring figures for many effects artists is filmmaker and animator Ray Harryhausen.

Hats off to Harryhausen

If there’s one effects artist who has inspired more people in the business than any other, it’s probably Ray Harryhausen. The fantasy epics he made through the second half of the 20th century –heavily populated with fantasy landscapes and stop-motion creatures – have motivated animators, VFX artists and practical effects professionals alike.

“At about the age of five, my father awakened me to watch a scene from the network TV premiere of Jason and the Argonauts,” Alec Gillis remarked. “At 9:30 pm it was late, and my little brain squirmed and then permanently twisted as I watched a man in a mini-skirt toss monster teeth on the ground. Who knew skeletons grow from monster teeth?! Had it not been for my dad and Ray Harryhausen, I might not have gotten hooked on effects.”

Sean Sansom was also seduced by stop-motion. When asked what had inspired him to get into the effects business, he replied, “Definitely Ray Harryhausen! Seeing the Cyclops from The 7th Voyage of Sinbad on TV as a little kid made me believe that monsters were alive and well, roaming the land.”

Ray Harryhausen Cyclops

For Phil Tippett, his first experience of a Harryhausen movie not only encouraged him to pursue animation as a career, but also to seek out the very man who had inspired him. “When I was seven, my parents took me to see The 7th Voyage of Sinbad, and that was the moment for me,” Tippett commented. “It was Ray Harryhausen who inspired me to become a stop-motion animator, and he was good enough and approachable enough to answer my calls all those years ago and become a mentor to me. We had a close relationship right up until his death in 2013. He was a truly wonderful guy – artist, friend, everything. Very inspiring.”

In his turn, Tippett has passed the baton down the line to people like David Duke, who said, “It was a combination of two elements for me. The first was seeing Phil Tippett’s stop-motion tests for Jurassic Park.” Duke then added, “The other was seeing all of the behind the scenes footage from Stan Winston Studio in creating the animatronic dinosaurs. Suddenly I became aware of this whole world of practical effects. Before, I had known that what I was seeing in films was an illusion, but it wasn’t until this moment that exactly how the illusion was created came into sharp focus.”

Too Many Movies to Mention

Ask anyone in the movie industry to list their favourite films, and you might as well pull up a chair and settle in for the night. Monster makers are no exception.

Steve Newburn reeled out a list including “Forbidden Planet, The Time Machine … all those great ‘50s and ‘60s genre films that got me wanting to do nothing more than figure out how they created those images and hopefully work on them myself some day. Throw in some Star Wars, Planet of the Apes, and Universal Monsters …”

Mark Coulier remarked, “As a teenager, I had so many influences: Jason and the Argonauts, The Elephant Man, The Godfather. I think the moment that I really decided was when I saw a trailer for The Howling, where they had this man turn into a werewolf before your eyes. I’d never seen anything like that before. It had always been done with lap dissolves, which even to a ten year old are pretty obvious. Now it was like, ‘Wow, how the hell did they do that? I want to make rubber monsters for a job. That seems like a good way to spend your working life!’ I never looked back.”

John Rosengrant noted, “Alien made me decide that this was going to be my career,” while Shaun Smith agreed that favourite films are indeed a major motivating factor: “Movies played a big factor, particularly John Carpenter’s The Thing – that blew me away.”

For this full-body transformation shot from "An American Werewolf in London", actor David Naughton was positioned beneath the set with only his head and arms protruding up through a hole in the floor. The rest of his body was fabricated by Rick Baker's EFX crew and articulated from below via concealed rods.

For this full-body transformation shot from “An American Werewolf in London”, actor David Naughton was positioned beneath the set with only his head and arms protruding up through a hole in the floor. The rest of his body was fabricated by Rick Baker’s EFX crew and articulated from below via concealed rods.

A shuddering Sean Sansom added to the list by commenting, “After being traumatised at my friend’s eighth birthday party by the recently released An American Werewolf in London, I was never the same!”

Rob Gillies completed the picture, remarking, “Growing up in the ‘80s on films like Star Wars, Robocop, E.T.: The Extra-Terrestrial and Labyrinth was hugely inspiring to me wanting to make practical effects. Getting into the business of effects – having the opportunity to be a part of making a plasma rifle, or a ten-foot-tall assault robot for a film – was a dream come true.”

Famous Monsters of FilmlandLearning from the Experts

Seeing inspirational images on the silver screen is all very well, but how do you learn about how they were made? For the inquisitive monster maker, there have always been books, magazines and videos ready to reveal the secrets of the masters.

Tom Woodruff Jr. identified one revered source: ”Famous Monsters of Filmland magazine, which showed behind-the-scenes makeup and creature pictures.”

And there was no better place than Famous Monsters of Filmland in which to find adverts for another of Woodruff Jr.’s youthful inspirations: “Aurora monster models, which put monsters in my hands.”

Aurora Models advertisement from the 1965 Famous Monsters of Filmland Yearbook.

Aurora Models advertisement from the 1965 Famous Monsters of Filmland Yearbook.

"Three-Dimensional Makup" by Lee BayganFor Mark Coulier, it was a chance discovery at a market stall that helped set him on his path: “Big thanks to Lee Baygan and his book on prosthetics, Techniques of Three-Dimensional Makeup, that I found on Cambridge market. That book showed me how to get started.”

Seeking similar kinds of source material was Mike McCarty. “Magazines like Starlog and Fangoria introduced me to a whole new group of names like John Chambers (Planet of the Apes), Tom Burman (Food of the Gods, Invasion of the Body Snatchers) and Rob Bottin (The Thing). I stumbled across two books: Making a Monster by Al Taylor and Sue Roy and Tom Savini’s Grande Illusions. These books gave me step-by-step instructions on how to ruin carpets and wreck my mom’s mixing bowls!”

A young Will Furneaux of Weta Workshop performs one of his first makeup tests in 1988.

A young Will Furneaux of Weta Workshop performs one of his first makeup tests in 1988.

Will Furneaux also found inspiration in the work of a legendary monster maker. “When I was about twelve, I got a video out from the local video rental store called Scream Greats, Vol. 1: Tom Savini, Master of Horror Effects,” Furneaux recalled. “It introduced me to Tom Savini, makeup effects and Fangoria magazine, and got me hooked on horror makeup effects. I got some latex and clay and started making my own effects.”

Gruesome Games

There’s a certain playful quality to the whole business of makeup and prosthetics, as noted by Barney Burman: “I believe that what really inspired me – and still does – is my continuous need to play ‘dress up’. I love being part of a team that comes together to build an alternate world, put characters in it, and watch it all come to life. That’s the fun in making movies! And that’s where I feel at home.”

Hide-and-seek fantasies filled with secret disguises inspired Mark Coulier, who commented, “I read some Enid Blyton books where this kid solved mysteries by going round in disguise, making fake noses and moustaches and such. I loved that idea as a kid – that you could alter your face like that.”

Game-playing in its most literal form had a huge influence on Shaun Smith. “Like most young boys, I was fascinated by robots, aliens, ninjas and monsters,” Smith explained. “My path was really set in middle school, when I started playing the Gary Gygax and Dave Arneson Dungeons & Dragons game with my pals. Little did I know it would be a catalyst that would set me on the path to a dream job: breathing life into these fantastical images. I was often sidetracked from my scholastic obligations, instead spending time painting lead figures and studying game manuals.”

In the Blood

From what we’ve heard so far, it’s clear that there are many parents keen to share their love of monsters with their kids … however young they may be. The role of family in encouraging a monster-making career is just further proof of the importance of blood ties.

“My father, who worked for JPL/NASA, was my most direct influence,” stated Steve Newburn. “A huge fan of sci-fi, he exposed me to effects-based films from as early as I can recall.”

Barney Burman, as the son of award-winning special makeup effects artist Thomas R. Burman, and the grandson of makeup pioneer Ellis Burman Sr., puts family influences front and centre. “My father hired me and taught me some,” he remembered. “My mother added urgency when she told me to go out and make my own way and pay my own bills (for which I’ll always be grateful). My brother allowed me to set my inner artist free. And my son, simply by being born, helped to solidify my need for serious commitment to making a career of it.”

Tom Savini's Scream GreatsA Monstrous Meeting

Few things can compete with the thrill of discovering the true north of your own personal compass. But what happens when you finally get to meet one of the idols who inspired you to follow your chosen path? Will Furneaux found out, when the makeup effects hero of his youth visited his workplace.

“One day, Richard Taylor was giving Tom Savini a tour of Weta Workshop, and he brought him into the 3D department,” Furneaux related. “I was nervous. I stood up, shook his hand, and not being the most articulate person in the world said, ‘Hi! I was a big fan of yours!’

“Tom said, ‘Oh … what happened?’ Before I could explain, Richard hurried him off to see the milling machines, which are a lot more interesting to look at than nerds working on computers! I was so embarrassed! Of course what I’d meant to say was that when I was young I was a massive fan, but had since moved into a different area. Tom had a huge influence on me when I was young, and he’s part of the reason why I’m now at Weta Workshop.”

Read more articles in our Inspiring FX series:

Special thanks to Ri Streeter and Niketa Roman.

The Foundry Sold in New Growth Commitment

The Foundry - Brad Peebler, Bill Collis, Nic Humphries and Simon Robinson

Brad Peebler, Bill Collis, Nic Humphries and Simon Robinson

Explore any business, and you’ll quickly find there are essential pieces of industry-standard software that keep its wheels turning. In the visual effects industry, one of those is the compositing, editorial and finishing tool Nuke.

Nuke is so widely-used that recent rumours that its creators – UK-based The Foundry, which also produces VFX tools Mari and Katana – might be taken over by software giant Adobe, sent ripples of concern through the ranks of VFX artists around the world.

Today, May 20, 2015, those rumours were resolved when The Foundry announced a majority investment from private equity firm HgCapital, who in a deal worth £200 million ($312 million USD) will now take over ownership of the company from The Carlyle Group.

Commenting on the company’s new place within the Technology, Media & Telecommunications (‘TMT’) sector of HgCapital, The Foundry’s CEO Bill Collis said:

“Knowing the direction we plan to take The Foundry, we identified that HgCapital was the ideal partner to build on what The Carlyle Group have helped us achieve. Nic and his team have such deep software experience, take a long term view on investing and have an amazing track record in taking already solid companies to even greater levels of success. HgCapital achieve this through investment, both in R&D and people, with a deep respect for customer loyalty and satisfaction. With this deal, we remain one of the few independent companies solely focused on creative industries. This lets us pursue our best-in-class strategy, prioritizing research and innovation; and teaming with other companies to create powerful collective solutions.”

Nic Humphries, managing partner of HgCapital and the head of the TMT team, added:

“There are so many elements about The Foundry that we find attractive, not the least of which is the core management team. This is a company that constantly innovates, both in terms of their technology, as well as their business. Bill and his team love nothing more than running head on into the challenges facing creative industries, developing exciting disruptive technologies that have huge potential.”

In a tight-lipped but optimistic press release, The Foundry asserts that the HgCapital team shares its “vision to deliver disruptive technologies to creative industries”:

“This will leverage our world-class experience in visual effects, design and games. It continues our leadership in emerging trends around collaborative ideation, the automation of creativity, concurrent marketing and manufacturing, as well as new behaviours in storytelling, media collaboration, creation and consumption, such as VR and AR.”

Dream Landscapes – Outer Space

“The kinds of landscape I try to find in my films exist only in our dreams” – Werner Herzog

Space, as every good Trekkie knows, is the final frontier. It’s also the place where no one can hear you scream. Arthur C. Clarke, author of 2001: A Space Odyssey, claimed that space “can never be conquered”. And, in The Hitch Hiker’s Guide to the Galaxy, writer Douglas Adams gleefully informed us that:

“Space is big. Really big. You just won’t believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space.”

As for taking a trip through that great celestial void, well, for most mortals the promise of interstellar travel is just a dream – despite the recent surge in commercial space tourism projects like Virgin Galactic and Space Adventures. I mean, have you seen their ticket prices?

If you really do want to explore other worlds, however, the answer is really very simple: go to the movies. Not only are the tickets cheaper, but also you get popcorn.

But just how do filmmakers go about putting the wonders of the cosmos, big and small, on to the silver screen?

The sea of stars visible in "Destination Moon" comprised some 2,000 automobile headlight bulbs strung on 70,000 feet of wire.

The sea of stars visible in “Destination Moon” comprised some 2,000 automobile headlight bulbs strung on 70,000 feet of wire.

Seeing Stars

Look up into the sky on a clear night and you’ll see a vast starfield comprised of a gazillion tiny pinpricks of light. You might think it’s easy to re-create that view by using, well, tiny pinpricks of light. But, as discovered by the special effects team on George Pal’s 1950 production of Destination Moon, seeing stars isn’t quite as simple as that.

According to screenwriter and author Robert Heinlein in his essay Shooting Destination Moon, first published in the July 1950 edition of Astounding Science Fiction:

“The greatest single difficulty we encountered in trying to fake realistically the conditions of space flight was in producing the brilliant starry sky of empty space. We fiddled around with several dodges and finally settled on automobile headlight bulbs. They can be burned white, if you don’t mind burning out a few bulbs; they come in various brightnesses; and they give as near a point source of light as the emulsions can record – more so, in fact. We used nearly two thousand of them, strung on seventy thousand feet of wire.”

Watch a behind the scenes video that was broadcast live from the set of Destination Moon in 1950 as part of KTLA’s City at Night talk show:

Six years later, when creating starfields for the sci-fi classic Forbidden Planet, A. Arnold “Buddy” Gillespie rejected light bulbs in favour of tiny discs of reflective Scotchlite material, meticulously adhered to a slab of black masonite. Illuminated by a ring of floodlights arrayed around the camera lens, this low-budget solution is described by Gillespie in his book of collected memoirs, The Wizard of MGM:

“[A] jet void, bejewelled with diamond stars and suns, and billions of light-years-away other galaxies with their own myriad stars and suns … It worked beautifully and inexpensively.”

After early experiments with backlit holes drilled in sheet metal, the stars for "2001: A Space Odyssey" were created by airbrushing constellations on to black paper.

After early experiments with backlit holes drilled in sheet metal, the stars for “2001: A Space Odyssey” were created by airbrushing constellations on to black paper with white paint.

In 1968, Wally Gentleman, the original special effects supervisor on 2001: A Space Odyssey, planned to generate starfields for Stanley Kubrick’s seminal sci-fi film by drilling small holes into metal sheets and backlighting them. As Gentleman explained in Cinefex 85, the concept worked well enough … but only as long as the camera remained stationary:

“As the camera tracked by, the holes became elliptical with relation to the lens, and the light intensity changed and the stars either faded out or twinkled.”

Frustrated by the challenges of working with the film’s notoriously demanding director, Gentleman eventually walked, leaving the film’s credited special effects supervisor, Douglas Trumbull, to step up to the plate. In the same Cinefex article, matte artist Richard Yuricich revealed Trumbull’s homespun solution to the starfield problem:

“Douglas would take an airbrush, and he would turn it down to where the pressure was maybe a couple of pounds. He would take this black paper, stretch it and glue it to a piece of board; and he would just sit there and squirt out these stars. In a matter of seconds it would be done – ‘There it is, boys!’”

Trumbull’s airbrushed starfields were photographed on an animation stand, using polarised light to eliminate any glare coming from the black paper. This technique – and variations upon it – became the industry standard for many years to come.

Over time, the randomly spitting airbrush gave way to the modern VFX artist’s ever-growing box of digital tools. In 1998, Dream Quest created scientifically accurate starfields for Armageddon using a database of constellations as seen from Earth – a procedure refined further by the VFX team at Double Negative for the recent hit Interstellar, as explained by visual effects supervisor Andy Lockley in Cinefex 140:

“Rather than painting star fields or trying to build little bits of geometry to project them onto, Oliver [James] wrote this renderer that referenced NASA star field maps. We could tell the renderer where it was pointing into space, and it would reference that star map and place pixel dots representing each real star in that position.”

MPC created three categories of digital star for Danny Boyle's "Sunshine".

MPC created three categories of digital sun for Danny Boyle’s “Sunshine”.

Lighting Up the Sun

Seeing stars from a distance is all very well. But what happens when filmmakers want to get up close and personal with a blazing sun?

In the 1986 sci-fi adventure Star Trek IV: The Voyage Home, Captain Kirk and the crew of his captured Klingon “Bird of Prey” spacecraft enjoy a close encounter with Earth’s parent star. The sun’s flaming surface was created by ILM using – of all things – basic bathroom accessories, as described by effects cinematographer Don Dow in Cinefex 29:

“Pete [Kozachic] came up with the idea of taking two pieces of Flemish glass – which is plexiglass textured like shower doors – and motorizing one of them so that it would turn against the other and give us a moiré pattern. We backlit it with a gelled 10K to give it a yellowish color and we vaselined the areas around the edge where we wanted it to fall off.”

However, filmgoers had to wait until 2007 for a movie that put the sun front and centre. For Danny Boyle’s Sunshine – in which humanity’s guardian star was almost a character in its own right – MPC forged three categories of sun shots, using Autodesk Maya to model the basic geometry of the star, combined with particle and fluid systems to generate the complex motion of its blazing surface.

For "Gravity", Framestore created a hyper-detailed volumetric rendering of Earth capable of holding up during the film's long takes.

For “Gravity”, Framestore created a hyper-detailed volumetric rendering of Earth capable of holding up during the film’s long takes.

Planetscapes

Stars are too big and hot to stay around for any length of time, so let’s move on to something a little more manageable. Like planets.

One of the earliest cinematic representations of a planet appears in Walter R. Booth’s whimsical 1911 film The Automatic Motorist. However, if you’re looking for a realistic portrayal of this most essential of heavenly bodies, you’re in the wrong place. Just scan forward through this video to the 2:20min mark and you’ll see what I mean:

For Forbidden Planet, Buddy Gillespie created wide shots of the alien planet Altair-4 by hanging painted balls – the smallest only six inches in diameter – in front of his background of Scotchlite stars. Dimensional models were still being used in 1981, when Film Effects of Hollywood photographed a large, spherical miniature of Jupiter to create the background vistas seen in Outland. Atmospheric effects were added to the gas giant by superimposing airbrushed artwork over the original stage photography – not to mention the time-honoured trick of smearing Vaseline on the camera lens.

In this same era, all three films in the original Star Wars trilogy were proving that spectacular shots of distant planets can be created very effectively just by using flat artwork. Orbital visions of Tatooine, Hoth, Dagobah and the other exotic worlds in that galaxy far, far away came courtesy of matte artists Mike Pangrazio and Ralph McQuarrie.

When it comes to a planet as familiar as the Earth, the real-world imagery made available by NASA and other space agencies has in recent years given filmmakers a wealth of reference material to draw from. When Robert Zemeckis made Contact in 1997, senior VFX supervisor Ken Ralston’s multi-vendor team had access to hundreds of highly detailed satellite photographs, which they re-purposed to conjure up realistic views of the Earth as seen from space.

By the time Framestore constructed the Earth seen in Gravity in 2013, satellite imagery had been discarded for everything other than visual reference. In order to accommodate the film’s long takes – during which the camera typically passed over a huge swathe of the Earth’s surface – VFX supervisor Tim Webber’s team had no choice but to re-create the planet as a highly-detailed volumetric rendering that accurately reproduced not only landmasses and oceans, but also atmospheric and meteorological effects.

To create the asteroids used in "Star Wars Episode II: Attack of the Clones", VFX artists at ILM scanned the miniature rocks used in "The Empire Strikes Back" and mapped the textures on to new digital models.

To create the asteroids used in “Star Wars Episode II: Attack of the Clones”, VFX artists at ILM scanned the miniature rocks used in “The Empire Strikes Back” and mapped the textures on to new digital models.

Rock and Roll

Space might be big, as Douglas Adams contested, but if sci-fi filmmakers are to be believed, it’s also full of rocks. For the breathtaking asteroid field sequence in The Empire Strikes Back, the visual effects team at ILM used flat painted artwork, stacked up in multiple layers to create the sea of rocks seen in the backgrounds of the spectacular chase scenes.

Asteroids passing closer to the camera were constructed in miniature and photographed spinning against bluescreen using motion control cameras. Also present, of course, were the speeding Millennium Falcon and a parade of hapless TIE fighters. The resulting chaos was described by visual effects supervisor Richard Edlund in Cinefex 2:

“By the time you’ve shot four ships, ten or twelve separate rock elements, three background paintings, a star field, plus miscellaneous explosions and lasers, you wind up with maybe twenty-five separately photographed pieces of film, each of which has to be broken down into color separations and … have all the intermediate bluescreen steps to extract mattes. So all together, you have maybe a hundred and twenty pieces of film involved.”

Twenty-two years later, ILM modelled a set of digital rocks for the asteroid chase in Attack of the Clones by resurrecting all the asteroid miniatures originally used for Empire. Having scanned these old-school rocks, they projected their textures – combined with procedural crater maps – on to new geometry, creating a randomised field of digital space debris. In a nod to verisimilitude (and just possibly a sideways swipe at the real world) they also had Jango Fett blow up a NASA computer model of Eros, one of the large asteroids in our own solar system.

A miniature asteroid also featured in Armageddon, for which Dream Quest built three practical versions of the giant rock that’s closing in on the Earth. The biggest of these – which was sculpted from foam and mounted on a steel armature in front of a greenscreen – measured a whopping twenty-five feet by fifteen feet. Not quite big enough to destroy a planet, but quite enough to keep a crew of thirty special effects artists busy.

Dream Quest constructed a number of miniature asteroids for "Armaggedon", the biggest of which measured twenty-five feet by fifteen feet.

Dream Quest constructed a number of miniature asteroids for “Armaggedon”, the biggest of which measured twenty-five feet by fifteen feet.

Dreams of Outer Space

From big to small, from stars to asteroids, the endless expanse of outer space is full of wonders. However, you might be wondering why I’m talking about outer space at all, given that this series of blog articles is titled Dream Landscapes. After all, there’s no land in outer space, is there?

The answer’s simple. Landscapes are all about the view.

After I’ve bought my ticket to the movies, picked up my popcorn and taken my seat, what’s the one thing I want to see when the camera pulls back? The answer is a spectacular view. That might mean a burning sunset lighting up the ocean, or the craggy slopes of a precipitous mountain range. It might mean a sea of desert dunes, or a glistening Arctic wasteland.

Or it might mean a cosmic vista filled with fiery stars, spinning planets and tumbling rocks.

Whatever that wide shot may be, filmmaker Werner Herzog was right: the place such landscapes – or spacescapes – really come to life is in our dreams.

In other words, when we go to the movies.

All the Cinefex articles quoted can be read in their entirety as part of the Cinefex Classic Collection, available for iPad. Related blog articles:

“Destination Moon” photograph from the May 1950 edition of “Popular Mechanics”. “The Hitchhiker’s Guide to the Galaxy” photograph copyright © 2005 by Touchstone Pictures “2001: A Space Odyssey” photograph copyright © 1968, 2001 by Turner Entertainment Company. “Sunshine” photograph copyright © 2007 by Twentieth Century Fox. “Gravity” photograph copyright © 2013 by Warner Bros. Entertainment. “Star Wars Episode II: Attack of the Clones” photograph copyright © 2002 by Lucasfilm, Ltd. “Armaggedon” photograph copyright © 1998 by Touchstone Pictures and Jerry Bruckheimer, Inc.

Preorder Cinefex 142 Now

Cinefex 142 contents

Updated 18 June 2015 – Cinefex 142 is now on sale! Pick up your copy from our online store.

Summer is just around the corner, which can mean only one thing – the next issue of Cinefex is nearly here … and available for preorder right now!

Issue 142 of the premier magazine for visual effects professionals and enthusiasts goes deep on the VFX of four of this years biggest movies. First up is Jurassic World, which brings together a team including ILM, Phil Tippett and key Stan Winston Studio crewmembers who were there for the first Jurassic go-round. The reunion promises thrills that could only have been dreamed of for the original, made when computer animation and other relevant technologies were in their infancy.

Next up is Avengers: Age of Ultron, for which VFX supervisor Christopher Townsend assembles a team that includes ILM, Double Negative, Animal Logic, Luma Pictures and Framestore, with special effects supervisor Paul Corbould and the mechanical wizards at Legacy Effects lending practical effects support.

For Mad Max: Fury Road, special effects supervisors Andy Williams and Dan Oliver provided in-camera action with picture vehicle supervisor Geoff Naylor, while hair and makeup designer Lesley Vanderwalt, prosthetic supervisor Damien Marton and Tinsley Studios assisted with mutant characters. Visual effects producers Alex Bicknell and Fiona Crawford guided visual effects with vendors including Iloura and Method Studios, Dr. D Studios, The Third Floor and Stereo D.

Finally there’s San Andreas, which sees special effects supervisors Brian Cox and Matt Kutcher creating practical earthquake destruction and flood effects, while visual effects producer Randall Starr oversees digital enhancements at visual effects studios that include Hydraulx, Scanline VFX, Cinesite, Method Studios and Image Engine.

 

Only one question remains. Which of these effects-heavy blockbusters will feature on the cover of Cinefex 142? All will be revealed soon, but in the meantime why not vote for your favourite in our fun poll?

The VFX of “Bad Land: Road to Fury”

Nicholas Hoult and the Simulit Shadow in "Bad Land: Road to Fury"

Science fiction films aren’t always set in the future. Nor are they always packed full of exotic hardware. Sometimes, science fiction exists in the cracks, walking that fine line between the everyday world and a subtly different reality.

One such film is Bad Land: Road to Fury (originally titled Young Ones), in which homesteader Ernest Holm (Michael Shannon) and his teenage son, Jerome (Kodi Smit-McPhee), struggle for survival in a remote desert land where water is the most precious commodity imaginable.

While the desiccated world created for the film by writer/director Jake Paltrow looks much like our own, what we see on screen also gives us glimpses into an alternate reality. Foremost among these is the “Simulit Shadow”, a robotic mule purchased by the Holms when the harsh desert environment finally takes its toll on their flesh-and-blood beast of burden.

The filmmakers based the design of this lumbering automaton – known as the “Sim” – on Big Dog, an experimental rough-terrain robot developed by Google subsidiary, Boston Dynamics. The Sim was created using a cunning blend of physical and digital techniques, combining the practical skills of Cape Town-based Cosmesis with visual effects by Windmill Lane VFX in Dublin, Ireland.

Watch a video breakdown of the Simulit Shadow effects created by Windmill Lane VFX and Cosmesis for Bad Land: Road to Fury:

According to Windmill’s visual effects supervisor, Ditch Doy, the director’s original intent was to use a fully-functioning robot. “Jake went to Boston Dynamics and tried for a couple of weeks to get a real robot for the film,” Doy revealed. “That just wasn’t realistic, but if Jake had had his way it would have been a completely robotic creation.”

Despite his initial plans being thwarted, Paltrow insisted that the Sim channel the spirit of the Boston Dynamics machines. “Jake wanted it to be clunky,” Doy commented. “He didn’t want it to be a sleek and sexy robot, but more like a piece of agricultural equipment. It was a kind of retro-futuristic look. And he was adamant that it had to look 100% believable – seeing it couldn’t be a visual effects ‘moment’.”

The robot’s robust, utilitarian design perfectly complemented the environment in which the film would be shot: a harsh stretch of desert terrain in Namaqualand, South Africa, just 50 miles south of the Namibian border.

“The locations were fantastic, but they were in the middle of nowhere,” said Doy, recalling his first meeting in the desert with Paltrow and the film’s cinematographer, Giles Nuttgens. “I flew to Cape Town on the red-eye, then had a seven-hour drive up to Springbok, which is a very small mining town. All that for a one-hour meeting with Jake and the cinematographer, Giles Nuttgens. A couple of weeks later I flew down again, and stayed out there for eight weeks on the shoot.”

The faithful Simulit Shadow robot was created by Windmill Lane VFX and Cosmesis using a combination of digital and practical effects techniques.

The faithful Simulit Shadow robot was created by Windmill Lane VFX and Cosmesis using a combination of digital and practical effects techniques.

A Simulit in the Desert

Also making the trip to Springbok was a set of full-scale Sims, each one over two metres long, constructed by Cosmesis. “We built a lightweight version that was hollow, and a fully functional version with hydraulic legs, lenses and functional lights operated by remote control,” revealed prosthetics, puppet and makeup effects designer, Clinton Aiden Smith. “The functional version was not used for walking, but only in scenes where the Simulit was stationary, or needed to stand up from a lying position ready for walking. We also had a version of the Sim that could be dented and take squibs for scenes where it was attacked.”

Cosmesis created three practical versions of the Simulit Shadow robot.

Cosmesis created three practical versions of the Simulit Shadow robot. Left to right: Adrian Smith, Gerald Clark Sutherland, Pierre Smith, Anja Rechholtz, Terri Nicole, Lisa-Marie Bothma. Photograph by Clinton Aiden Smith.

For scenes in which the Sim was required to walk, the lightweight version was used. Legless, it was supported and puppeteered by two performers, most frequently a pair of South African free runners hired for the task. Depending on how it was dressed, the mobile prop weighed in at anything from 23-55kg. “The weight was important, because the parkour performers needed to be able to operate and control its movement in very rocky terrain where temperatures could reach 110°.”

The Sim was initially developed as a 3D computer model using Autodesk Maya. “After we had basic approval of from Jake Paltrow, we projected the front and side views on to cardboard to establish the final size, keeping in mind the weight and the needs of the two performers,” Smith explained. “Once the basic size was locked off, we created a cardboard and PVC pipe mock-up, which we took to the location to get an idea of what we were up against. We realised that the Sim had to be even lighter than the mock-up, as the conditions were so demanding.”

Adrian Smith, Jake Paltrow and others assess an early cardboard and PVC mockup of the Simulit Shadow. Photograph by Gerald Clark Sutherland.

Adrian Smith, Jake Paltrow and others assess an early cardboard and PVC mockup of the Simulit Shadow. Photograph by Gerald Clark Sutherland.

The Cosmesis team created the final lightweight version of the Sim using vacuum-formed PVC components moulded from CNC-cut Superwood formers for the body and legs. The resulting hollow shell was reinforced with aluminium, which also provided anchor points for the puppeteers to grip. The cargo basket perched on top of the main superstructure was constructed using carbon fibre pipes.

While Paltrow was keen that the Sim should have an ungainly quality while walking, nevertheless it had to remain balanced and level regardless of the terrain – a considerable challenge for performers who were bent double with poor sightlines. A distinctly low-tech solution was devised to deal with this. “We hung four strings with weights from the bed of the Sim to remind the performers at what height they needed to operate,” Smith revealed. “This was crucial in showing them how much to bend their knees or backs while performing.”

Lightweight though the final rig was, operating the Sim was punishing. Desert conditions made dehydration and overheating a constant concern, despite the battery-operated cooling suits worn by the puppeteers. “The parkour performers were used to controlling their bodies, which was a plus for us,” remarked Smith. “But there were some days where I and Adrian Smith, our key Sim fabricator, had to stand in to relieve them – believe me, it is not as easy as it seams! Our main performers, Ryno Keet, Chris Jones and a local town boy from Springbok, went beyond expectations to breathe life into the Sim.”

Nicholas Hoult as Flem Lever in "Bad Land: Road to Fury"

Nicholas Hoult as Flem Lever in “Bad Land: Road to Fury”

A Digital Sim

Not only did having the practical Sim on location create a tangible robotic presence for the cast and crew, but for many shots it provided Windmill with a starting point for their visual effects. “A lot of the time we just painted out the puppeteers and stuck in our CG legs,” remarked Doy. “So what we got was a practical robot for the actors to interact with, that also had some nice organic movement. That was very important for Jake. He wanted the Sim to have a way of correcting itself gyroscopically. So whenever you see it, it’s always got a bit of life about it, as if it’s trying to self-adjust and stay upright. We could have done that with straight animation, but this way we got a more quirky performance.”

Despite Doy’s confidence in the approach, not everyone was convinced by what they saw. “It was a bit worrying on set, because it didn’t always look fantastic,” Doy admitted. “We were getting some very strange looks from the cast. In fact, Nicholas Hoult said to me, “You are going to make this look cool, aren’t you? Please?” I had to tell everyone, “Honestly, I know it looks weird now, but we’ll make it look great!”

Windmill also created a fully digital Sim for scenes in which it was not feasible to use the practical prop. By mixing and matching techniques, Doy and his team aimed to keep even the most attentive audience member guessing as to how the Sim had been created in any one shot: “Sometimes we used the puppet rig. Sometimes it was fully CG. We were always moving around the connection points, so hopefully you’ll never be able to spot which shots are purely digital, which ones are half-and-half, and where the joins are.”

Windmill's fully digital Simulit Shadow peers into a treacherous pit on a desert trail.

Windmill’s fully digital Simulit Shadow peers into a treacherous pit on a desert trail.

The digital Sim was modelled using Autodesk Maya, and rendered in Solid Angle Arnold. “This was the first feature where we used our new Arnold renderer, which is very good at doing metals and hard surfaces,” Doy observed. “For every shot we would take HDRIs to use as lighting guides. Mostly we were in desert sun, but we had some night shoots, and also had to shoot in the rain. The shots were composited in Nuke, and our whole pipeline is built around Shotgun.”

In a key scene, a highly emotional Jerome vents his anger on the Sim, attacking it with a sledgehammer. The direct physical contact between boy and machine created significant challenges for the Windmill team.

“We had to use a fully digital robot for that scene,” Doy explained. “I don’t think the puppeteers would have liked us abusing them that much! Originally, Kodi was just going to be swinging the sledgehammer through thin air. I said, “No, we have to have a contact point, because otherwise he’s just going to be swinging through nothing.” So we hastily rigged up some crates, and got Kodi to hammer away at those.”

For a scene in which Jerome attacks the Simulit Shadow with a sledgehammer, actor Kodi Smit-McPhee struck crates covered in green fabric. Windmill later replaced these with their digital robot, timing its animation to McPhee's performance.

For a scene in which Jerome attacks the Simulit Shadow with a sledgehammer, actor Kodi Smit-McPhee struck crates covered in green fabric. Windmill later replaced these with their digital robot, timing its animation to McPhee’s performance.

Even though the crates were covered in green fabric, the lighting conditions meant it was almost impossible for Windmill’s visual effects team to extract Smit-McPhee and the sledgehammer from the background, allowing the Sim to sit behind them in the frame. Much of the keying work was therefore done manually, with artists rotoscoping the action frame by frame, and manipulating the end of the sledgehammer digitally to ensure a good visual lock wherever it made contact with the CG robot. Windmill’s 2D supervisor for Bad Land: Road to Fury was Andy Clarke.

Most importantly, close attention was paid to the animation of the digital Sim, in order to maintain its indomitable spirit throughout the ordeal, despite the abuse being hurled at it. “Jake was adamant that he wanted the robot to have no emotions,” stated Doy. “It’s not stupid – it just doesn’t have any feelings. Jerome is going at it with the sledgehammer, but the robot is unflinching. It just keeps trying to re-stabilise itself and get back to a neutral position.”

The rough, agricultural design of the Sim is carried through to its navigation system – a single laser-equipped lens that constantly scans its surroundings. The Sim’s routine recording of everything it scans ultimately becomes an important plot point, when Jerome accesses its memory and discovers a dark secret. The filmmakers were therefore keen to get the look of the robot’s “Sim-vision” just right.

“Jake wanted to go for a retro, monochromatic display,” Doy explained, “and he didn’t want it to look like it was created through photography – no light or shade. But it still had to put across important narrative points, and you had to recognise certain characters. Jake also wanted a 3D aspect to the Sim-vision, so we projected footage on to crude geometry, and then animated it.”

Kodi Smit-McPhee and Michael Shannon as Jerome and Ernest Holm, accompanied by the Simulit Shadow.

Kodi Smit-McPhee and Michael Shannon as Jerome and Ernest Holm, accompanied by the Simulit Shadow.

Life Support and Future Tech

Visual effects were also used to enhance the appearance of one of the film’s human characters. Ernest Holm’s wife, Katherine (Aimee Mullins) resides at a medical facility, having suffered catastrophic injuries in a car accident. Paralysed, and missing her lower legs, she is able to move around using a complex ambulatory apparatus hooked up to a mobile life-support system.

Mullins, who underwent a double amputation as an infant, found fame as a successful para-athlete, model and actress. For her role in Bad Land: Road to Fury, Mullins wore specially-designed prosthetic legs, while the Rube Goldberg contraption that both keeps her alive and stimulates her spinal cord was added digitally by Windmill.

Windmill Lane VFX used digital effects to create a futuristic life-support system for Katherine Holm (Aimee Mullins) in scenes where she meets her son and husband (Kodi Smit-McPhee and Michael Shannon) in hospital.

Windmill Lane VFX used digital effects to create a futuristic life-support system for Katherine Holm (Aimee Mullins) in scenes where she meets her son and husband (Kodi Smit-McPhee and Michael Shannon) in hospital.

“The rig is a weird back-brace thing that goes up to a life-support machine,” Doy explained. “Jake wanted something that was driven by pistons and steam, rather than an expensive piece of medical equipment. The spinal cord coming from her back joining her to the overhead machine is all CG. Some of the cables are real, but during post, Jake hit on the idea of making Aimee look almost like a marionette. So we were tasked with sticking on extra cables. We weren’t really anticipating that, and it made the tracking much harder, and the shots more involved. But it works.”

Watch a video of Aimee Mullins as Katherine Holm in Bad Land: Road to Fury:

In the film’s later stages, water finally comes to the desert, and crops begin to grow. Despite the arid conditions at the Springbok location, the production initially attempted to cultivate real plants. “We did try to grow some greenery,” Doy commented, “but the sun was so fierce that everything just shrivelled and died. So all of the greenery towards the end of the film is mainly matte painting work. We also did a time-lapse shot of a wheat field growing, which involved a lot of complicated work in Houdini.”

During a brief trip out of the desert to a city across the border, Jerome encounters Anna (Liah O’Prey), a young woman who provides him with safe passage through the high-security checkpoint. Illustrating the difference in technology between the down-at-heel desert world and the neighbouring urban sprawl is Anna’s cell phone, which opens up like a Chinese fan.

“The fan was Jake’s creation,” revealed Doy. “He wanted to show that, across the border, they have nice cars and flashy mobile technology. On the set, it literally was just a fan. Alex opened it up, and we added all the graphics inside. It’s one of those moments in the film where you realise that this isn’t our universe.”

Windmill created interactive graphics for Anna's quirky futuristic fan-phone.

Windmill created interactive graphics for Anna’s quirky futuristic fan-phone.

A View of the Desert

In total, Windmill Lane Pictures VFX delivered around 250 shots for Bad Land: Road to Fury. As well as the practical Sim, Cosmesis also created special dentures for the main characters, while makeup artist Natasha Du Toit managed a progressive range of dry skin, dirt, sweat and tattoos. A number of scenes were enhanced by blood effects and prosthetics, including two broken legs – one human, one mule – while a range of prosthetic bellies simulated the various stages of pregnancy of Mary Holm (Elle Fanning).

Having worked on Bad Land: Road to Fury for about a year, Ditch Doy’s abiding memories are of his time on location in South Africa. “It was the hottest, driest place on the planet – and I think it shows on the screen,” he reflected. “That’s not visual effects – it really was as hot and dry as it looks. It was the toughest film experience I’ve ever had. We all went a bit stir-crazy. But we’d all fallen in love with the desert a little bit by the end. To watch the sun rise and set over that kind of landscape … it really is something you don’t get to experience every day.”

Clinton Aiden Smith craved only one thing while on location in Namaqualand: “Shade! At times, base camp was a long way from the set, so crew were hiking up steep, rocky hills with heavy bags and cases of gear. Man, if you could find some shade under a rock or a tree, you were lucky!”

Fittingly, Doy’s favourite “effects” shot in Bad Land: Road to Fury comes courtesy of Mother Nature. “It never quite rained where we were, but one day, in the distance, we saw a thunderstorm. Jake was smart enough to whirl the camera round and say, ‘Film that!’ So we did, and we caught this great lightning bolt, which is in the film. I’m hoping people don’t know is real, so I can say it’s one of our effects!”

Bad Land: Road to Fury is currently available from Signature Entertainment on Blu-Ray, DVD and Digital HD. Watch the trailer:

Special thanks to Signature Entertainment and Witchfinder. Photographs copyright © 2014 by Signature Entertainment and courtesy of Windmill Lane VFX and Cosmesis.

Maps to the Stars – VFX Q&A

"Maps to the Stars" - visual effects by Switch VFX

Veteran filmmaker David Cronenberg has a reputation for tackling the kind of subject matter that make audiences squirm. His early feature work – which includes Scanners, Videodrome, and The Fly – exposed mainstream audiences to the shock effects of “body horror”, while at the same time getting under their skins with unsettling psychological subtexts.

While Cronenberg’s most recent feature, Maps to the Stars, explores similar themes to some of his earlier films, it dials down the grossness in favour of the psychological. Set against – and satirising – a Hollywood that’s simultaneously larger-than-life and desperately shallow, it explores the intertwined stories of scarred pyromaniac Agatha Weiss (Mia Wasikowska), fading movie star Havana Sagrand (Julianne Moore), TV psychologist Dr. Stafford Weiss (John Cusack), his control-freak wife Cristina (Olivia Williams) and struggling actor Jerome Fontana (Robert Pattinson).

Visual effects for Maps to the Stars were provided by sole vendor Switch VFX, with the exception of a small number of minor shots undertaken by Deluxe. Under the supervision of its co-founder Jon Campfens, Switch delivered around 145 shots, including matte paintings, set extensions and greenscreen composites.

There was a little body horror, too – well, this was a Cronenberg film, after all. In addition to the “invisible” environment work, Switch also developed complex fire simulation effects for a key sequence in which Cristina Weiss sets light to herself beside a swimming pool, and enhanced other scenes involving blood and gore.

In this Q&A for Cinefex, Jon Campfens discusses the challenges involved in turning Toronto locations into the Hollywood Hills, and reveals just what David Cronenberg is looking for when it comes to setting a woman on fire.

Julianne Moore and Mia Wasikowska star in David Cronenberg’s “Maps to the Stars”

Julianne Moore and Mia Wasikowska star in David Cronenberg’s “Maps to the Stars”

How did you get involved with Maps to the Stars?

The line producer, who I know quite well, introduced me to David Cronenberg. We were brought on board a bit late in the project, during prep, as the original VFX company had decided to pass on it. They had already gone far enough down the road on some of the VFX approaches that they could not be changed – especially concerning the fire sequence – so we were locked into doing it that way. We had a number of discussions with David and the creative team to ensure we were all on the same page before we went to camera.

Tell us more about the fire sequence.

During the sequence, Dr. Stafford Weiss discovers that his wife, Christina, has set herself on fire. He proceeds to push her into their swimming pool with a deck-chair to try and save her. Beau Parsons, one of our on-set supervisors, attended the shoot along with Brandon Rogers, a member of our CG team and lead for the department that would be using Fume FX.

What direction did you receive from David Cronenberg regarding the action by the pool?

It was expressed to us by David during the prep stages that the performance comes before anything else. Despite our suggestions that shooting fire elements or using a stunt double would provide more realistic fire, David wanted to go the CG route in order to preserve Olivia’s performance. I am not a great lover of CG fire, but it was the only way to proceed in order to get what David wanted. It was important to him that the audience knew it was really her and not a stunt double using fire retarder – which has a very reflective quality to it.

For the immolation sequence, actress Olivia Wilde was photographed on location, in costume and wearing minimal tracking markers.

For the immolation sequence, actress Olivia Wilde was photographed on location, in costume and wearing minimal tracking markers.

Artists at Switch VFX, under the supervision of Jon Campfens, added CG fire effects simulated using Fume FX, and motion-tracked to Wilde's performance using PFTrack.

Artists at Switch VFX, under the supervision of Jon Campfens, added CG fire effects simulated using Fume FX, and motion-tracked to Wilde’s performance using PFTrack.

Given those parameters, how did you go about executing the fire?

Face replacement was out of the question, because she is moving around quite a bit – as you would be if you were trying to put out a fire. We knew motion capture would be key, to get the fire to interact with her body in a believable way; we also knew we would need to use a fairly minimal rig. We ended up using two DSLR witness cameras synched to the shooting camera to triangulate Olivia’s position. These provided motion capture data through PFTrack – not the most elegant way of doing it, but it’s fast, unobtrusive and cheap.

During the scene, the gown worn by the character Christina burns away to reveal her burned flesh beneath. How did you achieve the transition?

We shot the scene twice, once with Olivia in her gown and once wearing a burn suit. The burn suit was actually very problematic. It took much of the night to prepare, and the tracking markers didn’t want to stay on. By the time it was ready, the sun was starting to come up and we only had time for one run-through. In any case, as soon as she fell in the pool the suit became unusable, so it was very important to get it right on the first try.

How did you integrate the CG fire with Olivia Williams’ performance?

The key to getting the CG fire to react realistically to Olivia’s flailing body was to get a good, useable track. For this we used PFTrack and its motion capture capabilities. Using a three-camera setup, we were able to generate data that allowed us to track her movements in 3D space.

For shots later in the sequence, CG fire and smoke were combined with live-action plates of Wilde wearing a burn suit.

For shots later in the sequence, CG fire and smoke were combined with live-action plates of Wilde wearing a burn suit.

Using a rigged 3D model of Olivia, we attached specific joints to the corresponding locator data returned by PFTrack’s motion capture output. The model was then hand-animated to compensate for the joints that did not return valid tracking data. Once a camera track was achieved, and the witness cameras were lined up properly, we then did an object track for every tracking marker.

What was your approach to the fire and smoke simulation?

We used Fume FX, but we couldn’t get much information on, or examples of, using it with Maya – and we’re a Maya-based company. So we did a lot of R&D, working through all the kinks, errors, and complications.

We generated smoke and fire from the tracked geometry, and rendered countless simulations in order to get what David was looking for. To make adjustments and revisions less time-consuming, we broke down the simulations and renders into smoke and fire passes for individual body parts – arms, legs, torso, etc. This also allowed for more time-friendly simulations at a smaller voxel spacing, creating extra detail in the fire tendrils and smoke. Animated texture maps were created in Nuke, and piped into Fume’s emitter channels. This permitted the fire to start at her legs and spread upwards, completely engulfing her.

Was it difficult keeping the flames locked to her body as she moved?

At times, the fire simulations would run into issues when Olivia’s arms were flailing around. For example, there might not be enough voxels being emitted because of the speed of her movements, or the fire might shoot out in wrong directions.

By setting keyframes on specific parameters within Fume, we were able to tame the behaviour of the simulation so that it would behave exactly as we needed. Using parameters like Wavelet Turbulence in Fume, we also achieved more time-friendly simulations, adding extra detail to low-resolution simulation caches, while preserving the overall motion. We added reflections and interactive light in compositing. So, lots of work was needed for this sequence!

In another scene, Agatha attacks Havana with one of the acting awards she’s won. What did Switch VFX contribute to the scene?

Production had applied some blood to Julianne Moore’s forehead, face and chest, but it wasn’t enough, and the forehead prosthetic didn’t look real. Throughout the sequence of shots, we added more and more blood and damage to her forehead, and blood spray to the chair and lamp.

David wanted to have the forehead wound look concave, so David Alexander, head of our 3D department, made an appropriate element in Maya. The plates were tracked using PFTrack, with our lead tracker and coordinator, Beau Parsons, making sure the CG element moved properly, because Julianne did fling her head around quite a bit. The CG element was composited into the shot with colour correction, using Nuke, enhancing highlights so it would look wetter and bloodier, and we added extra dripping blood.

Viewing the sequence on a large screen, we realised that the chair Julianne was sitting on was moving subtly with her, so the added blood spray wasn’t tracking properly. The audience would probably never see it, but we went in anyway and fixed it with a gridwarp track in Nuke.

For a scene in which Havana Sagrand (Julianne Moore) is bludgeoned with a movie award, Switch FX added extra blood and gore, and enhanced the appearance of the forehead prosthetic worn by the actress.

For a scene in which Havana Sagrand (Julianne Moore) is bludgeoned with a movie award, Switch FX added extra blood and gore, and enhanced the appearance of the forehead prosthetic worn by the actress.

Many of David Cronenberg’s films contain iconic “body horror” scenes. Was there any sense that you were adding to that body of work?

There was at first, but after seeing the film twice, now I’m not so sure. Maps to the Stars deals a lot with apparitions seen by some very unstable people, many of which appear around that same pool area. The immolation scene seems to come out of left field. There is no gas can or anything that would cause her to ignite as she does. We clearly see the pool, but where does her body go after? So I’m not sure if that scene happened at all, or if it was in Stafford’s head – not an unusual thing for a Cronenberg movie, but it was definitely more subtle than his earlier work.

David is very good at getting under your skin and making you feel uncomfortable, whether it’s with graphic body mutilations, or with the way people communicate or interact with each other with topics that the audience might feel unsettled.

Switch VFX also created a number of set extensions and background replacements. Tell us about the scene in the coffee shop, where you replaced the Toronto exterior with a Los Angeles street scene.

In the script, the location takes place inside Denny’s on Hollywood Boulevard. The interior sequence was shot in Toronto at a coffee shop/diner, on one of the major roads here in the city. A greenscreen was placed outside the window at the curb to allow extras to walk back and forth along the sidewalk outside the diner.

"Maps to the Stars" background replacement by Switch VFX.

It was a pretty straightforward set-up. The foreground plates were shot, with camera information recorded, and then comped with a location plate of Hollywood Boulevard. In order to get the right perspective from inside the diner, we had to manipulate the background plate by resizing the image, creating a few matte painting building elements, and add some vehicles driving by. Also, there were timing issues on the extras walking in the background – some of them had to be removed or replaced.

Another thing we did for this sequence was a continuity fix with a reverse angle on Jerome, which required us to remove a waitress, and make her appear on the angle on Agatha. We also had to add some arm movement to the fry cook – in a lot of his takes he looked frozen, like a still image. We did this by rotoscoping out his arm from another take and compositing it into the hero plate. This sequence was mostly handled by Barb Benoit using Digital Fusion.

What else did you do to turn Toronto into Hollywood?

During the editing process, there were a number of shots where David wanted to add the Hollywood Hills to the original Toronto location. All of them required rotoscoping of the actors and the objects within the frame. Luckily, the shots were filmed outside, and the sky was quite uniform, so we were able to get a rough key for most of it. It did still require a lot of rotoscoping of any fine hair on Mia and John Cusack, as well as moving objects with in the frame.

All the Hollywood Hills shots were tracked using Nuke’s 3D and conventional trackers. The background plates were shot with a still camera in L.A. from a number of angles, because we were not sure which one would work best for the shots until we started putting them together. Also, couldn’t actually shoot where the scenes took place in the script, so our matte painter took the stills and tiled a number of them together to create the correct view.

We had to remove a lot of foreground buildings, the Hollywood sign, electrical cables and so on, in order to get exactly what David was looking for. We replaced all the skies as part of the matte painting process as well. This was then put all together by our senior compositor, Gudrun Heinze, using Nuke.

How involved was David Cronenberg with the visual effects?

He’s a very technically-oriented guy, so you can have a conversation with him about what steps are needed to achieve these shots, and he gets it. He also knows when to take a step back. I feel he treats his departments the same way he treats his actors – he wants them to perform as best they can, and he knows to achieve that he doesn’t need to direct every single detail. He was very clear in describing what he was looking for, so after he had taken us through it, he pretty much left us to decide how we wanted to execute it.

Most of the shots in Maps to the Stars were straightforward and seamless, so not much was required creatively. However, the fire sequence was very much a back-and-forth process with David, because the outcome was very important to him. As everyone knows with fire and water simulations, you set one up and see what the outcome is, so we had to go through quite a number of simulations before he was happy. Overall, David was a pleasure to work with.

How do you feel about Switch VFX’s work on the show, looking back at it now?

We are very happy with the work. Most of what we did are seamless effects, and they are integrated into the film quite well. Some critics seemed to focus on the fire scene as taking you out of the film because of the clearly CG nature, but considering what was involved, and knowing we had to go this route – and because the scene is very surrealistic and unsettling – we feel that it works quite well.

In this industry, we all wish we had more time to perfect things. But if we were left to do that we would never deliver, because as artists we are never satisfied! We are very proud of the end result, and we look forward to getting an opportunity to work again with such a respected filmmaker as David.

Maps to the Stars was released on DVD/Blu-ray and Netflix on 14 April 2015.

Maps to the Stars photographs copyright © 2014 by Focus Features and courtesy of Switch VFX.

Some Very English VFX

Aidan Turner in "Poldark"

With its dramatic coastline, bleak moorlands and rolling hills, the picturesque West Country of England has long been popular with filmmakers seeking landscapes filled with both character, and a sense of history.

The counties of Dorset, Devon and Cornwall have featured in films like Barry Lyndon, The French Lieutenant’s Woman and War Horse – not to mention almost every Jane Austen adaptation ever made. They’ve even made appearances in recent big-budget blockbusters including World War Z and Edge of Tomorrow. Coming right up to date, these very English landscapes play a significant part in two new productions – one made for the big screen, and one for the small.

First up is the feature Far from the Madding Crowd, adapted from the Thomas Hardy novel and directed by Thomas Vinterberg for DNA Films. Distributed by Fox Searchlight Pictures, the film tells the tempestuous story of Bathsheba Everdene (Carey Mulligan), and her attempts to juggle three different suitors, in rural 19th century England.

Second is Poldark, a new television adaptation of the classic novels by Winston Graham. The Mammoth production is directed by Edward Bazalgette and Will McGregor, and stars Aidan Turner as Ross Poldark, recently returned to England after fighting in the American Revolution to find his father dead and his inheritance in tatters.

Both productions feature a range of visual effects by two Soho-based companies, designed to enhance the natural landscape and help in the task of whisking viewers back through time to a bygone age.

Watch a Union VFX breakdown reel showing their work on Far from the Madding Crowd:

Far from the Madding Crowd

As sole vendor on Far from the Madding Crowd, Union VFX created visual effects for a number of key scenes, including a burning-barn sequence using a combination of CG and real fire elements, with embers and smoke created using Houdini.

Equally dramatic was a scene in which a flock of sheep is herded over a cliff by a sheepdog. In order to choreograph this complex action, a small group of sheep was photographed leaping over a small drop. Union took these live-action plates and multiplied them many times over to create the illusion of the entire flock in free-fall. Similar replication techniques were used to enhance additional harvest and snow scenes.

Tim Caplan, Union’s co-founder and lead VFX producer on the film, commented, “It was inspiring to work in Dorset – such a beautiful part of the world – and re-imagine it as it was at the turn of the 20th century. It was also a privilege to work with one of the film industry’s most exciting directors in Thomas Vinterberg. And the project itself was an exciting challenge. The big fire sequence, for example, came with a host of logistical, and health and safety restrictions. We had to work out how to keep that sense of real jeopardy and authenticity in the picture – but without of course endangering any of the cast or crew. Far From The Madding Crowd has been a fascinating experience and we are hugely proud to have been a part of it.”

Watch a trailer for Far from the Madding Crowd:

Poldark

Visual effects for Poldark were supervised by Alexis Haggar, and include a number of set extensions and digital matte paintings created by Lexhag VFX.

One of the major challenges facing the production was the re-creation of the 18th century Cornish tin mining insdustry. While some mining architecture still survives in the region, much of it is in ruins. The task of restoring it to full working order therefore fell to the Lexhag team.

51-PD(19JUN)2014_240

The first step towards building a digital tin mine involved a detailed 3D scan of the location. “All of the major set extensions were started with a LiDAR scan,” Haggar explained. “For Wheal Leisure – Ross Poldark’s mine – the art department built the lower areas around an existing mine on the Cornish coast and we took over for the higher elements, such as the roof structure and windows. Grambler, the large mine set into the hillside, was a combination of digital matte painting and 3D elements all composited in Fusion Studio.”

Visual effects artist Ken Turner elaborated, “All the 3D elements were rendered as .exr files and brought into Fusion to relight and grade. The .exr files handle multiple light passes, and masks for all of the separate elements, which gives you a lot of control for interactive adjustments in Fusion. Once the still frame was close to the finished article, I then took it into Photoshop for a final paint, breaking up the clean CG edges with grime and rock before taking it back into Fusion where I added people, smoke, grain, lens aberrations and lots of little tweaks to make the still matte painting come to life.”

Commenting on the demands of television production timescales, Haggar concluded, “Speed is the key for us. Keeping your creativity alive while compositing has always been a challenge. Waiting for elements to render or playback is always frustrating. Fusion provides the best of both worlds: fast compositing or high accuracy “pixel pushing” for absolute perfection.”

42-PD(09JUN)2014_816

Special thanks to Stephanie Hueter and Cheryl Clarke.