Visual Effects – Art or Science?

by Graham Edwards

Is visual effects and art or a science?

In this digital age, visual effects has stopped being an art and become a science. Now that computers rule the roost, movie magic has become an endless round of number-crunching, pixel-wrangling, and worshipping before the Great God of Physics-Based Simulation.

Or has it?

Isn’t the opposite true? Don’t the infinite possibilities of CG mean that visual effects professionals have the freedom to craft moving images that are more spectacular and dramatic than ever before? Physics be damned – surely the primary aim of visual effects is just what it’s always been: to tell an amazing story.

Truth or beauty? Which is it? Or is there room for both in the ever-evolving world of visual effects?

There’s only one way to find out: ask the experts. So that’s what I’ve done, by putting this simple question to a range of leading VFX professionals from around the world:

  • Which is more important – obeying the laws of physics, or producing a shot that’s artistically right?

Here’s what my panel of experts had to say:


Joe Letteri
Senior VFX Supervisor, Weta Digital

“Realism informs everything that we do. But there’s no way to get all of the science absolutely right. Most of it is still unknown, or can’t be solved exactly. So in the end we use artistic interpretation and instinct to make audiences believe in what they are seeing.”


Joan Panis
Head of FX, MPC Montreal

“What’s important is the end result. It’s all about composition and timing. This can vary from shot to shot, but a lot of the time real physics doesn’t actually achieve good results. CG artists often need to cheat by reducing gravity or changing its direction, adding forces to get a plume of smoke in the right position, or manipulating the way explosion debris flies towards the camera or past an actor.

“Sometimes, things just don’t look right, even if they are physically accurate. So we create invisible forces to make it feel right. All that matters is that the composition of a shot throughout its frame range makes sense to our brains. I find that our eyes and brains have a surprisingly high tolerance for real-world inaccuracies.

“For the end battle sequence of Guardians of the Galaxy, our FX team built a library of ship explosions that were placed – mostly automatically – through our crowd system. One of the main problems we anticipated was that, as the ships flew up, down and sideways, the accompanying explosions would have to keep their directionality. But we also had to keep in mind that gravity should be pulling the exploding pieces downwards.”

Guardians of the Galaxy - MPC

“For the end battle sequence of “Guardians of the Galaxy”, our FX team built a library of ship explosions that were placed – mostly automatically – through our crowd system. But CG artists often need to cheat by reducing gravity or changing its direction, adding forces to get a plume of smoke in the right position, or manipulating the way explosion debris flies towards the camera or past an actor.” Joan Panis, MPC

“We thought about creating a large number of library elements to cover all the angles at which the ships were flying and exploding. In the end, it turned out we didn’t need to do as much as we’d thought. We mostly used just three types of element: the ship flying straight horizontally, flying upwards at an angle of 60 degrees, and flying downwards at an angle of 60 degrees. That was all we needed in order to sell the physics of the battle.

“Technically, the ships flying at 45 degrees and exploding have a gravity skewed by 15 degrees. But as most of the shots were fast-paced action shots, and because the explosions fire pieces in all directions, our brains don’t have time to register the skewed gravity.”


Andy Hayes
Head of Effects, Framestore

“We’re always searching for the best-looking motion to enhance the director’s vision for a shot. This means making things move correctly so that the illusion of reality is not questioned. But it also means adjusting that motion to convey the many aesthetic targets – composition, timing, scale, peril, etc – that are key in making the shot and sequence work.

“Starting with something that’s physically correct is the norm for us. But a shot naturally evolves with feedback. Physical parameters are tweaked or animated, and additional constraints or forces are applied to make things move in a more specific way. So we end up with something which may not be 100% physically correct, but which conveys the director’s vision – which is far more important.

“As an example, for the destruction of the Hoverbots and Necrocraft in Guardians of the Galaxy, we didn’t rely on simulation alone. Our underlying physical rigs utilised many different constraints and forces to pull the craft into specific crash locations, based on compositional art direction.”

Guardians of the Galaxy - Framestore

“For the destruction of the Hoverbots in “Guardians of the Galaxy”, we didn’t rely on simulation alone. Our underlying physical rigs utilised many different constraints and forces to pull the craft into specific crash locations, based on compositional art direction.” Andy Hayes, Framestore.

“Abstract effects with simulation qualities can add additional complexities. For example, the Tengu Monk fight sequence from 47 Ronin showed swathes of a cloth-like substance extruding from the demonic monks. In these shots, we were totally breaking physics in that the simulation material grew and changed over time – and ultimately even transformed into a gaseous substance. It was important to wrangle this material into something that was physically correct initially, but once that was done, our Creature FX department head, Carl Bianco, employed a LOT of tweaks to make the simulation twist and fold in the desired way.”


Daniele Bigi
DFX Supervisor, MPC

“In every production I’ve been involved in over the last 12 years, the main goal has always been to achieve the filmmaker’s artistic vision. How the team accomplishes that is really secondary.

“Building a set-up that obeys the laws of physics can help to produce a controllable and predictable result. Nevertheless, there are many situations where the director’s vision calls for a specific look or effect that’s completely artistically driven. In this situation, the software we use has to be flexible enough to allow artists and TDs to step back from real-world physics.

“On Guardians of the Galaxy, for example, at the beginning of the final battle between Star-Lord (Chris Pratt) and Ronan (Lee Pace), a gigantic hemispherical plasma shield is deployed by the Ravagers in order to hide their ships from the Dark Aster mothership.

“The shield is made up of a red-orange storm cloud effect, scattered with thousand of electric bolts running through it. The shield distorts the sky, and its opacity changes based on the camera angle, making the Ravagers invisible only from the Dark Aster’s point of view. Anyone trying to tackle this complex effect by obeying only the laws of physics would find it very difficult indeed! I think we created a visually beautiful shot without being restricted by the rules.

“As DFX supervisor, however, I must say that I would probably pick scientific accuracy over artistic interpretation. I think the industry is moving naturally in that direction anyway and the quick and wider adoption of PBR (Physically Based Rendering) is just one example of that.”


Ron Herbst
CG Supervisor, Digital Domain

“Hands down, no matter what, the shot must ‘work’, and physics be damned. What we do is entirely about storytelling. If the shot doesn’t tell the story, it is not serving its purpose.

“My answer is both philosophical and pragmatic. A client will reject a shot that in his or her opinion doesn’t work. Their reasons for why something does or doesn’t work are theirs to give, and ours to interpret to the best of our ability. In the end, what we deliver must satisfy the client, and any energy we exert in the interest of scientific correctness risks delaying a show’s delivery or chewing up the margin.

“That said, how we move towards getting a shot to work must begin with correct physics. There’s something in the subconscious of all audiences that looks for what it expects. If some of the basic physics isn’t there, the viewer won’t believe it’s really happening.”


Lon Molnar
Owner and VFX Supervisor, Intelligent Creatures

“That’s a tough question. This is the constant internal battle: challenging our technical integrity with the creative ability to bend reality, in order to tell the story.

“I think that, in the world of filmmaking, what we ultimately do is create illusions. We do this by any means necessary.  We aim to replicate reality, yet we might also create an alternate reality. Our goal is to satisfy the director’s unique vision – sometimes at the sacrifice of real world physics. If the rules need to bend in order to accomplish this, so be it. At the end of the day – if we’ve done our job – the audience is lost in the spectacle.”


James Reid
Head of Effects, Milk VFX

“When designing an effect, it’s important to find the right balance between realism and art. An awareness of the science behind natural phenomena is key to building something that’s believable, even when dealing with magical effects. This approach can be useful in refining an effect, and thinking of ways to add interesting detail.

“However, it ‘s paramount to know when to be more flexible and stray away from realism. From a technical point of view, similar results can often be achieved through a variety of methods, and the most scientifically accurate approach is rarely the most efficient, both in terms of artist time and computation time. Furthermore, the requirement to choreograph the movement of an effect often dictates a more visually-driven approach.”


Kevin Campbell
Director of Production Technology, Rising Sun Pictures

“The most important thing is producing a shot that the director likes. This is generally something inspired by physical laws, but not bound by them.

X-Men: Days of Future Past - Rising Sun Pictures

“Most of the effects in the Quicksilver sequence were inspired by physically correct simulations, but we retimed them in an art-directed way which broke most physical laws.” Kevin Campbell, Rising Sun Pictures.

“The Quicksilver kitchen sequence in X-Men: Days of Future Past is a good example. Most of the effects in this sequence were inspired by physically correct simulations, but we retimed them in an art-directed way which broke most physical laws. The results are largely physically plausible and artistically satisfying but not physically correct.”


Ara Khanikian
VFX Supervisor, Rodeo FX

“I think this issue exists in almost every shot I’ve worked on! I also think VFX should always obey the laws of physics … for the first take or two at least. After that, it’s up for grabs.

“Most feature film VFX work tries to replicate real-world physics and behaviour. It’s an on-going challenge to always produce photoreal effects … but then again, what is photorealism? Is it what we see with our own eyes, or what we’re used to seeing through a lens? The difference is a huge one.

“I believe that if you create VFX using real-world parameters like gravity, weight, laws of inertia and momentum, and proper lighting, it’s easier to then dial in artistic liberties than if you start off by considering only the artistry. The laws of physics ground the shot, and the artistic interpretation makes it appealing to look at, and communicates motivation and performance.”


Sean Faden
VFX Supervisor, Pixomondo

“Ideally, the simulation software allows the artist to experiment with the behaviour, while being able to rely on basic physical principles to give the effect its underlying realism.

“Five years ago, we would probably have used a combination of physical sim software and hand animation or procedural particles. However, today’s sim engines are faster and more reliable, to the point of letting the artist or TD create something pretty great in a few iterations and with minimal manual adjustment.

“This is especially true with water sims. Sure, we can art direct things like spray and secondary effects, but the overall behaviour when left to tools such as Realflow or Houdini fluids is pretty amazing. Sims for volumetric effects (including fire) leave a little more room for interpretation and fudging of parameters to get the desired final effect or feeling of scale.

“A recent project required us to create burning flags for a battle sequence. Since the shots were hyper-dramatic, we simulated the fire at a much larger scale to emphasise the slow-motion and high detail. We used Fume’s simulation toolset, but with cheated physical parameters to hit the desired look.”


Erik de Boer
Animation Director, Method Studios

“In the work I do – the photorealistic integration of animals and creatures – understanding the laws of physics is crucial. But most often, selling that physicality means that once we’ve determined the ‘real-life rules’, we happily break those rules for dramatic and artistic reasons.

“Maybe the top speed of an F-16 feels too slow relative to the camera, and makes the shot a bit boring. The way an animal’s limb changes shape when it meets the ground might need to be exaggerated, or its fur might need extra overlap to show a violent percussive impact.

“Such deviations from Newton often annoy our technical animation department or FX colleagues who like to play by the rules. But although we’ve often tried to implement algorithms or interfaces that help or even force us as animators to obey the rules, we usually find that they get in the way, and that we need to have some fun with them to make the performance work.”


Max Solomon
Animation Supervisor, Framestore

“On Gravity, the challenge was to create physically extreme, visceral shots that were totally believable. It was a fine line to tread. If the audience thought for one moment that Sandra Bullock wasn’t really slamming into the ISS, or spinning off into space at the end of the Shuttle’s Canadarm, the spell would be broken and they would cease to share in the jeopardy of the character.

“However, we were aware that if we had stayed within the limitations of what the human body can endure without sustaining serious harm, we would have ended up with some pretty boring shots. So we needed to push all the action to the max.

“In reality, the G-force she would have experienced while attached to the end of the Canadarm would probably have made her head explode, and when she slams into the solar panels on the ISS at 60 miles an hour … well, you’ve seen car crashes!”

Gravity - Framestore

“On “Gravity”, we were aware that if we had stayed within the limitations of what the human body can endure without sustaining serious harm, we would have ended up with some pretty boring shots. So we needed to push all the action to the max.” Max Solomon, Framestore

“We were certainly helped by the fact that the action was taking place in zero gravity, and the astronauts were in suits which cushioned the impacts, so the audience subconsciously were less critical of what they were seeing. But ultimately it was about making everyone go, ‘Wow!’ and not, ‘Oh yeah that looks just like NASA footage on YouTube.’”


Geoff Scott
VFX Supervisor, Intelligent Creatures

“Both are important, to a degree. It’s paramount to always start with reality as your base, whether it’s the motion of a creature or person, how light reacts when passing through a certain type of material, or a complex fluid simulation. The laws of physics always provide us with our foundation.

“Once we establish this baseline, that’s when we enhance the effect according to the client’s direction. The key goal is not to get locked into the mentality of “but this is real”. When dealing with a director or producer, that doesn’t matter – it’s about the integrity of the scene and the shot. Our goal is to support that; if we need to bend the laws of physics, so be it.

“For example, for the documentary Battlefield: Cell, we worked with Cambridge University and various scientific advisors to recreate a viral invasion of a host cell. Our mandate was to design the film as if it were a space epic – à la 2001: A Space Odyssey – showcasing huge expanses between sister cells and the inner cell was massive. The reality is that cells are so densely packed there’s no negative spaces between them, plus we’re not even sure if light works the same way at that scale. So, although our protein structures were accurate, the entire film is an impossibility.”


Lee Carlton
CG Supervisor, Digital Domain

This is a great question. It should really be answered by the person you’re trying to please with the shot direction. More often than not, the answer is that you need more control in artistic direction, to be able conform to a preconceived concept already developed as an ideal in that person’s mind, and waiting to be realised in digital form.

A perfect example is a recent shot we did involving a ship in space being obliterated by foes. We needed to art-direct the shot to create an ideal, crowd-pleasing explosion, rather than a truly realistic explosion as it would happen in space. By the way, the shot came out fantastic!


Wil Manning
VFX Supervisor, Pixomondo

“I find the idea of obeying the laws of physics to be seriously flawed. Our own eyes and brains play tricks on us constantly and, in my opinion, visual effects is a bunch of magician’s tricks designed to make people feel amazement, emotion and excitement.

"The Prestige" poster“In defence of this argument, I’m going to borrow from Christopher Nolan’s The Prestige (perhaps with irony, as his films are known for invisible effects and an avoidance of CG).

“The first step in any magic trick is ‘The Pledge’, where we show something ordinary: “It’s film, it’s through the camera, what you see is real.” Next is ‘The Turn’, where we take the ordinary and we make it extra-ordinary. With VFX, that means doing something that’s impossible: travelling into space; making someone run faster than the speed of sound; venturing into a boys dreams.

“Then comes ‘The Prestige’. This is the art: once the audience suspends disbelief of the unreal, we can show them fantasy. From space, we take them into a black hole; our speed runner catches bullets; the boy dreams of turning into an automaton. What you end up with is immersion in fantasy, and that’s pretty cool.

“For me, to make physically accurate visual effects is very important – it’s the foundation on which the artistry rests. If someone believes what you’re doing could be real, then you have them in your pocket and you can start to truly amaze them. You can perform magic.

“In addition, cinema is a language of codes and signals that’s tacitly understood by the audience, and it shouldn’t surprise us when we move to another culture than the language of cinema is as different there as the day-to-day spoken language. I work a lot in China, and it amazes me what’s important to the Chinese market, culturally speaking, compared to the USA and Australian markets (where I hail from).

“In China at the moment, there is much less emphasis on physically accurate work and a much stronger focus on making something that amazes – even if the work at times seems implausible. This can be pretty hard to deal with, because it’s in the nature of Western VFX to put realism before artistry. Most of our tools are biased this way – look at the recent tidal shift to Physically Based Rendering.

“The language of Chinese cinema seems more forgiving of the unreal and informal. Film makers as diverse as Tsui Hark, Jiang Wen, Zhang Yimuo, Wong Kar-wai, John Woo and Jia Zhang Ke (to name just a few) all speak with voices etched in unreality. I feel that for them realism takes second place to story and raw impact, and for Westerners working in VFX in China that sometimes that means leaving our preconceptions at the door. It’s a great experience!”


Conclusion

Well, the vote seems unanimous. For animation directors, VFX supervisors and FX experts alike, the answer remains the same: real-world physics may be the perfect starting point for most visual effects challenges, but it takes a healthy dose of artistry to make a truly sensational shot.

Is this a surprise? Hardly. As has frequently been observed, a computer is just a tool. It’s the person behind it who pulls the rabbit out of the hat.

And visual effects remains what it has always been: magic.


Special thanks to Stephanie Bruning, Jenny Burbage, Ian Cope, Anouk Deveault, Dave Gougé, Joni Jacobson, Melissa Knight, Che Spencer, Liam Thompson, Jonny Vale and Karl Williams. Guardians of the Galaxy and X-Men: Days of Future Past photographs copyright © 2014 by Marvel Entertainment and Twentieth Century Fox. Gravity photographs copyright © 2013 by Warner Bros. Entertainment. All rights reserved.

3 thoughts on “Visual Effects – Art or Science?

  1. Good article. VFX whether optical or digital, have always been about both art and science. Only the tools have evolved.

  2. Pingback: Visual Effects – Art or Science? – Cinefex Blog ‹ netzfilter™

Comments are closed.