When was the last time you went to a planetarium? What did you see? An astronomer telling you all about the night sky? A bunch of stars projected on to the inside of a dome? Maybe a laser making psychedelic patterns to a prog-rock beat.
Well, things have moved on.
One of the places they’ve moved to is the UK’s National Space Centre in Leicester. As well as containing a pair of rockets, a real chunk of moon rock, and the Sir Patrick Moore Planetarium, this futuristic visitor attraction also houses NSC Creative, a leading creator of fulldome immersive experiences.
Fulldome shows started popping up at planetariums during the 1990s. Suites of nifty new projectors meant you could suddenly do more with your dome than just throw a few dots up on the ceiling. Instead, you could project a movie – but not any ordinary movie. A fulldome film extends beyond the confines of the traditional cinema screen, presenting moving images that both fly over your head and creep up on you from behind.
Fulldome really is film in the round.
I spoke to Paul Mowbray, Head of NSC Creative, and Aaron Bradbury, CG Supervisor, about what life is like under the dome.
The Fulldome Format
In the early days, fulldome experiences were cobbled together using a range of technologies old and new. NSC Creative’s first show – Big, a documentary exploring the vast scale of the universe – employed a traditional opto-mechanical star ball for those all-important dots, but also featured cross-dissolving panoramic photography, computer animation, and even some stop-motion, courtesy of the team’s ex-Aardman staff.
“We had this crazy menagerie of different analogue technologies, plus a bit of early digital,” Mowbray recalled. “We had a partial dome system with three enormous CRT projectors running NTSC video, plus two banks of all-sky 35mm film projectors. We had a laser; we had lighting effects. It was a real hodge-podge.”
These days, the jumble of equipment has given way to a streamlined bank of six digital projectors. They’re fed a single 4K diameter fisheye image, 360° x 180°, rendered as a circle on a square frame and then sliced into segments, one for each projector. Each segment is projected on to the dome in perfect alignment with its neighbour.
“We use software edge-blending, as well as hardware blends – that means physical combs around the edge of the projector lens,” explained Mowbray. “There’s a computer dedicated to each projector, each of which plays its slice of the pie as a high resolution MPEG.
“Other people play back uncompressed footage from one computer, off a fast SSD striped RAID with a 6-head graphics card. 4K digital cinema projectors are also quite common, because you get more pixels with fewer projectors. With that you have two projectors facing each other; each has a half-fisheye lens with a blend across the middle. 4K is the current accepted standard, but we’re starting to see 8K systems using six or more 4K projectors. We’ve been doing research to work out how far you need to go before you can’t actually perceive any difference in the resolution.”
Aaron Bradbury added: “Overall, we think 16K is a happy medium. You capture most of the audience at retinal resolution, and the people sat near the edge of the dome are still getting a reasonable experience. The image is distorted for them, but that’s no different to be sat to the side of the cinema when you’re watching a feature film.”
NSC Creative specialises in producing quirky, family-friendly science documentaries like Astronaut (narrated by Ewan McGregor), We are Astronomers (narrated by David Tennant) and We Are Aliens (narrated by Rupert Grint).
The in-house productions are interspersed with commissioned work for other dome centres or commercial clients. A recent commission was Back to the Moon for Good, promoting the Google Lunar XPRIZE, which offers a $30 million prize to private companies for developing a new lunar lander.
Typical turnaround for a 25-minute fulldome show is around a year. Under extreme circumstances that can be reduced to just four months. The key to success under such tight timescales is planning … and sticking to the plan.
“We’re brutally efficient,” said Bradbury. “Everything we do ends up on the dome. We do the storyboard, we do all the layout and previs, then the cameras are locked. We do that with our in-house shows, and not just the commercial ones. I think that surprises some producers.”
The business of asset creation, modelling and rendering is very much industry-standard, with Maya, 3DS Max and SoftImage being used to pump out the polygons. But producing that all-important fisheye image limits the choice of renderer.
“At the moment we’re tied to Mental Ray, because that’s the only one with the ray-traced spherical lens shader we need,” said Mowbray. “The other technique is to stitch five cameras together, but we prefer a one-click renderer.”
Most members of the NSC Creative team are generalists, although recent growth has seen them assign more specific roles in areas like lighting, FX and environment. All face the very specific challenges that fulldome brings.
“Just pulling the data off the network drives is insane – it kills everything,” Mowbray reflected. “We’re constantly trying to figure out workarounds for the creative and technical challenges of the format, all while trying to manage an ever-growing team of CG artists. At the same time we’re developing a new cinematic language. We’re effectively growing a whole industry.”
The Language of Fulldome
A debate about cinematic language might sound a bit esoteric for a commercial operation, but under the dome it’s a live issue. Rapid transitions in the 360° environment can be overwhelming, and so a typical fulldome show tends to feature long takes, favours dissolves over cuts, and pays particular attention both to where things are in space, and how fast they’re moving.
“You have to be a lot slower,” said Mowbray. “The rule of thumb is: if it feels right in flat screen, then it’s too fast for fulldome.”
Then there’s bounce. When images are projected on to the light grey curved ceiling of a typical dome environment, reflected light can wash things out. This must be taken into account early in the production process, and shots designed to mitigate its effects.
“For shots where the focus is towards the front of the dome, we darken off everything towards the back,” Bradbury explained. “Nobody notices there’s a big dark section behind them because they’re looking straight ahead. But you can also have shots that are more experiential, where the audience is free to explore the frame. That’s the beauty of fulldome. We don’t normally vignette those experiential shots; instead, we keep them dark overall to reduce the bounce.”
Mowbray added: “The new projectors are getting better, with a good balance of brightness and contrast that mostly overcomes these issues. There are projection technologies out there derived from flight simulator tech that have extra panels to give true black. The ideal would be an OLED dome surface.”
There’s one thing that works particularly well in the dome: making people feel sick!
“In Astronaut, we have a centrifuge scene that first spins you around, then rolls the camera, inducing motion sickness,” said Bradbury. “It’s won several ‘most immersive shot’ awards. It’s a big ‘wow’ moment that leaves a lasting impression.”
Paul Mowbray stressed the need for restraint. “If someone actually feels ill when they’re watching one of our shows, then we’ve failed. It’s a really fine line. We have to use it to the advantage of our narrative, and not just for a cheap trick.”
Finally there’s the whole business of frame rates. Love it or hate it, there’s no denying that HFR is a hot topic. Especially under the dome.
“With The Hobbit, a lot of people think HFR detracts from the story,” Mowbray stated. “But we’re trying literally to transport people to other places. In the dome, that weird high frame rate thing actually works. 60fps is where we’re heading at the moment, in 8K.”
The 10,000-Frame Shot
One of NSC Creative’s recent commissions is Tomorrow Town – an architectural visualisation project for the Schindler Group. The centrepiece of Tomorrow Town is a 10,000-frame shot during which the camera moves through a futuristic cityscape.
“It’s very hard to cheat shots like this,” explained Aaron Bradbury. “With fulldome, everything’s in the shot, all the time. As you move past buildings you can’t progressively lose them behind you. That’s the biggest challenge we have: setting up scenes so that we can move constantly and seamlessly through them.”
Mowbray elaborated: “The normal tricks you do in flat screen work to switch out scenes with a bit of judicious comp work are mostly not viable. We used Mental Ray proxies for the background city, as well as iToo Rail Clone – a parametric modelling tool perfectly suited to this kind of arch-viz work. The biggest killer was the trees. We couldn’t use cards because of the camera move so the geometry got quite dense. The total poly count was about 50 million for the big ‘Spiral Town’ model.”
The lengthy shot contains live action of actors walking around a futuristic concourse. Actors were shot at 4K against greenscreen with locked off cameras, with the footage mapped on to cards in the 3D scenes.
“We were pushing the limits of the technique,” said Mowbray. “Parallax issues occurred if we pushed the angle of the camera too much. But we mixed in lots of CG doubles and controlled the area of interest in the dome, so it worked pretty well.”
It’s tempting to think of dome as a closed environment – a specialist arena from which a show producer might never emerge. Not so. Emerging VR technologies like the Oculus Rift mean the immersive 360° experience is no longer trapped under the dome.
“You can’t have a dome at home, but now there’s an opportunity for anyone to have an amazing immersive experience,” said Mowbray. “The Oculus Rift doesn’t deliver the collective group experience, but we are looking at it as a secondary opportunity to monetise our shows and expertise.”
The Oculus Rift is also great for the production process. For the first time, it’s possible for artists to preview content at their workstations in immersive stereo.
“Up to now we’ve had to use a five-camera rig – front camera, two side cameras, a back camera and an up camera,” Bradbury explained. “With that we can get a good feeling of what’s going on, but it’s never the same. We still have to render it out, go down and watch it in the dome.”
Mowbray added: “The Holy Grail for us has always been having a dome attached to your workstation, so you’re working directly in the dome space. The Oculus Rift has the potential to realise that dream. For us, it’s a game changer.”
But is a show designed for the Oculus Rift the same as a dome show? Or is it something new? If fulldome is different to cinema, is VR different again?
“The experience is not quite the same,” Bradbury observes. “In the dome, it’s a bit like you’re in a craft, being flown to different places. With the Oculus, it’s more dreamlike. We’ve done tests with it in 360° stereoscopic, and you really are floating in the world. But it’s weird in that you’re looking around, but your body isn’t there. I can already see the differences in making something for the Oculus compared to the dome. And that’s interesting.”
“I think this is an opportunity for real-time filmmaking to come of age,” Mowbray concluded. “There’s a convergence point that hasn’t quite happened yet where gaming, interactive storytelling, immersion and virtual presence collide and create something new. I think that’s why so many people are excited about the Oculus. It really could be a whole new way of telling stories. I can see a point where we’re making dome films in real-time, and we’re not pre-rendering anything.
“At the end of the day we’re technologists only because we have to be. We just want to create amazing experiences using cool stuff. The more great dome experiences people have, the more potential there is for work and the more viable it becomes as an art form. If there was a dome in every multiplex, a dome in every art gallery in every city, that would change everything.”
The Dome at Home
One member of the NSC Creative team has already managed to transport the dome experience out of the science centre and into the wider world. Specifically, his living room.
“I built my own 1.6m geodesic dome,” Aaron Bradbury confessed. “I get a cushion and crawl in on my back. I use it to test the work I do at home, and then I go back into the bedroom and work on it some more.”
While Bradbury is enthusiastic about his construction, his wife is not so sure.
“Sometimes she brings friends around hoping they’ll have a negative reaction and she can make me get rid of it. Instead, everyone loves it! But now the Oculus has come along, I’ve started to think that maybe I can throw all that cardboard away.”
Special thanks to Ruth Coalson