Building the World of Tomorrow

December 7th, 2019 3D Artist, Film, Film Features

How striking visuals and an impossible deadline created a dieselpunk cult classic, and laid the foundation for the modern VFX industry

How far back can we trace current trends in Hollywood? Digital filmmaking in the late 1990s? Talkies in 1927? Celluloid, Georges Méliès and the Lumiere brothers in the 19th century?

How about an inexpensive action adventure yarn from 2004 that did okay with critics and so-so at the box office? Read on and learn how the way movies use VFX was cemented – possibly generated – with Sky Captain and the World of Tomorrow.

These days only a tiny portion of a real set is built in any self respecting blockbuster (a single room, a rocky outcrop, a city street), almost everything else in the frame to be animated later in the render farms the world over.

In fact it’s now gone beyond actors emoting to stage hands holding tennis balls. Disney’s live action remake of The Lion King harks more back to the 1994 original in that it’s completely animated – not a single real landscape, sky or animal was photographed.

Once upon a time they used what are called travelling mattes, where black draping would allow for double exposing a print during processing, letting the director expose a background and foreground together on new celluloid. 1903’s The Great Train Robbery used a matte to expose only the train windows in certain scenes, letting director George Albert Smith add backgrounds where the windows appeared.

Blue screens were first used in the 1930s for wipes and transitions between scenes, the first true on-screen blue screen effect appearing in 1940’s The Thief of Bagdad. As for why green screens were finally adopted instead, there’s a (perhaps-apocryphal) story that TV weather presenters tended to wear blue suits, harder to isolate against a blue chroma key and prompting a move to a less common colour.

Digital-era green screen

By the late 90s blue and green screen technology was pretty widespread. However well received (or not), the visuals in George Lucas’ Star Wars prequels were a quantum leap forward which included one of the first completely digital characters in Jar Jar Binks.

But in an age where Marvel superheroes leap between planets to battle intergalactic despots while cities crumble around them, almost every background and much of the action shots made or augmented by CGI, Sky Captain and the World of Tomorrow might be their unsung progenitor.

Written and directed by Kerry Conran, it was shot on sound stages at Elstree studios using minimal sets and props, every backdrop the work of VFX artists across the world.

A retro futurist adventure, it’s about the titular flying ace hero Joe (Jude Law), his feisty reporter love interest Polly (Gwyneth Paltrow) and a grand conspiracy by a fascist despot with terrifying warfighting technology. There’s a soft focus, Hollywood Golden Era sheen, the energy and aesthetic of World War II newsreels, and it was something audiences had never seen before.

Originally a short film made by Conran and his production designer brother Kevin, Sky Captain and the World of Tomorrow came together using technologies and production methodologies that have become the default standard today.

And if you’re in any doubt about Conran’s foresight, the film also showed an airborne aircraft carrier (the Flying Fortress) eight years before Avengers (2012), and would cast two future Albus Dumbledores in Law and Michael Gambon as Polly’s editor.

Why CGI

Conran attended Calarts, an animation-heavy school famous as the breeding ground for Pixar, and didn’t see Sky Captain as the future. He was actually planning on animation bringing his adventure tale to life with stylistic nods to early Disney and German expressionism.

Early tests convinced producer Jon Avnet to come on board, initial financing came from producer Aurelio De Laurentiis and Avnet and Conran recruited Law, whose passion for the concept was critical in attracting other cast and crew. The Conrans established a CGI facility in the LA Valley neighbourhood of Van Nuys, which would create the backgrounds and action plates while Conran shot live action in England.

With principal photography in the can, Conran returned to Van Nuys to establish the VFX workflow. He and his team worked up about 70 percent of the movie, including around half an hour of finished footage ready to show to a studio. The one that offered Avnet and Laurentiis a deal (Paramount) imposed a punishing release date, so after working steadily and producing scenes, Conran and his team suddenly found themselves assigned with additional VFX producers and supervisors, scenes they had no chance of finishing in time parcelled out to external vendors.

One was VFX producer Scott Anderson, who says the production from then on a ‘sprint’. “It wasn’t a normal approach for me, but Kerry had really come up with something fresh I wanted to support,” he remembers.

Deploying tech

“Accessible compositing programs like Adobe AfterEffects were only just arriving,” Conran remembers now, “so I thought there might be a way of applying the techniques of animation to live action. The first couple of tests were simple stuff, just taking old photographs and putting people into them. These beautiful vintage photographs had a real hue and sense of scope and scale I couldn’t possibly achieve [from scratch]. That’s when I first thought it could work.”

Conran says nobody set out to reinvent the wheel, that live action against completely animated backdrops was just a means to an end to get his ideas on screen with the tools available. The original plan was to make Sky Captain black and white with only photographs as the backgrounds, but when Paramount’s release date landed with a deafening thud he says all that went out the window, the whole production pivoting to CG.

Among the innovations that emerged from Sky Captain’s process was something that’s very much standard today – Conran had to previz the whole film so they knew what every shot would look like in advance.

What’s more, final shots were actually built out from the previz models. “It was like an animatic but it was more than that,” Conran says. “Previz today is mostly thrown away and effects shops rebuild it all from scratch with higher res models. We kept everything, working on top of the previz for the entire movie once the animation was blocked in.”

If he made Sky Captain today Conran says he’d go further, building everything in a game engine, never really in previz because you’re always working with finished quality.

“That to me feels like the way things are headed. You can have something that’s real time on a desktop, manage the entire shot and it’s always locked tight, so you’re effectively editing the film as you’re creating it. You can suddenly stop and reach into the frame to move a building or change the sun, hit play and be back live.”

Building the future

The compressed schedule meant late nights and pulling out of hair for everyone, but was there any sense either in Elstree or Van Nuys that they were laying down the bedrock of the methods the industry would later make routine?

“I think so,” says Anderson. “Peter Jackson and George Lucas had toyed with it in smaller sections of larger projects, but Kerry jumped in wholeheartedly. Sky Captain was the first to bet the whole project on it.”

“It inspired a lot of people get really wild and create a world nobody had seen, so [the film] was definitely part of what pushed the industry this way,” adds Joyce Cox, the senior VFX producer assigned to the production alongside Anderson.

Conran and his team originally intended to do all the compositing work at their Van Nuys home base, but the bond company that appointed Cox, Anderson and others to oversee production insisted work be sent out to make the date.

That means another of the practices Sky Captain established – sending sequence assignments all over the world – is now de rigueur, with vendors from South Australia, California and Quebec all weighing in.

Conran says nobody in England knew what to think about actors performing opposite nothing at all, but the first shot put together by digital effects supervisor Steve Yamamoto (part of the original crew) ‘completely transformed’ opinions. “It was unfinished and crude, but it was amazing,” he says. “Actors interacting with the virtual sets and props was a revelation for them.”

A VFX coordinator at Adelaide’s Rising Sun Pictures at the time (now VFX producer), Ian Cope thinks that although the industry was heading in this direction anyway, Sky Captain was the extreme – and bold – expression of such methods.

“When I saw it at one of the early screenings it was unlike anything I’d ever seen,” he says. “It allowed us to see what a kind of giant balloon of visual effects around storytelling looked like. We took that tool to the nth degree, which made us realise it needed to be part of the stock standard tools we use to make movies just like costuming, special effects and even acting.”

Down the pipe

Sending scenes and shots to companies around the world was an unproven method back then, and where Conran says some vendors like ILM and Rising Sun shined, others didn’t. What’s more, the style of shots was so specific his team had to provide them with a few select frames in a given sequence as a reference, digital effects supervisor Yamamoto adding the software presets in a kind of ‘shot kit’, not unlike from a how-to tutorial.

From there, co-ordinating what everyone was working on was gargantuan – the task Cox, Anderson and veteran VFX producer Brooke Breton were bought on to wrangle. “I spent an ungodly number of hours in a small room looking at footage and shots coming in,” Anderson says. “The logistics of juggling the creatives, the budgets and the work was amazing – not to mention the databases and shot tracking that had to be built for it.”

It was so intensive Anderson couldn’t even leave the building for long stretches, which made review of footage over the internet a must. It was his first time using CineSync, Rising Sun’s proprietary share and review system to view and discuss shots online in real time and another model that’s standard today. “It really is how everything’s done today but back then people thought we were crazy.”

So one of the biggest questions is; would anyone in 2004 have recognised the methods we have if they’d had a crystal ball? To Anderson things are the same – just much better. “We struggled trying to bring prosumer tools to feature level,” he remembers. “AfterEffects was relatively unproven on that scope and scale, the HD cameras had very limited dynamic range and resolution. It really was the technical equivalent of balsa wood and duct tape to create this giant world!”

Having said that, Ian Cope says expanded toolsets means expanded possibilities, resulting in an arms race between what we can do and what directors want. “It’s easier today but now we’re looking at 4K, which brings another set of challenges. We approach it in a similar way and better tools make it look better, but the shots get bigger and they want them quicker, so it never actually gets any easier.”

Conran’s intention for Sky Captain was actually for it to be a ‘handmade’ film, an ethos that was on track when they had enough time to finish the film their way. But Paramount’s release date made an artisanal approach impossible. Even though he was seeing (and creating) the future of wrangling multiple suppliers, it was tough giving up the chance to shape so much of it directly.

“It became a rush, and it usually is, it’s just that when you’re talking about a Marvel movie they have the resources and money to throw at it, so the quality they get is obviously the best,” he says. “We weren’t as lucky, a lot of the stuff we were getting back or doing ourselves was so rushed towards the end it was difficult to maintain the style and the quality. Some stuff is painful to look at now because it was crude even by our standards back then.”

Today, when handing work off is the norm, Conran says extensive previz and solid production design are what guarantees a good result. In fact today he thinks directors expect to get footage back that’s better than they imagined because many VFX artists are better filmmakers than the people making the films – he calls them ‘co-directors’.

Impact

But after all’s said and done the only really measure of success is how many behinds are in seats, and history remembers Sky Captain and the World of Tomorrow as a box office failure (although it’s not that simple – see boxout Hollywood accounting). Why didn’t it catch on? Might it simply have been ahead of its time like the Apple Newton, Laserdisc or Live Journal?

Conran set out to make what he calls a US$3m art film made in an unconventional way, imagining a story about how a protégé of Frank Capra’s snuck off during shooting the Why We Fight movies of World War II to make this film but never finished it.

Unfortunately, as history shows, it was turned into something that was meant to compete with a Mission; Impossible and an X-Men film, a modern milieu his original aesthetic might not have ultimately suited. “I don’t think kids back then or today know or care what serials are,” he says. “That sensibility is quite stilted for them.”

Scott Anderson wonders if the concept being so ahead of its time got a bit too much of the attention. “We were definitely at the bleeding edge creatively but you need the story to grab people,” he says. “The look might have made the audience think it was a gimmick, maybe the pitch should have been more about the story and let the look be part of the tale.”

But we might never know what impact Sky Captain and the World of Tomorrow really had. It’s tempting to imagine it showcasing VFX to Hollywood like never before, but maybe it simply cemented the idea among artists and creators that they could really do anything.

“The best indication of [the film’s] power was from a writer friend,” says Anderson, “who said ‘I used to self sensor what I wrote based on what I thought was possible, but after seeing Sky Captain I’m going to write anything I want and let you guys figure out how to make it.”

Anatomy of a shot

How shooting in the UK and VFX in the US were matched up for seamless compositing

First was the application of a grid system on the sound stage, a little like a giant game of Battleship and a precursor to the way James Cameron would shoot Avatar and Steven Spielberg would shoot Ready Player One years later.

A matching virtual representation of the sound stage at Elstree was built virtually in the computers back at the VFX headquarters at Van Nuys.

That meant the virtual and real sets were one to one with a perfectly matching reference, so when the team did previz such as a camera move or interaction with a prop, they could measure the allowances needed in the virtual, as-yet unbuilt sets, for example where an actor had to enter through a door

The camera crew were given a printed out shot list every day, and when the time came they’d know that a particular shot began – for example – at G7 and ended at F12 and that the camera tilted ‘x’ degrees, knowing every mark to hit as they set up. It resulted in the real and CG camera moves matching perfectly, ready to be composited together later and giving the actors room to improvise within the guidelines.

Back to the future

While actors today are used to performing against blank walls or spots on sticks, there’s actually a precedent only actors understand. Sky Captain and the World of Tomorrow writer/director Kerry Conran says stars Jude Law, Gwyneth Paltrow and Giovanni Ribisi took to it with aplomb.

“They were really incredible,” he says, “the nearest comparison we made at the time was kind of like theatre – on stage actors aren’t used to having every [prop], they pretend they’re in an environment that may not be represented.”

Ribisi was actually so taken with the process he later enrolled in Maya classes, but the production only had Angelina Jolie (who plays no-nonsense hovering aircraft carrier captain Frankie) for two days, so Conran says she didn’t have as much time to internalise how it was going to work.

“At one point she was more comfortable entering the scene from a different direction than we’d set-up, and it was difficult to explain that if she came in from that side there was going to be a wall there.”

Hollywood accounting

Author Winston Groom claimed he never saw a cent from Best Picture winner Forrest Gump (1994). Based on his best selling novel, his deal was for after-profits share but the studio claimed it never really made it into the black – despite making US$678m off a $55m budget.

Welcome to Hollywood accounting 101, an art more arcane and mysterious than all the acting, screenwriting and directing put together.

Sky Captain and the World of Tomorrow’s budget was US$70m and the global box office take was just short of $58m. That’s a failure in everyday economics. Except that director Kerry Conran was working to an initial budget of around $3m. When the release date and change in methodology shifted things to all-VFX backgrounds that changed to $12m, eventually topping out at $18m. The film sold to Paramount, who asked for a simply impossible deadline of May 2004.

A bond company took over, splurging on outside effects houses and extra staff to manage it all in the hopes of making the new release date of September 2004, the $70m figure published to position Sky Captain in a certain market but way above the film’s true cost.

Look closer

Obviously a classic movie geek at heart, Conran included some very cool Easter eggs. During the scene of Sky Captain descending into New York to battle Dr Totenkopf’s fleet of giant robots the Empire State Building is visible in the background. Look closely and you can see the tiny figure of a 40 foot gorilla on the uppermost spire. Later, during the undersea dogfight between Frankie’s (Angelina Jolie) forces and Totenkopf’s mechanical terrors, the wreck of the SS Venture is visible on the seabed.


Full client and publication list:

  • 3D Artist
  • APC
  • AskMen.com
  • Auscam
  • Australian Creative
  • Australian Macworld
  • Australian Way (Qantas)
  • Big Issue
  • Black Velvet Seductions
  • Black+White
  • Bookseller & Publisher
  • Box Magazine
  • Brain World
  • Business News
  • Business NSW
  • Campaign Brief
  • Capture
  • CHUD.com
  • Cleo
  • Cosmos
  • Cream
  • Curve
  • Daily Telegraph
  • Dark Horizons
  • Dazed and Confused
  • Desktop
  • DG
  • Digital Media
  • Disney Magazine
  • DNA Magazine
  • Empire
  • Empty Magazine
  • Famous Monsters of Filmland
  • Fast Thinking
  • FHM UK
  • Film Stories
  • Filmink
  • Follow Gentlemen
  • Geek Magazine
  • Good Reading
  • Good Weekend
  • GQ
  • How It Works
  • Hydrapinion
  • Inside Film
  • Internet.au
  • Loaded
  • M2 Magazine
  • Marie Claire Australia
  • Marketing
  • Maxim Australia
  • Men's Style
  • Metro
  • Moviehole
  • MSN
  • Nine To Five
  • Paranormal
  • PC Authority
  • PC Powerplay
  • PC Update
  • PC User
  • PC World
  • Penthouse
  • People
  • Pixelmag
  • Popular Science
  • Post Magazine
  • Ralph
  • Reader's Digest
  • ScienceNetwork WA
  • SciFiNow
  • Scoop
  • Scoop Traveller
  • Seaside Observer
  • SFX
  • Sydney Morning Herald
  • The Australian
  • The Retiree
  • The Sun Herald
  • The West Australian
  • thevine.com.au
  • TimeOut
  • Total Film
  • Video Camera
  • Video&Filmmaker
  • Writing Magazine
  • Xpress
  • Zoo