House of the Dragon: How virtual production is helping actors say goodbye to green screens

Based on George R.R. Martin's book "Fire and Blood," "House of the Dragon" is set roughly 200 years before the events of Game of Thrones. A much-anticipated prequel, the series focuses on the the House Targaryen (the House of the Dragon), the family from which Game of Thrones' Daenerys Targaryen descends. It follows the family (and their accompanying dragons) through a civil war of succession between Princess Rhaenyra Targaryen and her half-brother Aegon II.

The show has already received critical acclaim, the opening episode described as "gorgeous, opulent television" by the Guardian. Since the airing of the first episode, House of Dragon has released content on Twitter giving fans a behind-the-scenes look at how the episodes are created.

The first of these "behind-the-scenes" tours focused on the tournament incident in the first episode and discussed some of the challenges around creating the physical set—including its eye-watering cost. Yet, some of "House of the Dragon" is filmed without much of a physical set at all.

Instead, the series is filmed, in part, within a "volume" or "virtual set," using a relatively new film-making tool and technique known as virtual production.

A new way to shoot

In broad terms, virtual production is a way of making film and television which harnesses computer-generated content that allows real-time visualization and control of the digital environment in which you are shooting, for example by projecting them on a wall of LED screens. Importantly, virtual environments and special effects are typically captured on set, within the camera in real-time, rather than being added in post-production.

Virtual production draws on a range of technologies including real-time game engine technology (such as Epic's Unreal engine). A game engine is a software development program which was originally developed to create video games but which is now used to control virtual sets and environments in virtual production.

Virtual production also includes systems such as motion capture and camera tracking. These are important for ensuring that the imagery displayed on the LED volumes reacts appropriately to the movement of the physical camera, calculating the correct relative position of the virtual camera and therefore the angle of the object being displayed on the LED wall.

Virtual production has recently been used on projects such as Netflix's new upcoming mystery-horror series "1899" and Disney's "Star Wars" series "The Mandalorian" . The creators of "House of the Dragon" were the first to use the new virtual production stage at Warner Brothers Studios in Leavesden in the U.K.

This LED volume stage is one of the biggest in the world, comprising more than 2,000 LED screens and 92 motion capture cameras. The scale of this LED volume means that production cast and crew are immersed in a moving CGI environment in real time, and are able to respond to the virtual environment as it changes before their eyes.

One of the key uses of virtual production in the show so far has come in episode two with Princess Rhaenyra's flight to Dragonstone (the castle which forms the ancestral seat of House Targaryen), where her Uncle Daemon has fled with the dragon egg she had chosen for her late brother Baelon.

In the Dragonstone bridge scene, where Otto and Deamon have a stand off, the actors are real and are standing on a physical unadorned bridge set. However, Dragonstone itself and the rocky environment surrounding the bridge is entirely virtual, having been designed in pre-production and projected onto the LED walls via the game engine during production.

The arrival of Rhaenyra on her dragon Syrax is also entirely virtual, up until the point you see Rhaenyra up-close, dismounting from Syrax and walking through the middle of the king's guards.

At this point all of the actors and the bridge in the foreground are once again real but the background remains entirely virtual. The lighting of the entirety of this scene is also likely to come directly from the LED volume, which can be adjusted to recreate the light at different times of day.

The benefits of virtual production

For a show like "House of the Dragon," virtual production also offers the opportunity to create fantastical sets, which do not exist or would be difficult or costly to film within real locations. This includes the site of San Juan de Gaztelugatxe in Spain, on which the Dragonstone bridge in the Game of Thrones series is based.

As one of the show's stars, Rhys Ifans, explained, virtual production was used to shoot the bridge scenes at Dragonstone as it offered the ability to shoot dawn or dusk scenes repeatedly. As opposed to filming "on location" where you are limited by hours of daylight, virtual production enables the crew to shoot with any lighting and atmosphere they require for as long as needed. This is because the LED volume itself provides the majority of the dynamic lighting needed for a scene.

Although this bridge proved logistically impractical to shoot on for "House of the Dragon," the virtual production team were able to recreate the bridge virtually through LIDAR (laser) scans of it, and then shooting with it virtually within the LED volume. This ability to shoot scenes with the bridge virtually may also have been particularly beneficial given the travel restrictions and social distancing rules prompted by the pandemic.

The real-time element of these fictional sets being projected onto the LED walls also presents another benefit of virtual production: the ability for the actors to be able to see and react to scenes in real time. This is something which was not previously available with technologies such as green screens where scenes and special effects had to be added in post-production. We've all seen behind-the-scenes footage of actors talking to a ball held by a man in a full green morph suit.

During the pandemic, virtual production also enabled film and TV production to continue despite social distancing and travel restriction measures, as it removed the need to film on-location. As we move into a post-pandemic era of filmmaking (and further into the "House of the Dragon" series), it will be interesting to see what the "alternative tool" of virtual production means for the future of film and TV.