Stagecraft Technology in VFX | Integrating Real and Virtual Scenes

Virtual production has changed the way filmmakers plan and capture visual effects. One of the best-known examples is ILM StageCraft, a virtual production toolset developed by Industrial Light & Magic (ILM) that combines LED display technology, real-time rendering, camera tracking, and on-set workflows. Rather than treating every digital environment as a background to be added after filming, StageCraft can bring selected virtual environments onto the set while the cameras are rolling.

What Is StageCraft?

StageCraft is ILM’s virtual production toolset for planning, creating, and filming scenes that combine physical sets with computer-generated environments. A key part of many StageCraft productions is an LED Volume: a stage surrounded by large LED displays, often with an LED ceiling, on which digital environments are shown during photography.

The terms are related, but they are not interchangeable. Virtual production is the broader production approach; an LED Volume is a physical display environment that can be used for virtual production; and StageCraft is ILM’s particular suite of tools, technology, and production services. Other studios can build LED Volumes without using ILM StageCraft.

StageCraft became widely known through The Mandalorian. ILM and its partners developed the workflow for the series after preliminary testing in 2018. The technology was publicly unveiled in February 2020, after the show’s first season had already premiered in November 2019.

How Does StageCraft Work?

A StageCraft production brings together several technical and creative systems. The exact setup varies by project, but an LED Volume workflow typically includes the following stages.

1. Building the virtual environment: Artists create or adapt digital locations, sets, skies, and other background elements. The environment may be based on original CG, scanned locations, photographic references, or a combination of techniques.
2. Preparing the scene for the stage: The virtual art department and production teams coordinate the digital environment with the physical set, props, lighting plan, and intended camera positions. Previsualization and virtual scouting can help the crew plan shots before filming.
3. Displaying the environment in Realtime: High-resolution LED panels display the digital scene around the performers. In the first season of The Mandalorian, Unreal Engine was used for Realtime rendering. Later StageCraft productions have also used ILM’s proprietary Helios rendering system, or other options within the StageCraft toolset, depending on the production.
4. Tracking the camera: A tracking system measures the camera’s position and orientation. The rendered view can then update to match the camera’s perspective, helping nearby and distant elements shift correctly as the camera moves. This perspective change is known as parallax.
5. Capturing the scene in-camera: The LED environment appears in the camera image during filming. It can also provide interactive light and reflections on actors, costumes, props, and practical set pieces, depending on the content and lighting setup.
6. Finishing the shot: The footage is reviewed and completed through the production’s usual post-production pipeline. Some shots may need little additional work, while others still require compositing, cleanup, CG elements, or other visual effects.

Why Camera Tracking and Perspective Matter

Simply displaying a beautiful image on an LED wall is not enough to create a convincing environment. If the background does not respond correctly to camera movement, the scene can look like a flat backdrop. StageCraft workflows use camera tracking and real-time rendering to align the displayed perspective with the camera’s view.

When the camera moves, the virtual viewpoint can change so that foreground and background elements appear to move at different rates, as they would in a real location. The production team must also coordinate lens data, camera calibration, display geometry, and the virtual scene. The level of integration depends on the equipment and workflow used for a particular production.

Advantages of StageCraft and LED Volume Production

• More creative decisions on set: Directors, cinematographers, production designers, and VFX teams can evaluate the environment and framing while filming, rather than relying entirely on a later composite.
• Interactive lighting and reflections: LED imagery can contribute color, direction, and reflections to the photographed scene. This can be especially useful for reflective costumes, vehicles, glass, and polished props, although conventional lighting is still often required.
• Reduced location travel: Some scenes can be filmed on a controlled stage instead of moving the entire production to a remote location. This may save time and simplify logistics when the setup suits the scene.
• Useful visual context for performers: Actors can see the environment surrounding them, which may help with eyelines, atmosphere, and performance compared with an entirely blank chroma-key background.
• Greater control over conditions: A studio environment can make it easier to manage weather, access, continuity, and shooting schedules.
• Potentially less downstream compositing for suitable shots: When the environment is captured successfully in-camera, some background replacement work may be reduced. This does not eliminate post-production VFX from the project.

Limitations and Technical Challenges

StageCraft is a powerful production option, but it is not the best solution for every shot. Its results depend on the creative brief, stage design, display specifications, camera package, virtual content, and available budget.

• High setup and operating costs: A professional LED Volume requires suitable display panels, processing and synchronization systems, camera tracking, powerful rendering hardware, a properly equipped stage, and a specialist crew. The cost of creating and preparing the virtual environments must also be considered.
• Display and camera constraints: Pixel pitch, brightness, refresh rate, scan behavior, color response, viewing angles, and the camera’s shutter settings can affect the captured image. Moiré patterns, flicker, or unwanted artifacts may occur if the setup is not carefully tested.
• Reflections and lighting control: LED walls can provide useful interactive light, but they do not automatically produce the correct illumination for every shot. Their brightness, color, and position must be balanced with dedicated film lighting and practical effects.
• Limits on camera movement and framing: The physical dimensions of the Volume, display edges, perspective requirements, and the content available outside the camera frame can restrict certain wide shots or camera moves. Larger or custom stages can address some—but not all—of these constraints.
• Virtual content must be production-ready: Environments need suitable detail, correct scale, believable lighting, and reliable performance at the required frame rate and resolution. Revisions may involve substantial work by the virtual art department and technical teams.
• Not every change is easy after filming: Because the environment is photographed in-camera, some creative decisions are effectively baked into the footage. A background can still be replaced or altered in post, but doing so may require additional VFX work.
• It does not remove the need for traditional VFX: Green screen, location photography, practical sets, miniatures, motion control, and conventional CG remain valuable. A production may combine several methods from shot to shot.

Where Is StageCraft Most Useful?

StageCraft is particularly useful when the production benefits from a controlled stage, visible digital surroundings, interactive lighting, and repeatable camera setups. It can support interiors, vehicle scenes, science-fiction environments, distant landscapes, and locations that would be difficult or expensive to access.

• Studio-based scenes: Controlled interiors, corridors, rooms, and partial sets that can be extended digitally.
• Vehicle and cockpit scenes: Digital surroundings can move outside windows while contributing reflections and light to the actors and set.
• Remote or hazardous locations: Selected environments can be represented on stage when travel, weather, safety, or logistics make location work impractical.
• Scenes requiring repeatable conditions: A carefully prepared digital environment can help maintain continuity across multiple takes, provided the stage and content remain consistent.

A StageCraft stage is not simply a room with screens. It requires space for the physical set, camera and lighting equipment, crew movement, tracking systems, and technical infrastructure. Some productions use permanent volumes; others use custom or temporary stages designed for a particular project.

StageCraft vs. Green Screen: Which Is Better?

Neither method is universally better. The right choice depends on what the shot needs to achieve.

• LED Volume / StageCraft: Often advantageous when the visible background, perspective changes, reflections, and interactive light are important during photography.
• Green screen or blue screen: Often advantageous when the background must remain highly flexible in post-production, when the environment extends far beyond a practical stage, or when the required imagery is not suitable for display on LED panels.
• A hybrid workflow: A production can combine LED backgrounds, green screen, location plates, practical sets, and CG. Even a scene filmed in an LED Volume may use traditional VFX to refine or replace parts of the image.

The most effective workflow is chosen shot by shot. LED Volume production does not make chroma key obsolete; it adds another option to the visual effects toolkit.

Notable Productions That Have Used ILM StageCraft

StageCraft has been used across television and feature-film productions. The examples below are included because ILM or an official industry source identifies StageCraft involvement; the technology’s use does not mean every shot in a production was filmed inside an LED Volume.

• The Mandalorian (2019–2023): The series brought StageCraft to widespread attention. ILM reported that more than 50 percent of the first season was filmed using the new methodology. Its first-season workflow used Unreal Engine for Realtime rendering of more than 60 photoreal virtual environments. For Season 2, ILM expanded the Volume and introduced its Helios real-time cinema engine for the imagery displayed on the LED stage.
• The Book of Boba Fett (2021–2022): ILM’s official credits identify a dedicated StageCraft team on the series.
• Obi-Wan Kenobi (2022): ILM’s credits list a StageCraft Volume Supervisor and a virtual production “Brain Bar” crew.
• The Old Man (2022): ILM included the series among the productions that used the StageCraft tool suite.
• Thor: Love and Thunder (2022): ILM built a bespoke StageCraft LED Volume at Fox Studios Australia for the production, and its official credits list StageCraft roles.
• The Batman (2022): ILM’s official credits identify virtual production by StageCraft.
• The Midnight Sky (2020): ILM identified the George Clooney film as a project served by a custom “pop-up” StageCraft Volume.

These examples should not be confused with productions that used virtual environments, computer-generated imagery, or other virtual production techniques without confirmed use of ILM StageCraft. For that reason, this article does not list The Lion King or Avatar: The Way of Water as confirmed StageCraft productions.

John Knoll and the StageCraft Team

John Knoll is a veteran visual effects supervisor and a senior creative figure at ILM. He is credited in connection with major ILM productions, including The Mandalorian and The Book of Boba Fett. His involvement should not be taken to mean that he invented StageCraft single-handedly: the system emerged from collaboration among ILM artists and engineers, filmmakers, technology partners, and production specialists.

StageCraft illustrates how modern VFX increasingly depends on collaboration between traditional filmmaking departments and real-time technology teams. Virtual art departments, environment artists, camera-tracking specialists, LED technicians, rendering engineers, cinematographers, production designers, and VFX supervisors all contribute to making the workflow successful.

Frequently Asked Questions About StageCraft

Is StageCraft the same as Unreal Engine?

No. StageCraft is ILM’s virtual production toolset. Unreal Engine is a real-time 3D engine developed by Epic Games and was used for rendering environments in the first season of The Mandalorian. ILM later introduced its own Helios rendering system for StageCraft, and ILM has described flexibility to use Helios or Unreal Engine within its toolset.

Does StageCraft eliminate post-production?

No. It can capture some visual effects directly in-camera and reduce certain compositing tasks, but productions may still need cleanup, CG characters and objects, set extensions, compositing, color work, and other post-production processes.

Can any studio use StageCraft?

ILM has expanded StageCraft services beyond the original The Mandalorian production and has built permanent and custom volumes for different clients and projects. Access depends on the production’s needs, budget, schedule, location, and the services available from ILM or other virtual production providers. An LED Volume built by another company is not automatically an ILM StageCraft stage.

Is an LED Volume always better than a green screen?

No. LED Volumes can be especially effective when in-camera backgrounds, reflections, and interactive lighting matter. Green screen can be more practical when the environment needs extensive changes in post-production or when the scene does not fit the physical and technical limits of the LED stage. Many productions combine both methods.

Conclusion

ILM StageCraft is more than a wall of LED screens: it is an integrated virtual production workflow that connects digital environments, camera tracking, real-time rendering, display technology, and on-set filmmaking. Its main strength is that it lets filmmakers evaluate and capture selected virtual environments during photography, with potential benefits for lighting, reflections, creative collaboration, and production logistics.

However, StageCraft is not a universal replacement for location shooting, green screen, practical effects, or conventional post-production. The best results come from choosing the right method for each shot and bringing the creative and technical teams together early in the process.

References & Further Reading

ILM — StageCraft Technology Created for The Mandalorian ILM — The Mandalorian (Season 1) ILM — The Mandalorian (Season 2) ILM — StageCraft Receives an Engineering Emmy Award ILM — The Book of Boba Fett Credits ILM — Obi-Wan Kenobi Credits ILM — Thor: Love and Thunder ILM — The Batman Credits ILM — Expansion of StageCraft LED Virtual Production

Sadjad Jahangiri

VFX Artist & Instructor

Sadjad Jahangiri is a VFX, animation, and digital arts specialist and educator. His areas of expertise include 3D simulation, compositing, VFX pipeline design and management, 2D animation, design, and motion graphics.