As Gaussian Splats continue to gain traction across virtual production, VFX, and real-time filmmaking workflows, many producers and creatives are asking the same question: how do these emerging technologies fit into production today?
To explore the opportunities, challenges, and practical considerations of working with splats, we spoke with Kathryn Brillhart, Virtual Production Supervisor and Cinematographer. Drawing from her experience across volumetric capture, visual effects, cinematography, and emerging AI-driven workflows, Kathryn shares her perspective on where Gaussian Splats deliver the most value, how productions can successfully integrate them into their pipelines, and the creative possibilities they unlock for filmmakers.
From capture planning and department alignment to common misconceptions and future applications, here are Kathryn’s thoughts on the evolving role of Gaussian Splats in modern production.
Having worked across volumetric capture, visual effects, and cinematography, I see Gaussian splats already being used in a few clear ways—bringing real-world environments onto LED volumes, supporting virtual scouting, and accelerating environment creation for VFX without traditional modeling. They are also starting to extend into hybrid AI workflows, where captured data can be refined, reinterpreted, or integrated with generative processes. What’s more interesting to me is how well they preserve both spatial structure and light. When you’re designing a shot—whether it’s a dolly, a handheld move, or even a subtle shift in framing—the way light and reflections change is what makes the space feel real. If that behavior isn’t captured correctly, you end up trying to rebuild it later, and it’s very difficult to match.
The starting point with a producer or creative team is typically: what problem are we trying to solve, and what kind of filmmaking workflow are we designing for? That could mean traditional post workflows like green screen compositing or rotoscoping, real-time approaches like simulcam, or fully in-camera solutions using LED volumes for ICVFX. Each of those implies a different relationship between the camera, the environment, and when the final image is resolved.
In that context, splats are not just a background solution—they are a way to capture and re-photograph a real space as a navigable, three-dimensional dataset. Unlike green screen, where the environment is created or integrated later, or LED workflows where environments are often built or art-directed, splats allow you to record camera movement through an environment that already contains its lighting and spatial complexity. That becomes particularly valuable when subtle parallax, reflections, and natural light response are part of the creative intent.
This is where the distinction between capture methods becomes important. Photogrammetry reconstructs geometry from image correspondence, which means it often struggles with sky, reflections, and transparent surfaces. NeRFs take a different approach by learning a view-dependent radiance field from densely sampled image data—often derived from video—allowing reflections and specular highlights to shift with camera perspective, but they remain computationally heavy. Gaussian splats are a more production-ready representation of radiance field data, which can be packaged into formats like .nvol for use in real-time pipelines—preserving much of the lighting information while being optimized for real-time use in engines like Unreal.
From a production standpoint, that opens up a deeper conversation with producers about how to structure the pipeline—where to rely on capture, where to build, and how to balancetraditional workflows with newer hybrid or AI-assisted approaches depending on the needs of the project.
Following the same logic as when to use splats, the departments that need to be involved are determined by the context in which they’re being captured and used. Any department that overlaps in both capture and downstream usage needs to be part of the conversation early, because decisions made during acquisition directly affect how the data performs later in the filmmaking workflow.
At a minimum, that typically includes the DP, VFX Supervisor, and Virtual Production Supervisor. The DP is critical because splats are fundamentally capturing light behavior—exposure, reflections, and environmental conditions all become part of the dataset. The VFX Supervisor evaluates how that data will integrate with other assets and what level of control or manipulation will be required. The Virtual Production Supervisor ensures the data is usable in real-time contexts, whether that’s Unreal Engine, LED playback, or simulcam workflows.
What’s shifting now is who is actually capturing the data. Traditionally, VFX teams or specialized vendors handled reality capture or volumetric acquisition. Now, with smartphones and accessible tools, producers, DPs, and production designers can capture scans directly. Those lighter captures are often useful for reference—camera blocking, general lighting direction, or spatial understanding—but they typically lack the precision needed to be reused as final assets without additional reconstruction or texturing.
A more robust approach is to treat early captures as part of a broader pipeline. If a scout is successful, the production can return with a VFX Supervisor or VP team to capture higher-quality data that can be processed in multiple ways, whether through high-resolution stills or video-based capture depending on the needs of the reconstruction—photogrammetry, NeRF, or Gaussian splats—maximizing the value of a single dataset. From a cinematography standpoint, this distinction is critical. If a splat is captured during a scout, the lighting from that moment is effectively baked into the dataset. That can be useful for reference, but if the goal is to build a lighting setup in Unreal that responds to physically accurate lights, additional modeling and texturing is still required.
That’s why alignment across departments matters. The way something is captured determines whether it remains reference, becomes a reusable asset, or can support a fully realized technical or creative workflow. When that alignment happens early, it allows teams to design a capture strategy that supports multiple outcomes, rather than limiting the data to a single use.From a production standpoint, that opens up a deeper conversation with producers about how to structure the pipeline—where to rely on lightweight capture, where to invest in higher fidelity acquisition, and how to balance traditional workflows with newer hybrid or AI-assisted approaches depending on the needs of the project.
I tend to think less in terms of what is “safe” or “risky,” and more in terms of what is appropriate for the intended use. Most of the work I do involves experimenting with emerging tools, so the question is really about understanding the strengths and limitations of each approach and applying them intentionally.
Gaussian splats perform well in situations where environmental fidelity is doing most of the work—shots that benefit from subtle parallax, natural light behavior, and the complexity of real-world surfaces. In ICVFX workflows, they can be very effective for controlled camera moves or moments where you want a high degree of photorealism without building a full environment. That tends to include slower moves, lateral shifts, or compositions where the interaction between the subject and environment is primarily visual rather than physical.
Where they become more challenging is when a shot requires a high degree of control—extensive relighting, interaction, or structural modification of the environment. Because splats carry captured, or ‘baked,’ lighting and spatial information, they don’t behave like fully relightable CG assets when you need to art direct or physically manipulate the scene. That’s not necessarily a limitation, but it does define how they’re best used.
In practice, the strongest results often come from hybrid approaches. Rather than relying on a single technique, productions tend to combine methods—using splats where photorealism and speed are critical, and leaning on more traditional workflows where control and flexibility are needed.
So the real answer is less about which shots are “safe” and more about understanding what each technique is designed to do, and building a workflow that plays to those strengths.
Planning for splat capture—and really any reality capture workflow—needs to happen earlier than most productions expect. Whether you’re capturing for photogrammetry, NeRF, or Gaussian splats, those decisions ideally start during location scouting or pre-production, not on the day of the shoot.
At a technical level, the quality of the final dataset is directly tied to how the images are captured. Camera sensor characteristics, lens distortion, camera calibration and pose estimation, and color management all play a role in how accurately the data can be processed, whether by a human artist or an AI-driven pipeline. Consistency is key—if those variables aren’t controlled, the resulting dataset may look usable at a glance but won’t hold up across different applications. Maintaining consistent exposure and dynamic range is also critical, particularly when capturing environments with strong contrast or reflective surfaces.
There’s also an important distinction between still photography and video capture. Traditional photogrammetry workflows are often built around high-resolution still images, sometimes captured in RAW, which preserve maximum detail and color fidelity but require more deliberate coverage. Video-based capture, which is commonly used for NeRFs and Gaussian splats, provides dense, continuous sampling of a space, making it easier to achieve complete coverage quickly and to capture how light and reflections change across viewpoints. That density is critical for reconstructing view-dependent effects, but it comes with tradeoffs—video is typically compressed, which can introduce artifacts, reduce color precision, and affect how cleanly the data can be reconstructed. Factors like motion blur, rolling shutter, and compression can also impact stability if camera movement isn’t controlled. In practice, both approaches have value. Stills tend to provide higher fidelity inputs for precise reconstruction, while video enables faster acquisition and more complete spatial coverage. Many production workflows benefit from a hybrid approach, using video to establish coverage and stills to refine detail, depending on how the dataset will be used downstream.
Scale and measurement are another critical factor. If the goal is to use the data beyond visual reference—whether for virtual production, techvis, or integration into a game engine—having reliable scale information and reference points is essential. Without that, the dataset becomes much harder to reuse in a meaningful way.
There are also practical considerations around how and when the capture happens. If you’re capturing during a scout, you need to allow time not just for documentation, but for intentional acquisition. Anything that isn’t captured—whether due to occlusion, limited access, or incomplete coverage—cannot be reconstructed later, so planning camera movement through the space is critical. Decisions about whether the space should be “clean” or include set dressing will affect how the data can be used later. On a live set, that extends to making sure there is enough time and access to capture the environment properly, especially as the scale of the space increases.
While newer AI-driven tools can assist in cleaning up datasets—removing people, equipment, or other unwanted elements—that work still comes at a cost. It can introduce additional processing time and, in some cases, degrade the integrity of the original capture. It’s often more effective to approach capture with the final use in mind, rather than relying on cleanup later.
Ultimately, these decisions should be discussed as part of the broader production plan. The way a space is captured determines how flexible and reliable that data will be downstream, soaligning on intent early is what allows the capture to support multiple uses rather than limiting it to a single purpose.
Many of the common issues during capture come back to a misalignment between what the data is being used for and how it’s being captured. A key question is whether the capture is intended for reference or for production use. For example, is the lighting setup accurate enough to inform creative planning, or does it need to hold up as part of the final image? That distinction affects everything from how the environment is captured to how much control is needed later.
From a producer’s perspective, the role is less about the technical execution and more about creating the conditions for that execution to succeed. That means making sure the right people are involved early, that there is enough time and access to capture the environment properly, and that the creative intent is clearly understood. It also includes practical considerations like permissions and licensing—ensuring that the data being captured can actually be used in the final product.
Producers don’t always get credit for how creative that role can be. The strongest producers I’ve worked with have a deep understanding of the tools and workflows, and they know how to structure a project in a way that allows those tools to be used effectively. They’re not just managing logistics—they’re shaping the environment in which the work happens.
From my own experience moving between producing and supervising, those perspectives inform each other. Understanding the technical and artistic requirements helps you design a better production environment, and understanding the production constraints helps you make more efficient creative decisions. The most successful outcomes tend to come from that balance—where the ecosystem is designed to support both the creative intent and the practical realities of execution.
Before the shoot day, the most important thing to test is not just the capture itself, but how that data moves through the entire workflow. It’s one thing to successfully acquire images or footage,and another to confirm that the dataset can be processed, scaled, and used in the way the production intends.
That typically means running a small end-to-end test, i.e.capturing a representative environment using the same capture approach—whether stills or video—that will be used in production, processing it through photogrammetry, NeRF, or Gaussian splats, and validating how it performs in its final context, whether that’s Unreal Engine, an LED volume, or a VFX pipeline. This helps identify issues with scale, alignment, color consistency, or performance before they become production problems.
It’s also important to test the capture methodology itself. Camera choice, lens distortion, exposure consistency, and color management should all be validated in advance to ensure the data is predictable and repeatable. Even small inconsistencies at the capture stage can create larger issues once the data is processed.
Testing isn’t limited to captured data either. As these workflows evolve, splats can also be generated from 3D environments built in Unreal Engine or other DCC tools, which opens up a different set of creative and technical possibilities. Those approaches still require validation—particularly around how accurately lighting, scale, and material response translate once converted into a splat representation and reintroduced into the pipeline.
From a production standpoint, this is really about aligning expectations. Testing gives the team a clear understanding of what level of fidelity is achievable, how much flexibility exists in the data, and where additional work may be required. It also helps define whether the chosen approach supports the creative intent or if adjustments need to be made before committing to a full capture.
Ultimately, these tests are what allow the team to move forward with confidence. They turn an experimental process into something that is understood, repeatable, and aligned with the needs of the project.
Many of the common on-set issues come down to misunderstanding what Gaussian splats are actually designed to do. One of the biggest misconceptions is that they’re just another tool for background replacement or set extension. In practice, they function more as a captured, three-dimensional representation of a real environment that can be re-photographed, which is a fundamentally different approach.
Another issue is assuming they behave like traditional CG assets. Because splats preserve captured lighting and spatial information, they don’t offer the same level of control for relightingor manipulation. If that isn’t understood on set, it can lead to mismatched expectations about what can be adjusted later.
There’s also often an assumption that they are an “AI solution” that will automatically resolve challenges. In reality, they are highly dependent on the quality of the capture and the intent behind it. If the data isn’t captured correctly, those limitations carry through the entire workflow.
Most of these issues can be avoided by aligning on how the data will be used before capture begins. Once that intent is clear, the technology tends to perform exactly as expected.
Because I started working with reality and data capture before AI-based processes like Gaussian splats and NeRFs existed, a lot of my learning came from understanding how to capture data in a way that is accurate, consistent, and reusable in a professional pipeline.
A big part of that came from working with traditional photogrammetry. It’s not very forgiving—if you miss coverage, you end up with gaps in your data, and you don’t always realize it until days later once the dataset has been processed. That forced me to think carefully about how I move around a subject or environment, how I plan coverage, and how to ensure I’m capturing complete, usable data from every angle. It also taught me how to balance coverage against image fidelity depending on the capture method.
What’s interesting about splats is that they’re much more forgiving and allow for faster iteration. With current processing speeds and AI-assisted workflows, you can often see results almost immediately, which makes them incredibly useful for planning and rapid experimentation. Smartphone-based captures have also made the technology more accessible, and they can produce surprisingly strong results.
But that speed and accessibility can also hide gaps in understanding. The discipline I learned from photogrammetry—how to plan coverage, how to think spatially about a subject, and how to capture with intention—still applies. In many ways, that’s what allows you to move from a quick scan that looks good to a dataset that can actually be used reliably in production.
So the lesson for me was that the technology may evolve, but the fundamentals of capture don’t change. The more intentional you are about how you acquire the data, the more flexible and valuable it becomes downstream.
What excites me most is how these technologies start to expand the way we think about image-making itself. Gaussian splats are often discussed in the context of capturing the real world, but they don’t have to be limited to that. You can generate splats from game engine environments, from traditionally built CG assets, or even from AI-generated imagery. That opens up a much broader creative space where capture and creation start to overlap.
Once you begin designing shots with that mindset, you’re no longer limited to building a world just to frame it through a 2D image. Instead, you’re working within a three-dimensional space that can be explored, captured, and reinterpreted from multiple perspectives. The final frame becomes one expression of that space, rather than the defining constraint.
For filmmakers, that shift can be significant. It moves the process from composing within a fixed frame to designing environments that support a range of possible viewpoints, camera moves, and lighting conditions. In some cases, that means thinking beyond even traditional 3D environments and considering more complex spatial representations that allow for greater flexibility in how an image is constructed.
Ultimately, it’s less about replacing existing workflows and more about expanding what’s possible. The tools are evolving quickly, but the opportunity is really in how they allow filmmakers to think differently about space, light, and composition—and to build worlds that aren’t limited by the constraints of a single frame.
Thank you for being part of our interview, Kathryn! We appreciate your insight!
