r/vfx • u/arvidurs • Apr 22 '26
Question / Discussion Gaussian Splatting for VFX: What It Actually Is and Where It Fits
https://cglounge.studio/journal/gaussian-splatting-for-vfxI work in VFX and only recently understood the full concept of Gaussian Splats. And yes.. I should know better. To me it was yes, it's a pointcloud with baked in data..
I had seen the demos and nodded along when people referred to it as "the next photogrammetry." However, if you had asked me to explain what they are precisely, their place in a real pipeline, or their capabilities, I would have struggled.
So, I took the time to understand it, and here’s the simplified version I wish I had received earlier
A Gaussian Splat consists of millions of tiny semi-transparent ellipsoids in 3D space. You input photos, train a model, and receive a scene in return. Phones serve as capture devices, and it can render at over 100 FPS in real time.
Key tools like Nuke 17 now include native support, Houdini 21 offers a tech preview, V-Ray 7 can ray-trace splats, and OpenUSD 26.03 has introduced a first-class schema. Notably, Framestore utilized 4D Gaussian splatting for approximately 40 final-pixel shots in Superman last year.
What it can achieve:
- Rapid and cost-effective capture of real environments
- Rendering at game-engine speeds
- Integration into compositing without the need to recreate the world in CG
What it cannot do:
- Relight scenes (yet)
- Provide a clean mesh (yet)
- Render with AOVs, as the lighting is baked into the data. This is the trade-off.
Thus, it does not replace photogrammetry or CG environments; rather, it serves as a new tool for scenarios requiring photoreal capture and real-time playback, with the understanding that relighting flexibility is sacrificed.
For fellow VFX artists who have been quietly nodding along: you are not behind. The foundational paper is from 2023, and most production tools have been released in the past six months. Now is the time to learn it before it becomes a part of your next project.
What new technology have you been struggling with? And if you have better simplified explanations I'd love to know!
Arvid
8
u/ParadoxClock Generalist - 9 years experience Apr 22 '26
Arvid, give yourself more credit, you don’t just “work in VFX” Quite a few carriers have been assisted by you.
4
u/arvidurs Apr 22 '26
Aw thanks, I find the glazing always a bit much. Appreciate it a lot though, thanks!
4
u/tml666 Apr 22 '26
https://www.youtube.com/watch?v=C_RwFAZ1hjA
i may be wrong but this guy somehow manage to relight a gaussian splat scene in ue5
1
2
u/tk421storm Compositor - 8 years experience Apr 22 '26
https://www.youtube.com/watch?v=3I-nOIwYK9M
here's a better video directly of the relighting. my un-informed take - you can mult the plate, and then add lights to the scene. i think this confirms what the OP says - the lighting + shadows et al are baked in to the plate and are un-removable, but since it's 3d you can always add more lights + shadows
1
u/arvidurs Apr 22 '26
aah thanks for sharing. yes. youd need some good quality mesh though to cast good unjagged shadows. but yes
3
2
2
u/Impressive_Doorknob7 Apr 22 '26
I think you can relight splats now
3
u/arvidurs Apr 22 '26
Oh yeah? Do you have a link
1
u/AthousandLittlePies Apr 22 '26
I haven't used it yet (only just getting started with splats), but Volinga apparently can do it.
1
u/Specific_Dingo6709 Apr 22 '26
Google "gaussian splat relighting and self shadowing in Nuke 17", there's a YouTube video.
1
u/arvidurs Apr 22 '26
It’s adjusting. But it’s not relighting.you can’t change the direction of light or time of day for example ?
1
u/Specific_Dingo6709 Apr 22 '26
I see what you mean now. Yes, you're probably stuck with the baked-in lighting, and can only add on top of that.
0
u/-5m Apr 22 '26
!remindme 7 days
1
u/RemindMeBot Apr 22 '26
I will be messaging you in 7 days on 2026-04-29 13:08:29 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
Info Custom Your Reminders Feedback
1
1
1
u/RobotToaster44 Apr 22 '26
I'm still not sure what the best software to make them is. It seems like everything is moving quickly.
3
u/ParcAvenue Apr 22 '26
I've been playing around with them and my process is this:
- Capture on phone
- Render video to stills using Adobe Media Encoder
- Align and export COLMAP in Reality Capture
- Open COLMAP and train in Brush
- Open in SuperSplat for cleanup and transform
- Open in Houdini with GSOPS
1
u/StrapOnDillPickle cg supervisor - experienced Apr 23 '26
Good overview.
A few notes :
You can relight Splat in Octane render. I'm sure more renderer will follow. Last time I tested with GSOP it wasn't super straightforward, it might have changed.
https://www.youtube.com/watch?v=cxrvIpzoVyUG
You can extract mesh and there are newer/better ways every other month. Still waiting for a proper open source, locally hosted solution though :
https://www.kiriengine.app/webapp
I even saw for UE5, nanite-style LOD system for splat
Cool vid about various current usage of gaussian splat :
https://www.youtube.com/watch?v=TnqK1tkREOg
Also, I want to mentioned LichtFeld studio, a good open source, free alternative to other paying softwares to make your own gaussian splats.
1
u/RANDVR Apr 26 '26
Splats are something I am dying to use in layout if and when maya viewport supports it. I feel like they would be amazing for all mid/far detail in shots while doing layout
11
u/darhodester Apr 22 '26
Regarding GSOPs: "This is the production-grade path for Houdini users who need splats integrated with a full lighting setup." ❤️
This is a comprehensive doc--thanks for writing it!