We’ve been writing a lot about Gaussian splatting recently. This high-fidelity alternative to photometry is a relatively new technique for turning 2D photos or video into 3D scenes. Adobe‘s free new app Substance 3D Viewer supports it, and we’ve seen some impressive results for 3D visualisations in Unreal Engine 5 (also see our V-Ray 7 review).
Gaussian splatting can be used to capture accurate lighting and reflections as well as geometry. Now researchers have just described a new tech could mean you don’t even need to capture the images or video for gaussian splats yourself. ActiveSplat is an autonomous tool that walks its way around an indoor space and produces a realistic and accurate 3D representation with incredible efficiency.
ActiveSplat: High-Fidelity Scene Reconstruction through Active Gaussian Splatting – YouTube
Developed by researchers from Beijing Institute of Technology and Tsinghua University Institute for Artificial Intelligence, ActiveSplat is “an autonomous high-fidelity reconstruction system leveraging Gaussian Splatting.” The research paper says the system can “build a 3D map on the fly” using a Gaussian map and Voronoi graph for “efficient and complete exploration with high-fidelity reconstruction results.”
The hybrid map representation makes use of both dense information about the environment and a sparse abstraction of the space, using sparse topology for efficient viewpoint sampling and path planning while also using view-dependent dense prediction to select viewpoints, “facilitating efficient decision-making with promising accuracy and completeness.” A hierarchical planning strategy based on the topological map helps mitigate repetitive trajectories and the system can adjust its yar and pitch to deal with invisible or high-loss areas.
(Image credit: Yuetao Li et al)
What is 3D gaussian splatting?
Gaussian splatting is a relatively new technique that has taken the 3D industry by storm in the last couple of years. It’s a method for scanning objects in the real-world via video or photo and reconstructing them in 3D. Blending rasterization and optimisation, it can model the real world with incredible detail and fidelity, accurately capturing geometry, lighting and reflections .
While triangle-based meshes have traditionally been the way to represent objects in 3D via photogrammetry , splats are based on volume. 3D data is defined using gaussians which are each defined by a position, covariance, colour and an alpha channel, and gaussian splats can be created automatically from just a few smartphone photos or videos using machine learning in apps like Polycam and Luma AI (also see our pick of the best 3D scanners and the best 3D modelling software).
Gaussian splatting allows real-time photo realism and direct editing and the ability to re-light them. Since the tech is so new, there are still many potential applications that have yet to be developed, including in the integration of 3D with VR and AR, and GUI-based production pipelines, like Postshot in beta, are only just starting to emerge.
