A team of Apple researchers has developed a new framework for high-resolution 3D scene rendering known as LGTM (Less Gaussians, Texture More). This framework addresses the inefficiencies of existing feed-forward 3D Gaussian Splatting methods that struggle with high resolutions.
As resolution increases, traditional feed-forward methods become cost-prohibitive, hindering the feasibility of high-resolution 3D scene generation. LGTM aims to enhance the capabilities of existing systems by separating geometric complexity from rendering resolution, allowing for simpler geometry while adding detailed textures.
The LGTM framework builds on previously established feed-forward methods by layering texture predictions atop geometric structures. Researchers trained the model using low-resolution images validated against high-resolution ground truths, ensuring the generated geometry remained accurate. Additionally, a second network focused on detailed textures from high-resolution images enables the system to produce visually rich results.
This innovation could significantly impact the Apple Vision Pro headset, which features displays totaling approximately 23 million pixels. Current feed-forward methods face limitations under such high resolutions, leading to computational bottlenecks when generating scenes rapidly and accurately. LGTM could facilitate smoother performance and sharper visuals in applications requiring high-resolution scene rendering.
By deploying LGTM, Apple may provide users with more immersive environments and improved passthrough experiences while reducing processing demands. The LGTM project page offers demonstration methods like NoPoSplat, DepthSplat, and Flash3D, showcasing enhanced results compared to previous techniques.
Sample videos and images from the project illustrate LGTM’s capacity to deliver richer details and results closer to high-resolution ground truths, demonstrating its potential benefits in practical applications.
