NVIDIA Research has developed a groundbreaking AI technology that promises to transform how visual effects are created across the film and gaming industries. DiffusionRenderer, presented at CVPR 2025, represents a fundamental shift in computer graphics by combining two traditionally separate processes—inverse and forward rendering—into a unified neural framework.
The system works through a two-stage approach powered by video diffusion models. First, a neural inverse renderer analyzes standard RGB video footage and intelligently estimates the scene's intrinsic properties, generating essential data buffers (G-buffers) that describe geometry (depth, normals) and materials (color, roughness, metallic) at the pixel level. Then, a neural forward renderer generates photorealistic images from these G-buffers without requiring explicit light transport simulation.
What makes DiffusionRenderer revolutionary is that it eliminates the need for precise 3D scene representations and computationally expensive path tracing that traditional physically-based rendering (PBR) requires. "Classic PBR relies on explicit 3D geometry, high-quality material properties, and lighting conditions that are often impractical to obtain in real-world scenarios," explains Sanja Fidler, VP of AI Research at NVIDIA.
For independent filmmakers and smaller studios, this technology represents a democratization of high-end visual effects. Creators can now film a single RGB video and use DiffusionRenderer to add CGI elements that cast realistic shadows and reflections—without expensive light stage systems, LiDAR scans, or specialized equipment. The system can also dynamically relight entire scenes, change materials, and seamlessly insert virtual objects into real footage.
Since its initial development, NVIDIA has integrated DiffusionRenderer with their Cosmos Predict-1 foundation models, achieving significantly higher quality results. The technology is available under Apache 2.0 and the NVIDIA Open Model License, with code and model weights accessible on GitHub. As resolution and dynamic range capabilities continue to scale, DiffusionRenderer is poised to become an essential tool for creators at all budget levels, fundamentally changing how visual storytelling is approached in film, advertising, and game development.