Researchers are using 2D diffusion models to generate 3D assets to bypass the shortage of available 3D training data.
This approach matters because high-quality 3D datasets are rare compared to the vast amounts of 2D imagery available on the internet. By leveraging the strengths of 2D diffusion, developers can create usable 3D models without needing massive libraries of native 3D data.
Lewis Stuart of the University of Nottingham said the process via Computerphile, detailing how 2D techniques can be repurposed for three-dimensional generation. The method focuses on bridging the gap between flat imagery and spatial volume, allowing the AI to infer depth and structure from 2D patterns.
This research aligns with a broader trend in the industry to move toward native 3D generation. For example, Tripo AI demonstrated its Tripo P1.0 native 3D diffusion model at GDC 2026 [1]. Such developments suggest a shift away from purely 2D-to-3D conversions toward models that understand volumetric space inherently.
The evolution of these tools follows a trajectory of rapid iteration in generative AI. Stability AI announced a new family of Stable Diffusion models in 2024 [2], which contributed to the foundation of how diffusion processes are handled across different media. Additionally, research into new AI methods for these generations was highlighted in reports as early as December 2024 [3].
Stuart said the goal is to create usable assets despite small or unavailable 3D datasets. By utilizing the existing power of 2D diffusion, the industry can accelerate the production of assets for gaming, virtual reality, and industrial design, reducing the manual labor required for 3D modeling.
“2D diffusion models can be repurposed to generate 3D models”
The ability to synthesize 3D objects from 2D data reduces the dependency on expensive, manually curated 3D libraries. This democratizes the creation of complex spatial assets, potentially lowering the barrier to entry for indie game developers and architects while pushing AI toward a more comprehensive understanding of physical geometry.
