UniWorld-View Tops WorldScore Benchmark, New Breakthrough in World Models

The difficulty of novel view synthesis lies in the 'large baseline'—given only a frontal view of a face, the AI must infer the side or even the back of the head. Past reconstruction approaches like NeRF and 3DGS rely on multi-view data; monocular video with insufficient view coverage easily produces holes and artifacts. Generative approaches can 'dare to draw', but most feed camera poses as abstract condition vectors into diffusion models, lacking explicit 3D modeling, and become uncontrollable at large angles.

Recent works like ViewCrafter and NVIDIA's GEN3C use geometric models to project the image into a 3D point cloud, then feed the point cloud render of the target view into a video diffusion model, significantly improving camera control. However, the UniWorld-View team identified two key issues with point cloud rendering: foreground-background tearing (depth boundaries lack connectivity, causing textures to be pulled onto the background when the viewpoint changes) and backside light leakage (point clouds have no 'surface' concept; when the camera moves behind, frontal points shine through directly).

To address the defects of point cloud rendering, UniWorld-View proposes occlusion-aware point cloud rendering. The first step is dual projection: project the source image to the target view and then back-project to the original view; pixels that cannot 'return' are occlusion regions. These are accumulated frame by frame along the camera trajectory to form an occlusion mask, which is removed during rendering to prevent misleading textures from confusing the generative model. The second step is normal filtering: estimate a normal vector for each point, compute the angle with the viewing direction, and directly discard points facing away from the camera, solving backside light leakage.

After solving geometric issues, texture detail deficiencies remain. UniWorld-View adopts dual-stream condition injection: the geometry stream encodes the point cloud render plus mask and adds it to the latent variable, ensuring generated content aligns with 3D structure; the appearance stream uses the Ref-DiT module, performing cross-attention with novel view features as Query and source video features as Key/Value, allowing the model to 'borrow materials' from the original video to complete textures, even repairing blurry textures caused by point cloud artifacts.

UniWorld-View uses the same model and code for both single-image 3D generation and video 4D generation. For dynamic videos, it decouples spatial transformation from temporal progression: first freeze time at the first frame, generate a full circular static surround view as appearance anchors; then generate synchronized multi-view videos at fixed camera positions, aligning the first frame of each position with the 'portrait', and progressively completing foreground self-occlusions through iterative generation. The resulting multi-view videos can be used for 4DGS optimization, enabling a complete pipeline from casual monocular capture to free-viewpoint roaming.

UniWorld-View has topped the WorldScore leaderboard from Fei-Fei Li's team, with training data from Peking University-affiliated startup YuanKong Intelligence. It has been adapted for domestic Ascend computing power, and the code and weights are open-sourced on GitHub. The Tuzhan Intelligence team states that they are currently productizing the related capabilities to push visual AI from models to practical applications.