Stop Wasting Hours on 3D Scene Alignment! 3D-Fixer Does It In-Place

Stop Wasting Hours on 3D Scene Alignment! 3D-Fixer Does It In-Place

What if I told you that everything you hate about 3D scene generation—those endless hours of pose alignment, the fragmented geometry nightmares, the per-scene optimization marathons—could vanish overnight? Here's the painful truth: traditional 3D scene generation methods have been holding you hostage. You've been forced to extract objects, align them in canonical poses, then desperately try to reconstruct spatial relationships that were obvious in the original image. It's backwards. It's broken. And it's costing you days of development time.

Enter 3D-Fixer, the CVPR 2026 breakthrough from Horizon Robotics that's rewriting the rules entirely. This isn't another incremental improvement. This is a paradigm shift. While competitors are still playing geometric Tetris, 3D-Fixer leverages a radical In-Place Completion approach—generating complete 3D assets directly where they belong, conditioned on fragmented geometry as spatial anchors. No explicit pose alignment. No time-consuming optimization. Just pure, layout-preserving 3D magic from a single image.

Ready to discover why top computer vision researchers are already abandoning conventional pipelines? Let's dive deep into the technical revolution that is 3D-Fixer.

What is 3D-Fixer?

3D-Fixer is a state-of-the-art 3D scene generation system developed by researchers at Horizon Robotics, including Ze-Xin Yin, Liu Liu, Xinjie Wang, and colleagues. Officially accepted at CVPR 2026, this project represents a fundamental departure from how we've approached 3D scene synthesis for decades.

The core innovation? In-Place Completion. Rather than following the conventional pipeline—extract visible geometry, generate objects in canonical poses, then align and compose—3D-Fixer extends 3D object generative priors to generate complete assets conditioned on partially visible point clouds at their actual spatial locations. The fragmented geometry from depth estimation methods isn't discarded or normalized away; it's embraced as the spatial anchor that preserves layout fidelity.

This matters because previous works fundamentally required explicit pose alignment, a brittle and computationally expensive step that introduces cascading errors. By operating directly in the scene's native coordinate system, 3D-Fixer maintains geometric consistency that alternative methods simply cannot achieve.

The project builds upon Microsoft's powerful TRELLIS framework but pushes it into entirely new territory. With pretrained models hosted on Hugging Face, an interactive Gradio demo, and the massive ARSG-110K dataset for training and evaluation, 3D-Fixer arrives as a complete, production-ready ecosystem—not just a research prototype.

Key Features

High-Fidelity 3D Asset Generation

3D-Fixer doesn't just fill holes—it generates photorealistic, structurally coherent 3D assets that seamlessly integrate with existing scene geometry. The coarse-to-fine architecture ensures both global structural correctness and local geometric detail. The system outputs both Gaussian Splatting representations and textured meshes, giving you flexibility for downstream applications.

Unprecedented Generalizability

Where many 3D generation systems collapse outside their training distribution, 3D-Fixer demonstrates robust generalization to real-world scenes and complex indoor environments. This isn't a synthetic-world toy; it's battle-tested on diverse, cluttered, realistically illuminated spaces that mirror actual deployment scenarios.

The In-Place Completion Paradigm

This is the secret sauce. By shifting from "generate-then-align" to "generate-where-it-sits", 3D-Fixer eliminates an entire category of failure modes. The partially visible point cloud—cropped from fragmented geometry—serves as both conditioning signal and spatial constraint. The model learns to hallucinate occluded portions while respecting visible evidence, producing completions that are geometrically plausible and spatially grounded.

No Per-Scene Optimization

Perhaps the most liberating feature: zero test-time optimization. Traditional neural radiance field methods and some generative approaches require hours of per-scene finetuning. 3D-Fixer is feed-forward inference. Single image in, complete 3D scene out. This throughput advantage transforms practical deployment from theoretical possibility to operational reality.

Production-Ready Infrastructure

With verified compatibility on NVIDIA RTX 4090 and RTX L20 GPUs, CUDA 11.8/12.8 support, and comprehensive evaluation pipelines across three benchmark datasets, 3D-Fixer arrives ready for serious research and development workflows.

Use Cases

Real Estate and Interior Design Visualization

Imagine snapping a photo of any room and instantly generating a fully explorable 3D environment. Designers can iterate on furniture arrangements, test lighting scenarios, and present immersive walkthroughs without manual 3D modeling. The layout-preserving nature of In-Place Completion means spatial relationships—critical for design decisions—remain physically accurate.

Robotics and Autonomous Navigation

For robots operating in human environments, understanding 3D scene structure from limited views is existential. 3D-Fixer enables occlusion-aware scene completion from single camera frames, allowing navigation systems to reason about hidden obstacles, plan paths through partially visible spaces, and build coherent world models without exhaustive exploration.

AR/VR Content Creation

The feed-forward inference speed makes 3D-Fixer ideal for real-time AR applications. Point your device at a space, and generate anchor-ready 3D content that respects the actual geometry. No cloud processing delays, no offline reconstruction pipelines—just instantaneous spatial understanding.

Game Development and Procedural Generation

Level designers can rapidly prototype environments from concept art or reference photography. The coarse-to-fine generation provides controllable detail levels, while the dual output formats (Gaussian Splatting and mesh) integrate directly into modern game engines. The ARSG-110K dataset's diversity—built upon TRELLIS-500K with accurate object-level ground truth—ensures training data that captures real-world scene complexity.

Step-by-Step Installation & Setup Guide

Getting 3D-Fixer running requires attention to hardware and environment details. Follow precisely.

Prerequisites

Before touching any commands, verify your system:

Clone and Initialize

# Clone with all submodules—critical for TRELLIS dependencies
git clone --recurse-submodules https://github.com/HorizonRobotics/3D-Fixer
cd 3D-Fixer

The --recurse-submodules flag is non-negotiable. 3D-Fixer inherits substantial infrastructure from TRELLIS, and missing submodules will cause cryptic failures during setup.

Environment Setup

# Execute the comprehensive setup script
# This creates the 'threeDFixer' conda environment with all dependencies
. ./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast

This single command installs the full stack^{↗ Bright Coding Blog}: xFormers for efficient attention, Flash Attention for memory-efficient transformers, differentiable octree rasterization, sparse convolution support, Mip-Splatting for anti-aliased Gaussian rendering, NVIDIA Kaolin for 3D deep learning operations, and nvdiffrast for differentiable rasterization.

For training or demo-specific needs, explore additional flags:

# View all available options
. ./setup.sh --help

Model Acquisition

Download pretrained weights from Hugging Face. For local loading, you'll also need MoGe v2 checkpoints. Update the scene_cond_model path in /path/to/3D-Fixer/pipeline.json accordingly.

Launch the Interactive Demo

# Download SAM2-Hiera-Large weights first
# Place in checkpoints/ directory per SAM2 installation instructions
# Then launch:
python app.py

The Gradio interface provides immediate visual feedback for exploring 3D-Fixer's capabilities without writing code.

REAL Code Examples from the Repository

Loading Pretrained Models (The Modern Way)

from three_d_fixer import ThreeDFixerPipeline

# Load directly from Hugging Face Hub—simplest path
# This downloads weights automatically on first use and caches locally
pipeline = ThreeDFixerPipeline.from_pretrained("HorizonRobotics/3D-Fixer")

# The pipeline encapsulates the full coarse-to-fine architecture:
# - Coarse Structure Completer (sparse voxel generation)
# - Fine Shape Refiner (detailed geometry)
# - Occlusion-Aware 3D Texturer (appearance)

This Hugging Face integration demonstrates 3D-Fixer's commitment to accessibility. No manual weight downloads, no path configuration—just standard from_pretrained semantics that the ML community expects. The pipeline object orchestrates all three architectural stages transparently.

Local Loading with Custom Geometry Conditioning

from three_d_fixer import ThreeDFixerPipeline

# For offline/air-gapped environments or custom modifications
# Download weights manually, maintain folder structure exactly
pipeline = ThreeDFixerPipeline.from_pretrained("/path/to/local/3D-Fixer")

# Critical: Update pipeline.json to point to your MoGe v2 checkpoint
# This depth estimator provides the initial fragmented geometry
# that serves as spatial anchor for in-place completion

The local path loading reveals 3D-Fixer's modular design. The scene_cond_model configuration decouples the generative pipeline from specific depth estimators, allowing substitution of alternative geometry estimation methods as the field evolves.

Inference on Custom Test Sets

# Run inference on ARSG-110K test set—your baseline for comparison
python inference_ours_testset.py \
    --output_dir ./results/arsg110k \
    --testset_dir /data/ARSG-110K-testset \
    --model_dir /models/HorizonRobotics/3D-Fixer \
    --rank 0 \
    --world_size 1

The distributed inference support (--rank and --world_size) enables scaling across multiple GPUs for large-scale evaluation. The output directory receives complete scene reconstructions with preserved directory structure for downstream metric computation.

Evaluation Metrics Computation

# Compute quantitative metrics against ground truth
python eval_metrics_ours_testset.py \
    --output_dir ./results/arsg110k \
    --testset_dir /data/ARSG-110K-testset \
    --assets_dir /data/object_assets

This separation of inference and evaluation promotes reproducible research. Raw generations are preserved, enabling re-evaluation with updated metrics or modified ground truth processing without regeneration costs.

Fine-tuning the Sparse Structure Flow Model

# Stage-one fine-tuning: adapt TRELLIS priors to non-canonical poses
# Critical insight: randomly rotated training data enables in-place generation
python train.py \
    --config configs/finetune/rand_rot_ss_flow_img_dit_L_16l8_fp16.json \
    --output_dir outputs/rand_rot_ss_flow_img_dit_L_16l8_fp16 \
    --data_dir /data/processed_scenes,/data/processed_objects \
    --auto_retry 3

This training command reveals a crucial technical detail: the random rotation augmentation. Standard TRELLIS generates objects in canonical poses—front-facing, normalized orientation. For in-place completion, objects must generate correctly at arbitrary orientations as they appear in scenes. The rand_rot configs implement this adaptation, fine-tuning the sparse structure diffusion model on randomly rotated assets.

Training the Complete 3D-Fixer Architecture

# Coarse Structure Completer: scene-level training with object pre-training
python train.py \
    --config configs/3d_fixer/scene_obj_pre_train_ss_flow_img_dit_L_16l8_fp16.json \
    --output_dir outputs/scene_coarse_completer \
    --data_dir /data/scene_data,/data/object_data \
    --auto_retry 3

# Fine Shape Refiner: resume from coarse completer checkpoint
# Update resume_ckpts in config to load pre-trained weights
python train.py \
    --config configs/3d_fixer/scene_fine_ss_flow_img_dit_after_obj_pretrain_L_16l8_fp16.json \
    --output_dir outputs/scene_fine_refiner \
    --data_dir /data/scene_data,/data/object_data \
    --auto_retry 3

The hierarchical training strategy—object pre-training, then scene-level coarse completion, then fine refinement—mirrors how humans perceive scenes: grasp overall structure first, then attend to details. The data organization requirements (/path/to/scene_data/, /path/to/object_data/) enforce clean separation between scene-level compositions and object-level assets.

Advanced Usage & Best Practices

Memory Optimization Strategies

The 24GB VRAM minimum is a hard constraint for full inference. For development and prototyping, consider:

• Gradient checkpointing during training (enabled in fp16 configs)
• Sequential stage execution rather than end-to-end generation when intermediate results suffice
• Downsampled depth inputs for faster iteration during method development

Data Curation Pipeline

ARSG-110K's quality stems from rigorous preprocessing. Follow all 14 steps in DATASET.md precisely. The dependency chain is real: Steps 1-4 and 9 enable sparse structure fine-tuning; Steps 1-11 enable decoder and flow model training; Steps 12-14 enable final scene rendering for 3D-Fixer training.

Multi-GPU Training

The --auto_retry 3 flag handles transient distributed training failures. For production-scale training, replicate TRELLIS's multi-node configuration. The codebase inherits robust distributed training infrastructure from its foundation.

Integration with Modern 3D Workflows

Output Gaussian Splatting representations import directly into PolyCam, Luma AI, and custom WebGL viewers. Mesh outputs require standard remeshing for game engine import—consider Instant Meshes or QuadWild for production topology.

Comparison with Alternatives

Why 3D-Fixer wins: The In-Place Completion paradigm eliminates an entire failure mode category. Competitors achieve speed through feed-forward architectures but still require explicit alignment steps that introduce errors. 3D-Fixer's native layout preservation means spatial relationships are correct by construction, not recovered through post-processing heuristics.

FAQ

What hardware do I absolutely need to run 3D-Fixer? An NVIDIA GPU with minimum 24GB VRAM. Verified on RTX 4090 and RTX L20. Linux operating system required. No Windows or macOS support currently.

Can I use 3D-Fixer commercially? The original code is Apache 2.0 licensed. Third-party components retain their original licenses. Review THIRD_PARTY_NOTICES.md carefully for commercial deployment.

How does In-Place Completion differ from standard 3D generation? Standard methods generate objects in canonical poses then align them to scenes. 3D-Fixer generates objects directly at their observed spatial locations, using fragmented geometry as conditioning. This preserves layout without explicit alignment optimization.

Is the training data available? Yes. ARSG-110K is released on Hugging Face, containing 110K scene-level samples with accurate 3D object ground truth, layout annotations, and diversity exceeding prior datasets.

Can I fine-tune on my own scene categories? Absolutely. The training pipeline supports custom datasets following the ARSG-110K structure. Object-level pre-training on your asset categories is recommended before scene-level training.

What's the difference between the demo and full pipeline? The Gradio demo (app.py) provides interactive exploration with SAM2 segmentation. The full pipeline enables batch evaluation, custom conditioning, and training on new data.

How do I cite 3D-Fixer in my research?

@inproceedings{yin2026tdfixer,
  title={3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image},
  author={Yin, Ze-Xin and Liu, Liu and Wang, Xinjie and Sui, Wei and Su, Zhizhong and Yang, Jian and Xie, jin},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  year={2026}
}

Conclusion

3D-Fixer isn't merely an improvement on existing methods—it's a fundamental reconceptualization of how 3D scenes should be generated from images. The In-Place Completion paradigm, born from careful analysis of where conventional pipelines fail, delivers what practitioners actually need: spatially grounded, layout-preserving, optimization-free 3D generation that works in the real world.

The technical execution matches the theoretical elegance. From Hugging Face integration to comprehensive evaluation pipelines, from verified hardware compatibility to extensive training documentation, Horizon Robotics has delivered a production-credible research artifact—not a paper-thin demo.

My assessment? If you're working in 3D scene understanding, robotics perception, AR/VR content creation, or any domain requiring single-image 3D reconstruction, 3D-Fixer deserves immediate evaluation. The elimination of pose alignment alone will save you weeks of pipeline engineering. The feed-forward inference speed will transform your throughput. And the layout fidelity will elevate your output quality in ways quantitative metrics only partially capture.

Don't wait for this to become standard—be early. Clone the repository, run the demo, and experience the future of 3D scene generation today.

👉 Get 3D-Fixer on GitHub

Explore the project page, read the full paper, and join the researchers already building with In-Place Completion.