NVlabs/ProtoMotions: GPU-Accelerated Humanoid Simulation Framework

NVlabs/ProtoMotions: GPU-Accelerated Humanoid Simulation Framework

Training physically simulated humanoids and robots at scale has traditionally required stitching together disparate tools—physics engines, motion datasets, retargeting pipelines, and deployment frameworks—each with incompatible conventions and steep learning curves. Researchers in animation, robotics, and reinforcement learning waste cycles on plumbing rather than experimentation. NVlabs/ProtoMotions addresses this fragmentation directly: it is a unified, GPU-accelerated simulation and learning framework that integrates motion learning, multi-physics backend support, and sim-to-real deployment in a single modular codebase. Built by NVIDIA Labs and released under Apache 2.0, it targets practitioners who need to move from idea to working policy on real hardware without rebuilding infrastructure at every step.

What is NVlabs/ProtoMotions?

ProtoMotions3 (the current major version) is an open-source Python^{↗ Bright Coding Blog} framework maintained by NVIDIA Labs for GPU-accelerated simulation and learning of physically simulated digital humans and humanoid robots. With 2,130 GitHub stars, 408 forks, and active development (last commit July 6, 2026), it sits at the intersection of reinforcement learning, physics simulation, and character animation.

The framework's architecture centers on three principles from its maintainers: modularity, extensibility, and scalability. Unlike monolithic RL environments that couple task logic, observations, and rewards into single classes, ProtoMotions decomposes these into swappable components. This design reflects its stated mission as a "fast prototyping platform" that bridges the animation, robotics, and RL communities.

ProtoMotions supports multiple physics backends—NVIDIA IsaacGym (Preview 4), IsaacLab 2.3.0, NVIDIA Newton, MuJoCo 3.0+, and has experimental interfaces for Genesis (untested, community-contributed). It also integrates with IsaacSim 5.0+ for high-fidelity rendering. The framework is permissively licensed under Apache-2.0, making it viable for both academic research and commercial applications.

A sibling project, MimicKit, provides a lightweight alternative specifically for motion imitation learning, suggesting ProtoMotions targets more complex, multi-objective training scenarios.

Key Features

Multi-Backend Physics Support ProtoMotions abstracts simulator-specific APIs behind a common interface in protomotions/simulator/base_simulator/. Users can switch between IsaacGym, Newton, and MuJoCo via command-line flags—enabling sim2sim validation where policies trained in one engine are tested in another. This is critical for identifying physics-engine-specific artifacts before real-world deployment.

Large-Scale Motion Learning The framework trains on the full public AMASS dataset (40+ hours of human motion) in approximately 12 hours on 4 A100 GPUs. For larger-scale experiments, it distributes across 24 A100s with 13K motions per GPU using the BONES-SEED dataset in SOMA skeleton format.

One-Command Retargeting Built-in retargeting via PyRoki (as of v3; earlier versions used Mink) transfers entire motion datasets to new robot skeletons. Changing --robot-name=smpl to --robot-name=h1_2 with prepared retargeted motions enables training on different hardware targets without pipeline rewrites.

Sim-to-Real Deployment ProtoMotions exports policies as ONNX models with observation computation baked in, eliminating the need to replicate observation functions in deployment frameworks. The maintainers validated zero-shot transfer to a Unitree G1 humanoid using the RoboJuDo framework, requiring only a single policy file with no mandatory core changes.

Modular Task Composition Tasks are assembled from discrete components: control logic, observation kernels, reward functions, and experiment configs bound via MdpComponent and FieldPath descriptors. This avoids the "monolithic env class" anti-pattern common in RL frameworks.

Generative Policy Support Integration with MaskedMimic enables training policies that autonomously select discrete motion primitives, with support for PEFT task adapters and reusable latent priors via the GPC (Generative Policy Composition) system.

Use Cases

Research in Physics-Based Character Animation Animation researchers can train digital humans on the full AMASS dataset to learn naturalistic motion skills—walking, running, vaulting—without hand-crafted controllers. The SOMA skeleton format support and Kimodo text-to-motion integration enable rapid iteration from concept to physics-based policy.

Humanoid Robot Learning Robotics labs working with Unitree G1, H1_2, or custom humanoids can leverage pre-trained motion priors. The 12-hour training time on AMASS skills (with retargeting) and direct ONNX export to deployment frameworks like RoboJuDo creates a viable path from simulation to hardware.

Sim2Sim Policy Validation For safety-critical applications, engineers can verify policy robustness across Newton, MuJoCo, and IsaacGym. The README documents this explicitly: --simulator=isaacgym → --simulator=newton → --simulator=mujoco with policies using only hardware-observable quantities.

Synthetic Data Generation Procedural scene generation combined with RL-based motion adaptation supports scalable SDG workflows. Starting from seed motions, policies adapt to augmented environments—useful for training perception systems or testing generalization.

Custom RL Algorithm Development The modular agent system allows implementing new algorithms in approximately 50 lines. The README cites ADD (Advantage-Weighted Distillation) as an example in protomotions/agents/mimic/agent_add.py, demonstrating the framework's suitability for methods research.

Installation & Setup

The README directs users to the full documentation for detailed installation. Based on the framework's dependencies and structure, setup involves:

1. Clone the repository:

git clone https://github.com/NVlabs/ProtoMotions.git
cd ProtoMotions

2. Install core dependencies: ProtoMotions requires Python and integrates with NVIDIA's Isaac ecosystem. The specific package installation depends on your target simulator backend:

# For IsaacLab/IsaacGym workflows
pip install -e .
# Or with specific simulator extras
pip install -e ".[isaaclab]"

3. Verify simulator installation: IsaacGym Preview 4, IsaacLab 2.3.0, or MuJoCo 3.0+ must be independently installed. The framework detects available backends at runtime.

4. Download motion data (for training):

# Follow AMASS preparation guide
# https://protomotions.github.io/getting_started/amass_preparation.html

# Or BONES-SEED dataset for large-scale experiments
# https://protomotions.github.io/getting_started/seed_bvh_preparation.html

5. Quick validation: The Quick Start guide provides minimal working examples to verify GPU acceleration and simulator connectivity.

For deployment-specific setup, the G1 Deployment Tutorial covers data preparation through real robot execution.

Real Code Examples

The README provides architectural patterns rather than extensive copy-paste snippets. Below are the concrete examples present, reproduced with context:

Example 1: Switching Robot Targets The framework's design for robot-agnostic training is demonstrated by this minimal command-line change:

# Train SMPL human character
python train.py --robot-name=smpl

# Train Unitree H1_2 robot (after retargeting motions)
python train.py --robot-name=h1_2

This single-argument switch works because robot configurations are modular: add a MuJoCo XML to protomotions/data/assets/mjcf/, define parameters in protomotions/robot_configs/, and register in protomotions/robot_configs/factory.py. The observation kernels, reward functions, and control logic remain unchanged.

Example 2: Sim2Sim Backend Switching Validate policy transfer across physics engines:

# Train in IsaacGym
python train.py --simulator=isaacgym

# Test in NVIDIA Newton
python test.py --simulator=newton

# Validate on MuJoCo CPU
python test.py --simulator=mujoco

Policies shown in the README's sim2sim demonstrations use only observations available from real hardware—proprioception and IMU data, not simulator-privileged state. This constraint ensures validation relevance for deployment.

Example 3: Custom Simulator Interface For integrating a new physics engine, implement the base simulator API:

# Refer to base interface:
# protomotions/simulator/base_simulator/

# Community example for Genesis:
# protomotions/simulator/genesis/

The Genesis integration is explicitly marked untested, indicating this is an active extension point rather than a polished feature.

Example 4: Modular Task Definition The steering task composition illustrates the framework's component architecture:

This table from the README demonstrates that tasks are assembled declaratively from tested components rather than implemented as monolithic classes.

Advanced Usage & Best Practices

Start with IsaacLab for new projects. IsaacLab 2.3.0 represents NVIDIA's current-generation robotics framework; IsaacGym Preview 4 is legacy. The README badges both, but IsaacLab receives prominent placement.

Use sim2sim before sim2real. The Newton and MuJoCo test paths exist specifically to catch simulator-specific dynamics assumptions. Policies that fail sim2sim will likely fail on hardware.

Pre-train on AMASS, fine-tune on target motions. The 12-hour AMASS training time establishes strong locomotion priors. Task-specific adaptation requires less data and compute than training from scratch.

Bake observations into ONNX exports. The deployment pipeline's inclusion of observation computation in the exported model eliminates a common source of sim-to-real gaps—mismatched observation normalization or coordinate frames.

Contribute simulator backends via the base API. The Genesis example shows the intended extension pattern. For production use, expect to implement tensor-level operations matching the framework's GPU-accelerated assumptions.

Monitor PEFT adapter compatibility. The GPC system's support for parameter-efficient fine-tuning is documented but evolving. Verify adapter compatibility with your base policy architecture before relying on it for multi-task deployment.

Comparison with Alternatives

ProtoMotions distinguishes itself through tight integration of animation and robotics workflows—AMASS retargeting, motion prior training, and humanoid-specific deployment—rather than general robot learning. IsaacLab provides broader simulator support and larger community but requires more assembly for humanoid motion tasks. MuJoCo Menagerie offers verified models but no training framework. RoboSuite targets manipulation, not locomotion.

The trade-off: ProtoMotions optimizes for humanoid-specific velocity (12-hour AMASS training, one-command retargeting) at the cost of generality. For non-humanoid robots or non-motion tasks, IsaacLab may be more appropriate.

FAQ

What GPUs are required? The README documents training on NVIDIA A100s. GPU acceleration depends on the backend—IsaacGym/IsaacLab require CUDA-capable NVIDIA GPUs.

Is Windows supported? Not explicitly documented. NVIDIA's Isaac ecosystem historically targets Linux; verify against current IsaacLab platform support.

Can I use my own robot? Yes. Add MJCF XML, create a config file following protomotions/robot_configs/g1.py, and register in factory.py.

What motion datasets work? AMASS, BONES-SEED, PHUMA, and Kimodo-generated motions are documented with preparation guides.

Is Genesis support production-ready? No—explicitly marked "untested" in README badges. Use IsaacGym, IsaacLab, Newton, or MuJoCo for reliable results.

How does licensing affect commercial use? Apache-2.0 permits commercial use. Third-party assets (Unitree models, SMPL bodies) have separate notices in legal/.

Where do I report issues? The repository includes a CONTRIBUTING.md guide. For questions, consult the documentation first.

Conclusion

NVlabs/ProtoMotions occupies a specific, well-defined niche: researchers and engineers who need to train physics-based humanoid policies at scale and deploy them on real hardware without rebuilding pipelines for each target. Its modular architecture, multi-backend support, and explicit sim-to-real validation path address genuine pain points in the current landscape of fragmented RL tools.

The framework is best suited for: animation researchers leveraging AMASS-scale data, robotics labs working with Unitree or similar humanoids, and RL practitioners studying generalization across physics engines. It is less ideal for general robot manipulation or users needing mature, community-tested support for non-NVIDIA simulators.

With active maintenance, permissive licensing, and documented deployment to real robots, ProtoMotions represents a credible, production-oriented contribution to humanoid learning infrastructure—not a research prototype abandoned after publication.

Explore the framework, documentation, and deployment tutorials at https://github.com/NVlabs/ProtoMotions.