DART-GUI: The Secret RL Method Top GUI Agent Researchers Are Using

What if everything you thought about training GUI agents was wrong? For years, researchers have been trapped in a brutal cycle—burning through millions in compute, watching their reinforcement learning pipelines collapse under the weight of multi-turn interactions, and settling for agents that choke on real-world desktop tasks. The pain is universal. The frustration is real. And the waste? Insane.

But here's the twist nobody saw coming: a team of researchers just cracked the code on efficient multi-turn RL for computer-use agents, and they're giving it away for free. Meet DART-GUI—the open-source framework that's making traditional GUI agent training look like ancient history. Built on a radical approach called Decoupled Training and Adaptive Data Curation, this isn't another incremental improvement. It's a fundamental reimagining of how we teach AI to use computers like humans do.

If you're building the next generation of autonomous agents, skipping this repository could cost you months of wasted experiments. Let's pull back the curtain on what's really happening inside github.com/Computer-use-agents/dart-gui and why the smartest teams in agentic AI are already migrating their pipelines.

What is DART-GUI?

DART-GUI (Decoupled training with Adaptive data cuRation for GUI agents) is a cutting-edge reinforcement learning framework specifically engineered to train GUI agents through efficient multi-turn interactions. Born from research by Pengxiang Li, Zechen Hu, Zirui Shang, Jingrong Wu, and their collaborators—with guidance from advisors Qing Li and Zhi Gao—this project represents a significant leap forward in the computer-use agent space.

The repository emerged from a critical observation: existing GUI agent training methods suffer from catastrophic inefficiency when handling multi-turn dialogues. Each turn compounds error rates, reward signals get polluted, and the computational cost explodes exponentially. DART-GUI attacks this problem at its root through two architectural innovations: decoupled training that separates rollout generation from policy optimization, and adaptive data curation that intelligently filters and prioritizes the most valuable training trajectories.

What makes DART-GUI genuinely disruptive is its production-ready infrastructure. Unlike research prototypes that die in Jupyter notebooks, this ships with complete Docker^{↗ Bright Coding Blog} orchestration, MySQL^{↗ Bright Coding Blog}-backed experiment tracking, and distributed training support across GPU clusters. The team released their full pipeline in December 2025—including training code, sampling infrastructure, SQL schemas, and containerized deployments that scale from single-machine prototyping to multi-node clusters.

The framework builds atop proven open-source foundations, leveraging verl for RL orchestration and vLLM for high-throughput inference. But the magic isn't in the components—it's in how DART-GUI composes them. By decoupling the rollout service from the training loop, the system achieves something precious: continuous data generation without blocking gradient updates. Your GPUs never sit idle waiting for fresh trajectories. Your CPUs never stall waiting for model checkpoints.

The project's momentum is undeniable. With their 7B parameter model publicly available on Hugging Face and a comprehensive arXiv paper detailing the methodology, DART-GUI is rapidly becoming the de facto standard for serious GUI agent research.

Key Features That Separate DART-GUI From the Pack

Decoupled Training Architecture

The core innovation. DART-GUI physically separates rollout generation (collecting agent-environment interactions) from policy optimization (updating the model). This isn't just conceptual separation—it's container-level isolation with distinct Docker services communicating through a database-backed message queue. The rollouter container runs inference-heavy trajectory collection while the trainer container focuses purely on gradient computation. Result? Maximum hardware utilization and elimination of pipeline bubbles.

Adaptive Data Curation

Not all trajectories deserve equal attention. DART-GUI's curation engine automatically filters low-quality rollouts, deduplicates redundant paths, and prioritizes high-reward trajectories that teach the most per gradient step. This is active learning at infrastructure scale—reducing training time by focusing compute where it matters.

Multi-turn RL Optimization

Standard RL frameworks optimize single-turn responses. DART-GUI's reward shaping and credit assignment are explicitly designed for multi-turn GUI interactions, where actions span dozens of steps and delayed rewards are the norm. The system properly propagates credit through long horizon trajectories—a notorious failure point in conventional approaches.

Production-Grade Experiment Tracking

Every rollout, checkpoint, and training run is persisted to MySQL with a rigorous schema. The rollout_run table tracks trajectory metadata, rewards, and usage status. The checkpoint table manages model versions with full provenance—status tracking, storage paths, configuration snapshots, and soft-delete support for reproducibility.

Distributed Docker Orchestration

Ready-to-deploy containers with GPU passthrough, shared memory optimization (--shm-size=200g for rollouter, 1500g for trainer), and port-mapped services. The architecture supports heterogeneous hardware—GPU machines for inference and training, CPU machines for environment simulation.

Seamless Integration with UI-TARS

DART-GUI is designed to enhance and extend the UI-TARS-1.5-7B foundation model. The quick-start explicitly downloads this checkpoint as the initialization point, demonstrating the framework's role as a specialized RL finetuning layer atop powerful base models.

Use Cases: Where DART-GUI Absolutely Dominates

Autonomous Desktop Automation

Building agents that operate Excel, Photoshop, or IDE environments? Multi-turn interactions are unavoidable—opening files, navigating menus, executing commands, verifying outputs. DART-GUI's decoupled architecture ensures your training pipeline scales with task complexity, not against it.

Web-based Workflow Agents

Browser automation with dozens of sequential actions: form filling, data extraction, cross-page navigation. Standard RL chokes on the long horizon credit assignment. DART-GUI's adaptive curation surfaces the rare successful trajectories that complete entire workflows, accelerating convergence.

Cross-Application Task Completion

The holy grail of computer-use agents: tasks spanning multiple applications (spreadsheet → email → calendar). These require hundreds of precise interactions with sparse rewards. DART-GUI's infrastructure handles the scale; its curation ensures learning signal isn't drowned in failure modes.

Research Reproducibility & Benchmarking

The MySQL-backed tracking makes DART-GUI ideal for rigorous academic research. Every hyperparameter, every checkpoint, every trajectory is queryable. No more "it worked on my machine"—the schema enforces experimental discipline.

Enterprise Agent Fine-tuning

Organizations with proprietary GUI applications need custom RL pipelines without rebuilding infrastructure from scratch. DART-GUI's Docker-first approach and modular architecture enable rapid adaptation to internal tools and compliance requirements.

Step-by-Step Installation & Setup Guide

Let's get DART-GUI running. The setup spans two machines: a GPU machine for inference and training, plus a CPU machine for environment simulation.

Phase 1: Preparation

Pull the required Docker images:

# DART-GUI container with all dependencies pre-installed
docker pull crpi-iwtwdoj3ikoon38c.cn-beijing.personal.cr.aliyuncs.com/pengxiangli1999/dart-gui:v0

# MySQL for experiment tracking
docker pull mysql:8.0.44-debian

Download the base model checkpoint:

# UI-TARS-1.5-7B serves as the initialization point for DART-GUI training
huggingface-cli download ByteDance-Seed/UI-TARS-1.5-7B --local-dir <your local path>

Phase 2: GPU Machine — Container Initialization

Rollouter Container (trajectory generation):

docker run -dit \
  --name rollouter \
  --gpus all \
  -p 6008:6008 \
  -p 8881:8881 \
  -p 15959:15959 \
  --shm-size=200g \
  -v <your workspace>:<your workspace in docker> \
  crpi-iwtwdoj3ikoon38c.cn-beijing.personal.cr.aliyuncs.com/pengxiangli1999/dart-gui:v0

Trainer Container (policy optimization):

docker run -dit \
  --name trainer \
  --gpus all \
  -p 6009:6008 \
  -p 8882:8881 \
  -p 15960:15959 \
  -v <your workspace>:<your workspace in docker> \
  --shm-size=1500g \
  crpi-iwtwdoj3ikoon38c.cn-beijing.personal.cr.aliyuncs.com/pengxiangli1999/dart-gui:v0

Note the massive shared memory allocation for the trainer—1500g versus 200g for the rollouter. This accommodates the gradient accumulation and optimizer states for large model training.

Phase 3: Database Initialization

docker run -dit \
  --name mysql-server \
  -p 3306:3306 \
  -e MYSQL_ROOT_PASSWORD=admin \
  -v <your sql default path>:/var/lib/mysql \
  mysql:8.0.44-debian

Connect and execute the schema from the README. The database credentials are:

• User: root
• Password: admin
• Port: 3306

Phase 4: CPU Machine — Environment Setup

Follow the GUI-Docker-Env repository instructions to configure the environment simulation layer.

Phase 5: Execution Pipeline

Start services in sequence:

# On GPU machine — inside rollouter container
cd dart_rollouter
sh model_service.sh

# On CPU machine — inside configured environment
cd dart_rollouter
sh run.sh

# On GPU machine — inside trainer container, once data flows
sh examples/osworld/async/run_trainer_debug_w_rollout_stepwise_train_pt.sh

The run_trainer_debug_w_rollout_stepwise_train_pt.sh script name reveals critical implementation details: stepwise training with periodic rollout integration, debug mode for development, and PyTorch backend.

REAL Code Examples from the Repository

Example 1: Database Schema — Rollout Tracking

The rollout_run table is the nerve center of DART-GUI's decoupled architecture. Every trajectory generated by the rollouter is logged here before being consumed by the trainer:

-- Core table for trajectory metadata and quality filtering
CREATE TABLE `rollout_run` (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `run_id` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `trajectory_id` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL,
  `task_id` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL,
  `trace_id` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL,
  `split_dir` varchar(512) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `reward` double DEFAULT NULL,              -- Critical for adaptive curation
  `num_chunks` int DEFAULT NULL,             -- Multi-turn trajectory length
  `used` int NOT NULL DEFAULT '0',           -- Consumption tracking for idempotency
  `model_version` varchar(512) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `instruction` varchar(1024) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `create_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  UNIQUE KEY `uq_rollout_run_id` (`id`),
  UNIQUE KEY `uk_rollout_run_traj_run` (`trajectory_id`,`run_id`),  -- Prevent duplicate consumption
  KEY `idx_rollout_run_task` (`task_id`)     -- Fast task-based queries
) ENGINE=InnoDB AUTO_INCREMENT=1319846 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

Key insight: The used flag enables exactly-once consumption semantics—critical when multiple trainer processes compete for fresh data. The reward field drives adaptive curation, while num_chunks tracks trajectory complexity. The composite unique key on (trajectory_id, run_id) prevents race conditions in distributed setups.

Example 2: Checkpoint Management Schema

DART-GUI treats checkpoints as first-class entities with full lifecycle management:

-- Production-grade checkpoint provenance tracking
CREATE TABLE `checkpoint` (
  `id` bigint NOT NULL AUTO_INCREMENT COMMENT 'Primary Key ID',
  `name` varchar(50) NOT NULL DEFAULT '' COMMENT 'Checkpoint Name (Unique English Identifier)',
  `version` varchar(50) NOT NULL COMMENT 'Version Number (Semantic Versioning, e.g., v1.0.0)',
  `run_id` varchar(191) NOT NULL DEFAULT '',
  `status` varchar(20) NOT NULL DEFAULT 'PENDING' COMMENT 'Status: PENDING|RUNNING|COMPLETED|FAILED|DEPRECATED',
  `path` varchar(255) NOT NULL COMMENT 'Storage Path (e.g., s3://bucket/path/checkpoint.ckpt)',
  `source` varchar(50) DEFAULT NULL COMMENT 'Source (e.g., User Upload/Training Generated/System Migration)',
  `operator` varchar(50) DEFAULT NULL COMMENT 'Operator (User ID or System Account)',
  `remark` varchar(1024) DEFAULT NULL COMMENT 'Remark (Free text format)',
  `config_yaml` text COMMENT 'Full Deployment Config (Encrypted Storage)',
  `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Created At',
  `updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'Last Updated At',
  `deleted_at` timestamp NULL DEFAULT NULL COMMENT 'Soft Delete Time',
  `started_at` timestamp NULL DEFAULT NULL COMMENT 'Started At',
  `finished_at` timestamp NULL DEFAULT NULL COMMENT 'Finished At',
  PRIMARY KEY (`id`),
  KEY `idx_status` (`status`),
  KEY `idx_created_at` (`created_at`),
  KEY `idx_updated_at` (`updated_at`)
) ENGINE=InnoDB AUTO_INCREMENT=3117 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci COMMENT='Model Checkpoint Table (Records training checkpoints and deployment versions)';

Key insight: This isn't amateur hour. The schema supports semantic versioning, status workflows (PENDING → RUNNING → COMPLETED/FAILED), soft deletes for audit compliance, and encrypted configuration storage. The started_at/finished_at timestamps enable precise training duration analysis. Enterprise teams will recognize this as MLOps maturity.

Example 3: Docker Rollouter Deployment

The rollouter container configuration reveals DART-GUI's network architecture:

docker run -dit \
  --name rollouter \
  --gpus all \
  -p 6008:6008 \      # Model service API port
  -p 8881:8881 \      # Internal communication port
  -p 15959:15959 \    # Metrics/healthcheck port
  --shm-size=200g \   # Shared memory for vLLM KV cache
  -v <your workspace>:<your workspace in docker> \
  crpi-iwtwdoj3ikoon38c.cn-beijing.personal.cr.aliyuncs.com/pengxiangli1999/dart-gui:v0

Key insight: Three exposed ports suggest a service-oriented architecture: model inference (6008), inter-service messaging (8881), and observability (15959). The 200g shared memory is non-negotiable for vLLM performance—it stores the key-value cache that enables efficient batch inference across multiple concurrent rollouts.

Example 4: Training Launch Script

The actual training invocation:

# Located at: examples/osworld/async/run_trainer_debug_w_rollout_stepwise_train_pt.sh
sh examples/osworld/async/run_trainer_debug_w_rollout_stepwise_train_pt.sh

While the full script contents aren't expanded in the README, the path conveys critical information:

• osworld: Targets the OSWorld benchmark for open-ended computer tasks
• async: Asynchronous training—gradients computed while new rollouts generate
• debug: Development-friendly logging and reduced scale
• stepwise_train: Training proceeds in discrete steps with intermittent rollout synchronization
• pt: PyTorch backend (not JAX or TensorFlow)

This naming convention demonstrates thoughtful engineering—every component's purpose is immediately transparent.

Advanced Usage & Best Practices

Optimize Shared Memory Allocation

The default 1500g for trainers assumes A100/H100-class hardware with ample system RAM. For smaller setups, reduce proportionally but never below 400g—PyTorch's distributed data parallel will crash during large model synchronization.

Monitor the used Flag for Data Freshness

Build dashboards querying SELECT COUNT(*) FROM rollout_run WHERE used = 0 to visualize your data buffer health. If this drops to zero, your trainer is starving—scale rollouter GPU count or optimize inference throughput.

Leverage Checkpoint Status Workflow

Don't manually manage checkpoint files. Query WHERE status = 'COMPLETED' for proven training artifacts, and set status = 'DEPRECATED' rather than deleting—this preserves reproducibility for paper submissions or audit requirements.

Separate Networks for Security

The exposed ports (6008, 8881, 15959) should bind to internal network interfaces only in production. Use Docker network isolation or VPN tunnels—your model checkpoints are valuable intellectual property.

Scale Horizontally with Multiple Rollouters

The database-centric architecture supports multiple rollouter instances feeding a single trainer. Shard by task_id or run_id to parallelize environment diversity without synchronization overhead.

Comparison with Alternatives

Verdict: DART-GUI occupies a unique position—more opinionated and GUI-agent-specific than raw frameworks, more transparent and customizable than commercial solutions, and more infrastructure-complete than academic baselines.

FAQ

What hardware do I need to run DART-GUI?

Minimum: One GPU machine (A100 40GB or larger) for rollouter + trainer, plus one CPU machine for environment simulation. Recommended: Separate GPU machines for rollouter and trainer, with NVMe storage for the MySQL volume.

Can I use a different base model than UI-TARS-1.5-7B?

The codebase is architecturally compatible with any vision-language model, but you'll need to adapt the input/output tokenization and action space mapping. UI-TARS is the validated path.

Is DART-GUI suitable for non-GUI reinforcement learning?

The decoupled training and adaptive curation are domain-agnostic, but the action space definitions and reward shaping assume GUI interactions. General RL would require significant modification.

How does adaptive data curation actually work?

The README doesn't expose implementation details, but the schema suggests curation operates on reward, num_chunks, and used fields—likely prioritizing high-reward, diverse-length trajectories while filtering duplicates.

What's the relationship to verl and vLLM?

DART-GUI extends and specializes these projects. verl provides the RL algorithm framework; vLLM provides fast inference. DART-GUI adds GUI-agent-specific orchestration, data curation, and production infrastructure.

Can I contribute or request features?

The team explicitly solicits collaborations and GPU resource support. Contact Qing Li at dylan.liqing@gmail.com for research partnerships.

Is there a smaller model for prototyping?

The released checkpoint is 7B parameters. For faster iteration, consider modifying the training script for LoRA or QLoRA adaptation—though this isn't officially documented yet.

Conclusion: Why DART-GUI Demands Your Attention

The GUI agent landscape is crowded with promises and short on delivery. DART-GUI breaks that pattern by shipping battle-tested infrastructure alongside genuine algorithmic innovation. The decoupled training architecture eliminates pipeline stalls. The adaptive data curation extracts maximum learning from every compute dollar. The database-backed tracking transforms reproducibility from aspiration to enforcement.

This isn't a toy for Kaggle competitions. It's a production framework built by researchers who understand that breakthrough algorithms die without breakthrough infrastructure. The team's roadmap—merging with latest verl, codebase polishing—signals sustained commitment, not abandonware.

If you're serious about autonomous computer-use agents, fork the repository today. Experiment with the Docker setup. Trace the SQL schema. Run the OSWorld benchmark. And when your training pipeline finally converges on tasks that stumped your previous approaches, remember where you read about it first.

⭐ Star github.com/Computer-use-agents/dart-gui, read the paper, and join the teams who are already training the next generation of GUI agents the right way.

The future of computer-use AI isn't just about smarter models—it's about smarter training. DART-GUI proves you can have both.