SEAgent: The Self-Evolving AI That Masters Your Computer

SEAgent: The Self-Evolving AI That Masters Your Computer

What if your AI assistant didn't just follow instructions—it actually learned how to use your computer better every single day?

Here's a painful truth every developer building autonomous agents knows: most computer-use AI systems are glorified script runners. They follow predefined workflows, break when interfaces change, and require constant human hand-holding. You've spent weeks fine-tuning prompts, crafting DOM selectors, and building brittle automation pipelines—only to watch your agent fail spectacularly when a button moves three pixels to the left.

The cost? Burned engineering hours, frustrated users, and agents that never truly improve. Traditional approaches rely on massive static datasets and supervised fine-tuning, creating systems that memorize rather than understand. They don't adapt. They don't evolve. They certainly don't learn from their own mistakes.

Enter SEAgent—a radical departure from everything you thought you knew about computer automation. This isn't another wrapper around GPT-4 with some PyAutoGUI hacks. This is a self-evolving agent that autonomously learns from its own experience, continuously improving its ability to navigate GUIs, execute complex software workflows, and accomplish real-world tasks without human intervention.

Built by researchers from top institutions and accepted at ICML 2026, SEAgent represents a fundamental shift: from static, data-hungry models to dynamic, experience-driven systems that get smarter with every interaction. And the best part? It's fully open-source, with pretrained models you can deploy today.

Ready to stop babysitting your agents? Let's dive into what makes SEAgent genuinely different.

What is SEAgent?

SEAgent (Self-Evolving Agent) is the official implementation of the ICML 2026 paper "SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience" by Zeyi Sun and collaborators including researchers from leading AI labs. The project is publicly available at github.com/SunzeY/SEAgent.

At its core, SEAgent solves a deceptively simple problem that has stumped the AI community: how do you build an agent that improves at using computers without requiring humans to label thousands of new examples?

The breakthrough lies in SEAgent's autonomous learning loop. Unlike conventional computer-use agents that train once on fixed datasets and then freeze, SEAgent operates in continuous cycles of:

1. Task Generation — A curriculum generator creates increasingly complex training tasks automatically
2. Trajectory Sampling — The agent attempts these tasks, generating rich interaction histories
3. World State Evaluation — A dedicated "World State Model" judges success and failure without human labels
4. Self-Training — Successful trajectories become SFT data; contrasting outcomes fuel RL optimization

This architecture decouples agent improvement from expensive human annotation. The system generates its own training data, evaluates its own performance, and iteratively refines its policies—all while you sleep.

Why it's trending now: The release of SEAgent-1.0-7B and the World State Model-7B on Hugging Face has sparked intense interest because it delivers on the promise of truly autonomous improvement. While competitors like UI-TARS provide strong initialization points, SEAgent goes further by enabling continuous self-evolution. In an era where AI agents are moving from demos to production, the ability to self-improve without constant re-engineering isn't just nice to have—it's existential.

Key Features That Separate SEAgent From the Pack

🔄 Autonomous Curriculum Learning

SEAgent doesn't need you to hand-craft training scenarios. Its curriculum generator automatically produces task sequences that progressively increase in difficulty, ensuring the agent develops robust generalization rather than brittle memorization. The system adapts task complexity based on current performance, hitting the sweet spot between learnable and challenging.

🧠 World State Model for Self-Evaluation

The crown jewel of SEAgent's architecture. This dedicated 7B parameter model analyzes intermediate screenshots and action histories to determine whether the agent is making progress toward its goal. This eliminates the annotation bottleneck—no more paying humans to label thousands of trajectories. The World State Model achieves strong performance on AgentRewardBench, validating its judgment quality against human evaluations.

🎯 Dual Training Paradigm (SFT + RL)

SEAgent leverages both supervised fine-tuning and reinforcement learning in a unified framework:

• SFT (Behavior Cloning): Positive trajectories directly train the policy
• RL (GRPO Optimization): Contrasting positive and negative trajectories enables policy gradient optimization through the R1-V framework

This hybrid approach captures both the stability of imitation learning and the exploration benefits of reinforcement learning.

🖥️ Real GUI Environment Interaction

Built on top of OSWorld, SEAgent operates in genuine operating system environments—not simulated or mocked interfaces. It processes screenshots, accessibility trees, and executes actual mouse and keyboard actions. This grounding in real software (VS Code, browsers, file managers) ensures learned behaviors transfer to practical deployments.

⚡ Production-Ready Inference via vLLM

SEAgent deploys efficiently through vLLM serving, enabling high-throughput, low-latency operation with tensor parallelism. The 7B parameter size hits a pragmatic sweet spot—powerful enough for complex reasoning, small enough to run on modest GPU configurations.

Real-World Use Cases Where SEAgent Dominates

1. Automated Software Testing & QA

Traditional UI testing requires engineers to manually script interactions that break with every interface update. SEAgent learns application behaviors autonomously, adapting to UI changes by experiencing them. Deploy it across your product suite, let it explore functionality, and receive comprehensive test coverage that improves nightly without maintenance overhead.

2. Enterprise Workflow Automation

Finance teams processing invoices, HR onboarding sequences, customer service ticket routing—these processes live in complex software ecosystems that resist simple API integration. SEAgent navigates the actual applications your employees use, learning optimal workflows through experience rather than requiring brittle RPA implementations that shatter when Salesforce updates its layout.

3. Accessibility Assistance at Scale

For users with motor impairments, SEAgent can evolve into personalized computer assistants that learn individual preferences and software configurations. The self-improvement loop means the system adapts to each user's unique environment and needs without requiring specialized engineering for every deployment.

4. Research & Competitive Intelligence

SEAgent's ability to autonomously explore software environments makes it invaluable for systematic data collection across web applications, document processing pipelines, and multi-step research workflows. The curriculum generator ensures comprehensive coverage while the World State Model maintains quality control.

Step-by-Step Installation & Setup Guide

Getting SEAgent running takes minutes, not days. Follow this complete setup:

Environment Creation

# Create dedicated conda environment with Python↗ Bright Coding Blog 3.11
conda create -n seagent python=3.11
conda activate seagent

# Run the automated setup script
bash setup.sh

The setup.sh script handles dependency installation including PyTorch, vLLM, OSWorld environment setup, and model serving infrastructure.

Model Deployment for Inference

Deploy the pretrained SEAgent-1.0-7B model using vLLM's optimized serving engine:

# Launch SEAgent model server with tensor parallelism
vllm serve Zery/SEAgent-1.0-7B \
    --served-model-name "ui-tars-1.0-7b" \
    --host 0.0.0.0 \
    --port "${PORT}" \
    --tensor-parallel-size "${TP}" &

# Configure client connection
export UI_TARS_1_0_URL=http://YOUR.IP.ADDRESS:PORT/v1
model_name="ui-tars-1.0-7b"

Critical configuration notes:

• Replace YOUR.IP.ADDRESS with your server's actual IP
• Set PORT to your desired serving port (commonly 8000-8002)
• TP (tensor parallelism) should match your available GPU count—use 1 for single-GPU, 2-4 for multi-GPU setups
• The --served-model-name parameter maintains compatibility with UI-TARS client code

Running Inference on OSWorld

# Navigate to OSWorld benchmark directory
cd OSWorld

# Execute evaluation with optimized parameters
python run_multienv_uitars_1_0.py \
    --headless \                          # Run without display for server deployment
    --observation_type screenshot \       # Use visual observations
    --model ui-tars-1.0-7b \             # Target deployed model
    --result_dir ./results_en_test_1_0/all_l15_h5/${model_name} \
    --num_envs 8 \                        # Parallel environment count
    --sleep_after_execution 2.0 \         # Stabilization delay between actions
    --max_tokens 1000 \                   # Generation budget per step
    --top_p 0.9 \                         # Nucleus sampling parameter
    --temperature 1.0 \                   # Exploration vs exploitation
    --max_trajectory_length 15 \          # Maximum steps per episode
    --history_n 1                         # Context window for previous actions

For full OSWorld evaluation, use the batch script after updating IP configuration:

# Update IP addresses in script to match your deployment
bash run_multienv_uitars_1_0_full.sh

Training Setup

SEAgent supports two training modes:

# Reinforcement Learning via GRPO (Group Relative Policy Optimization)
src/r1-v/run_grpo_gui_8_7b.sh

# Supervised Fine-Tuning for behavior cloning
sft.sh

The GRPO implementation builds on R1-V, enabling efficient policy optimization without requiring a separate value function estimator.

REAL Code Examples: Inside SEAgent's Self-Evolution Loop

Let's examine the actual implementation patterns that power SEAgent's autonomous improvement. These examples are extracted directly from the repository and represent production-ready code.

Example 1: Curriculum-Based Task Generation

SEAgent's learning begins with intelligent task creation. Here's how the curriculum generator produces training scenarios:

# Step 1: Deploy foundation models for task generation
# Qwen2.5-72B generates diverse task instructions
CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve "Qwen/Qwen2.5-72B-Instruct" \
    --served-model-name qwen72b \
    --port 8002 \
    --tensor-parallel-size 4

# World State Model evaluates task feasibility and difficulty
CUDA_VISIBLE_DEVICES=4,5,6,7 vllm serve "Zery/CUA_World_State_Model" \
    --served-model-name gui_judge \
    --port 8001 \
    --tensor-parallel-size 4

This dual-model architecture separates generation (creative task proposal) from evaluation (quality and difficulty assessment), preventing the curriculum from collapsing into trivial or impossible tasks.

# Step 2: Generate curriculum tasks for specific software
python task_buffer/task_buffer_update_from_qwen.py \
    --judge_model gui_judge \       # Which model evaluates task quality
    --phase -1 \                    # -1 = initial curriculum phase; increases as agent improves
    --software vscode \             # Target application (vscode, chrome, etc.)
    --base_result_dir results_en_phase_qwen

Key insight: The --phase parameter implements curriculum scheduling. Phase -1 generates foundational tasks; as the agent masters each phase, the system automatically advances to more complex scenarios. This eliminates manual curriculum design.

Example 2: Trajectory Sampling with Evolution

Once tasks exist, SEAgent executes them to generate learning experiences:

# Deploy actor model for trajectory generation
CUDA_VISIBLE_DEVICES=0 python -m vllm.entrypoints.openai.api_server \
    --served-model-name ui-tars \
    --port 8001 \
    --model bytedance-research/UI-TARS-7B-DPO \  # Strong initialization point
    --limit-mm-per-prompt image=5 \               # Multi-image context for GUI state
    -tp 1

The image=5 limit is crucial—GUI agents must process multiple screenshots showing interface evolution, not just single frames.

# Execute and collect trajectories across environments
export VLLM_BASE_URL=http://YOUR.IP.ADDRESS:PORT/v1

for software in vs_code; do
    python run_multienv_uitars_new_traj_evolve.py \
        --headless \
        --observation_type screenshot_a11y_tree \  # Combined visual + accessibility input
        --model ui_tars_7b_dpo \
        --test_all_meta_path ./task_buffer/task_buffer_qwen_${software}_phase0.json \
        --result_dir ./results_en_phase0/7b_new_traj_multi_env=8_${software}_new_fix_nimg=1_maxtraj=15_t=0.0_r0 \
        --num_envs 8 \                               # Parallel exploration for diversity
        --max_tokens 1000 \
        --top_p 0.9 \
        --temperature 1.0 \                          # High temperature encourages exploration
        --max_trajectory_length 15 \                 # Prevent infinite loops
        --history_n 1 \                              # Single-step history for efficiency
        --software ${software}
done

The evolution mechanism: The _evolve suffix in the script name indicates this isn't mere execution—it's exploratory sampling where the agent attempts variations, encounters edge cases, and generates diverse behavioral data. The temperature=1.0 setting prioritizes exploration over exploitation during this phase.

Example 3: Autonomous Training Data Generation

Here's where SEAgent's self-supervision shines—converting raw trajectories into training signals without human labels:

# Step 1: Deploy World State Model for trajectory judgment
CUDA_VISIBLE_DEVICES=4,5,6,7 vllm serve "Zery/CUA_World_State_Model" \
    --served-model-name gui_judge \
    --port 8001 \
    --tensor-parallel-size 4

# Step 2: Batch evaluate all generated trajectories
bash judge_full_process.sh

# Step 3: Generate training datasets from judged trajectories
python gen_sft.py

The gen_sft.py script produces two critical outputs:

• 7b_positive_xxx.json — Successful trajectories for supervised fine-tuning (behavior cloning)
• 7b_negative_xxx.json — Failed trajectories for contrastive RL training

# Optional: Visualize generated training data for quality inspection
python visualize_data.py

Why this matters: Traditional agent training requires expensive human annotators to label thousands of screenshots with correct actions. SEAgent's World State Model performs this evaluation autonomously, reducing annotation cost to zero while scaling to unlimited data generation.

Example 4: World State Model Evaluation

Validate your deployment's judgment quality against established benchmarks:

# After installing AgentRewardBench and replacing evaluation scripts:
bash run_judge.sh

For OSWorld-specific validation:

python OSWorld/eval_osworld_bench.py

These evaluations confirm that SEAgent's self-generated labels match human judgment quality—a prerequisite for trustworthy autonomous learning.

Advanced Usage & Best Practices

Scaling Curriculum Complexity

Progress through phases systematically. Don't skip to phase 5 when phase 0 shows 40% success—premature difficulty causes policy collapse. Monitor results_en_phase*/ directories and advance only when success rates exceed 80%.

Multi-GPU Optimization

The repository's tensor parallelism configuration (--tensor-parallel-size 4) is optimized for 80GB A100s. For 40GB A6000s or RTX 4090s, reduce to TP=2 or deploy separate models across GPUs. The task generation and judgment models can run on different GPU groups for pipeline parallelism.

Temperature Scheduling

During trajectory sampling, use temperature=1.0 for exploration phases, then anneal to temperature=0.3 for final policy extraction. This two-stage approach maximizes behavioral diversity during data generation while ensuring deployment reliability.

Headless Deployment

The --headless flag is essential for server deployment but disable it during debugging—visual observation of agent behavior reveals failure modes invisible in logs. Use sleep_after_execution 2.0 minimum; GUI state transitions often require 1-2 seconds to stabilize.

Checkpoint Management

SEAgent's evolution generates substantial trajectory data. Implement automated cleanup: retain only top-performing trajectories (judged by World State Model confidence >0.9) and archive older phases to cold storage.

Comparison with Alternatives

The verdict: Choose SEAgent when you need continuous improvement without continuous engineering. UI-TARS provides excellent initialization but requires manual retraining for new domains. SEAgent's self-evolution loop eliminates this maintenance burden, making it ideal for long-running deployments in dynamic environments.

FAQ: What Developers Ask About SEAgent

What hardware do I need to run SEAgent?

Minimum: 1× GPU with 24GB VRAM for inference (RTX 3090/4090). Recommended: 4× A100 80GB for full training pipeline with curriculum generation and RL. The 7B model is specifically sized for accessible deployment.

Can SEAgent learn entirely new software applications?

Yes—this is the core innovation. Provide the application name to the curriculum generator, and SEAgent autonomously explores, generates tasks, and learns effective policies. Initial performance may be weaker than on familiar software, but the self-improvement loop guarantees progression.

How does the World State Model avoid reward hacking?

The World State Model evaluates based on screenshot sequences matching goal states, not just terminal outcomes. This dense reward signal prevents shortcut exploitation. Additionally, the model was validated on AgentRewardBench to confirm alignment with human judgments.

Is SEAgent suitable for production automation?

For research and internal tooling: immediately. For customer-facing automation: implement human-in-the-loop verification for high-stakes actions. The self-evolution improves reliability over time, but initial deployments benefit from oversight.

How does SEAgent compare to fine-tuning GPT-4V with my data?

GPT-4V fine-tuning requires expensive API calls and labeled data. SEAgent generates unlimited training data for free, runs locally without API dependencies, and continuously improves—advantages that compound over deployment lifetime.

Can I use SEAgent with my existing UI-TARS deployment?

Absolutely. SEAgent uses UI-TARS as its actor model initialization. The migration path is: deploy SEAgent-1.0-7B instead of UI-TARS-7B-DPO, enable curriculum generation, and activate the evolution loop. Existing infrastructure remains compatible.

What license applies to SEAgent?

Code is Apache 2.0 licensed. Data and models are CC BY-NC 4.0 for research use only. Commercial licensing requires contacting the authors. Usage must comply with OpenAI's terms of service.

Conclusion: The Future of Computer Automation is Self-Evolving

SEAgent isn't incrementally better than existing computer-use agents—it's architecturally different. By closing the loop between experience generation, autonomous evaluation, and self-supervised training, it achieves something unprecedented: an agent that genuinely improves without human intervention.

The implications are profound. We're moving from an era where AI capabilities are bounded by dataset collection budgets to one where agents bootstrap their own growth. SEAgent's 7B parameter size proves this doesn't require massive models—just intelligent architecture.

For practitioners, the message is clear: stop building brittle automation that breaks with every software update. Deploy SEAgent, let it experience your applications, and watch it evolve solutions you never explicitly programmed.

The repository is actively maintained with training code, inference pipelines, and pretrained models ready for immediate use. The research team has also released the World State Model for independent evaluation and extension.

Your next step: Clone github.com/SunzeY/SEAgent, deploy the 7B model, and run your first curriculum generation. In 48 hours, you'll have training data that would have taken weeks to annotate manually. In two weeks, you'll have an agent that handles your software workflows more reliably than when you started.

The age of self-improving AI agents isn't coming. It's already here, and it's called SEAgent.

Found this breakdown valuable? Star the repository, share with your agent-building team, and follow the project for upcoming training code releases based on OpenRLHF.