---
title: "Evolving Agent Interfaces"
authors:
  - "Saaket Agashe"
  - "Yuzhe Yang"
  - "Gonzalo Gonzalez-Pumariega"
  - "Bhuvan Chandra Koduru"
  - "Chih-Lun Li"
  - "Jiachen Yang"
  - "Ang Li"
  - "Xin Eric Wang"
year: 2026
venue: "arXiv preprint (Simular technical report)"
arxiv: "2610.28601"
doi: null
source: "https://arxiv.org/abs/2610.28601"
topics:
  - "continual-agent-skills"
  - "autoresearch-experiment-selection"
seed_rank: 2385
seed_batch: "frontier-2026-10-22"
reviewed: "2026-10-22"
pool: "agents"
relevance_score: 8
cites:
  - title: "Evolving Agent Interfaces"
    url: "https://arxiv.org/abs/2610.28601"
    year: 2026
    arxiv: "2610.28601"
    doi: null
  - title: "OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments"
    url: "https://arxiv.org/abs/2404.07972"
    year: 2024
    arxiv: "2404.07972"
    doi: null
  - title: "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering"
    url: "https://arxiv.org/abs/2405.15793"
    year: 2024
    arxiv: "2405.15793"
    doi: null
see:
  - 192-osworld-benchmarking-multimodal-agents-for-open-ended-tasks-
  - 001-swe-agent-agent-computer-interfaces-enable-automated-softwar
---

# Evolving Agent Interfaces

## One-sentence takeaway

Agent S4 treats a computer-use agent's interfaces (actions, observation formats and when-to-use-what knowledge) as a code repository that a meta-agent evolves from unlabeled synthetic task trajectories, with population-based selection on hidden validation tasks; with the evolved repository a Qwen3.8-27B agent reaches 21.3% binary success on the 108-task OSWorld V2, 3.8× the same model with fixed interfaces, at about $1.10 per task.

## Why it matters here

SWE-agent (card 001) showed that the agent-computer interface shapes performance; this paper makes that interface the object of automated search while holding the model and harness fixed. For Broadside it is a recipe for growing tool surfaces (shell shortcuts, file helpers, app-specific workflows) from our own agents' traces, plus a discipline worth copying: the evolving agent never sees validation tasks, and validation only picks parents.

## Key ideas

- **Interface evolution, not skill distillation.** Skills compose the preset actions; evolved interfaces can add new ones (for example a direct shell instead of typing into a GUI terminal), so they can extend what the agent can do rather than only reorder it.
- **Population search.** Descendants come from mutation of one repository or crossover of two, can add, revise or remove interfaces and knowledge, and join the population next to their parents; the meta-agent reads training trajectories only, with no success labels.
- **OSWorld V2.** 21.3% binary success for Qwen3.8-27B with the evolved repository, versus 13.0% for GPT-5.5 and 20.6% for Claude Opus 4.8 (batched actions) at about 4.3% and 1.5% of their per-task cost ($25.50 and $72.40).
- **ScienceBoard.** In a separate evolution run the evolved repository lifts success from 41.4% (Qwen CUA's human-designed interface) to 58.6%.
- **Ablations.** Population evolution and linear evolution tie on binary success with a scratch VM for the meta-agent (19.44%), with population ahead on partial score; the best binary score (21.30%) came without the scratch VM.

## Caveats

Vendor report (Simular) on a 108-task benchmark, so a few tasks move the headline number. Synthetic training tasks were generated by frontier models (GPT-6 Astra and GPT-5.6-Sol), so the cheap task agent depends on expensive models during evolution, and that evolution cost is not in the per-task figure. Closed-model baselines use their own harness settings. The variant with the higher partial score (51.06%) is not the one with the higher binary score.

## Links

- arXiv abstract: https://arxiv.org/abs/2610.28601
- PDF: https://arxiv.org/pdf/2610.28601
- Code: https://github.com/simular-ai/agent-s