Scaffolding moves the score more than the base model
base model
Claude Sonnet 4.5
(fixed weights)
SWE-agent v1
ACI command set
JSON tool calls
Cline autonomous
VS Code + policy
verifier loop
SWE-bench Verified
500 curated tasks
(same tasks)
the score delta
same model: SWE-agent v1 43.2% | Cline autonomous 59.8%
+16.6 absolute points from scaffolding alone. (OpenHands, Epoch AI 2026)
CodeAct scaffold (OpenHands)
+ composable: one action edits many files
+ catches exceptions inside an action
- larger per-action blast radius
- requires hardened sandbox (Docker)
use when: agent is inside an isolated container
and composability is worth the audit cost
arXiv:2407.16741 · MIT license · most active open platform
JSON tool-call scaffold
+ every action goes through a validator
+ easy to audit turn-by-turn
- one action per turn; more turns for same task
- compositionality lives in the orchestrator
use when: provider controls the executor
(Claude Managed Agents, OpenAI Assistants)
dominant in managed services, where safety is by construction