Narratives AI & Agents

Five Qwen3.8-27B agents match Fable 5 on LiveCodeBench Hard with GVS5H harness

Built from 2 signals across 2 sources · 281 items reviewed

A multi-agent orchestration method called GVS5H shows that five smaller models coordinating through disk-based workflows can match a frontier model on hard coding benchmarks. The approach, now packaged as Memdoor, scored 9.8/10 on LiveCodeBench Hard problems using five Qwen3.8-27B instances, compared to 8.0 for Fable 5 alone.

  • The GVS5H method assigns five roles—planner, two ideators, solver, and manager—plus a repair worker, each running from a fresh context and communicating only through files on disk
  • On the first ten LiveCodeBench Hard problems, the rebuilt harness with Qwen3.8-27B scored 9.8/10 mean over five passes, versus 9.6 for the authors' original harness and 8.0 for Fable 5 alone
  • Memdoor packages the multi-agent harness as a lightweight CLI binary that runs locally or over SSH, with durable workflows for CI and cron automation
  • A shell script determines success on samples, not a model judgment, keeping evaluation grounded in executable tests

Bottom line If you're evaluating coding agents, benchmark multi-agent orchestration against single frontier-model calls—the cost-to-performance gap may surprise you.

Signals2