Summary
The Huxley-Gödel Machine (HGM) is a self-improving coding agent that systematically searches a tree of codebase modifications, guided by a metric called Clade Metaproductivity (CMP). It is designed to overcome the Metaproductivity-Performance Mismatch — the observation that an agent’s immediate benchmark performance is a poor predictor of its potential for future self-improvement. HGM estimates CMP for each candidate agent and uses it as a reward signal, mimicking the behavior of a Gödel Machine under bounded resources. On SWE-bench Verified and Polyglot, HGM surpasses prior self-improvement methods while using fewer compute hours, and its optimized agents achieve human-level performance on SWE-bench Lite.
Key Points
- Identifies the Metaproductivity-Performance Mismatch: benchmark performance correlates poorly with an agent’s capacity for future self-improvement.
- Introduces Clade Metaproductivity (CMP), a metric that aggregates the downstream benchmark performances of all descendants of an agent in a self-modification tree.
- Proposes the Huxley-Gödel Machine, which estimates CMP and uses it to guide tree search, inspired by the self-referential optimization of the Gödel Machine and Huxley’s concept of a clade.
- Outperforms prior self-improving coding agent methods on SWE-bench Verified and Polyglot, requiring fewer CPU hours.
- Demonstrates strong transfer to other coding datasets and large language models; an agent optimized on SWE-bench Verified with GPT-5-mini achieves human-level performance on SWE-bench Lite with GPT-5.
Concepts
- Metaproductivity-Performance Mismatch: The discrepancy between an agent’s current benchmark score and its latent ability to generate highly capable descendants through self-modification. Existing methods that greedily select the best-performing agent fail because they ignore this mismatch.
- Clade Metaproductivity (CMP): A metric that measures the average or aggregated benchmark performance of all agents descended from a given agent. It serves as a proxy for the agent’s self-improvement potential.
- Gödel Machine: A theoretical machine that can rewrite its own code and provably optimize a utility function. HGM approximates this by using CMP as a stand-in for the utility it would calculate under idealized assumptions.