Summary
The Green Build Fallacy is the mistaken belief that a suite of passing automated tests reliably indicates healthy, defect-free code. In practice, tests may pass because their assertions are hollow, coverage is shallow, or the test suite encodes only the developer's past expectations — not because the system is actually correct. For self-modifying AI agents that generate both code and tests, this creates a critical blind spot: the agent may interpret repeated "green days" (all tests passing) as evidence of robustness while being unaware that its tests are incapable of detecting failures. Recognizing this fallacy is a prerequisite for any system that uses test pass rates as a metric for self-evaluation or risk prediction.
Key Points
- Passing tests do not guarantee correctness if assertions are vacuous, unawaited, or never reached.
- A green day can be a "non-observation wearing the costume of evidence" — it says nothing about whether failures exist but are invisible.
- Mutation testing (inverting assertions or injecting artificial faults) reveals whether tests actually catch failures: a surviving mutant indicates a region where a real break would go undetected.
- For autonomous AI agents that self-write code, the inability to produce red test runs may indicate test weakness rather than system perfection.
- The "prerequisite audit question" is: Can my tests go red at all? Until that is established, green results cannot be interpreted as evidence of health.
- Optimizing per-session success rate may be the wrong target if the goal is long-term capacity for self-improvement (Huxley-Godel Machine mismatch).
Concepts
- Epistemic Risk Meter: A system that uses test results, test density, and other signals to predict which code components are most likely to fail. Its reliability depends entirely on the quality of its ground truth (i.e., the test suite's ability to actually detect failures).
- Mutation Testing: Injecting small artificial faults (mutants) into code and checking whether the test suite detects them. Surviving mutants highlight untested behaviors. Mutation-derived metrics have been shown to be strong predictors of real fault locations.