Changelog defect-class transfer is a testing method: take a defect class that a vendor has just fixed or reported in its changelog, then probe whether a different tool exhibits the same class of defect. The unit of analysis is the abstracted failure pattern (e.g. "empty CA bundle accepted silently"), not the specific bug or patch. The method turns release notes into a cheap, prioritized source of test hypotheses for systems you own, and treats competitor release notes as a free source of adversarial test cases and design constraints.
Two worked transfer passes are documented here. The first transfers six defect classes from Claude Code's 2.1.318 changelog (Nov 4 2026) onto yoyo v0.4.0 (d834121a), classifying each outcome as already-conformant, open gap, partial, or not-applicable. The second scans Claude Code 2.1.319 and Codex CLI 0.167.0 (both dated 2026-11-06) for portable defect classes and triages them against yoyo at HEAD 612cd699 as already covered, relevant but not yet probed, or open gap.
Key Points
The changelog is treated as a catalog of failure modes someone found and cared enough to fix — evidence that the class is real and reachable, not just theoretical.
Each entry must be abstracted away from the vendor's implementation before it can be probed: the question is "can this class of misbehavior happen here?", not "does this tool have this bug?".
Transfer outcomes are not binary. The first pass uses a four-way classification: already conformant (target refuses/errors correctly), silent gap (target tolerates the bad state and says nothing), partial / unprobed variant (the base case is fixed but an adjacent path was never exercised), not applicable (the feature doesn't exist under the target's architecture). The second pass uses three verdicts: already covered, relevant but not yet probed, and open gap.
Silently-tolerated bad input is the most common transfer finding: the target "works" but fails to surface the problem, which is worse than a loud refusal.
Transfer is cheap and bounded: six classes probed in one pass, with only one variant left unprobed in the first pass.
A second-pass principle: failures and truncations are part of the tool-result contract, not side-channel noise; status fidelity requires that a command's reported outcome match reality.
In the second pass, the only confirmed open gap is the truncation-fidelity issue tracked as yoyo #1159, item 1.
Concepts
Defect class — an abstracted failure pattern, independent of any one codebase (e.g. "invalid config silently replaced by defaults when non-interactive").
Transfer target — the tool under test that receives the probe (here yoyo).
Named fix — an error message that tells the user the remedy, not merely that something is wrong; treated as the quality bar in several classes.
Silent skip — accepting input after discarding part of it without warning.
Conformance — the target already behaves at or above the vendor's post-fix standard.
Unprobed variant — a near-neighbor of a probed case that the probe did not exercise (a known coverage hole, not a known bug).
Deny-before-allow precedence — evaluation order in a permission system where any matching deny overrides a narrower allow; deny is authoritative and never escalates to a prompt.
denyInsteadOfPrompting — policy that converts what would be an interactive prompt into an automatic denial so a non-interactive turn can continue instead of hanging.
Status fidelity — the requirement that a command's reported outcome match reality; reporting success over a failed write is the canonical violation.
Bounded model-facing output — capping rows/lines/size of data rendered for the model (e.g. top-10 rows per list) so tables stay legible.
Compaction accounting — token/context accounting that must include output retained from failed turns and must tolerate missing usage metadata.
Truncation marker completeness — a truncation notice must state how much was omitted; a bare "output truncated" leaves the model unable to reason about what it lost.
Head-only vs head+tail truncation — keeping only leading bytes destroys trailing verdicts (e.g. cargo test results); head+tail preserves both ends.
Details
The transfer procedure
Rendering diagram…
The method is deliberately shallow per item: one probe per class, no attempt to reproduce the vendor's original bug. Its value is breadth — surveying many classes quickly to find where the target is silent. The same abstraction procedure carries over to the second pass's three-way triage of already covered, relevant but not yet probed, and open gap.
First transfer pass: Claude Code 2.1.318 → yoyo v0.4.0
#
Defect class
Claude Code 2.1.318 post-fix behavior
yoyo v0.4.0 result
1
Damaged / empty CA bundle
Uses the readable certs, warns on stderr
Partial gap: empty SSL_CERT_FILE is refused at startup with a named fix (better than vendor); but a bundle containing a garbage PEM block — first or last — is accepted with no notice that the block was skipped
2
Invalid config silently replaced by defaults on non-interactive launch
Fails, and explains how to restore
Conformant: malformed .yoyo.toml exits 2 with per-line diagnostics
3
-p reports success when an API error truncates a reply while a hook runs
Reports the failure
Partial: Day 250–251 #1026 aligned is_error and process exit status for cut/refused replies; the hook-mid-turn variant was never probed
4
Oversized-image error names the wrong limit
Names the correct limit
Conformant: Day 250 now reports yoagentmax_image_bytes
5
Fast-dying stdio MCP server re-probed on every start
(fixed)
Open gap: a pre-flight listing plus a real connect means a double spawn, and no memory of the failure persists across runs
6
-p JSON carries credential_expires_at
(added)
Not applicable: yoyo uses API-key auth, which has no such expiry surface
Reading the first-pass results
Class 1 is the most instructive finding. yoyo is stricter than the vendor on the empty-bundle case — it refuses and names the remedy — yet still silent on the adjacent case of a partially corrupt bundle. A single probe of "empty file" would have scored a clean pass and hidden the actual weakness; the transfer only paid off because the class was probed at more than one point on its input spectrum.
Class 5 shows a class where conformance requires state across runs. The double-spawn pre-flight pattern satisfies "detect the broken server" but not "stop re-paying for it", so the class remains open despite the check existing.
Class 6 marks the boundary of the method: a defect class presupposing a feature the target lacks generates no test, and should be recorded as not-applicable rather than as a pass.
Second transfer pass: Claude Code 2.1.319 + Codex 0.167.0 → yoyo HEAD 612cd699
Six transferable defect rows were identified:
#
Source
Defect class
yoyo verdict
1
Claude Code
Broader deny rule silently refuses a narrower session ask/allow rule
Already covered — PermissionConfig::check_with_suspended (src/config.rs:23) checks deny before allow
2
Claude Code
permissions.denyInsteadOfPrompting: a call that would block on a prompt is denied and the turn continues
Relevant to -p/non-tty mode; not probed
3
Claude Code
/rename reported plain success even though the save failed
Status-fidelity class; recurring in yoyo's own commands
4
Claude Code
/context keeps only the 10 largest rows per list so the model-facing table stays short
Bounded-output technique worth mirroring
5
Codex 0.167
Automatic compaction must account for retained output from failed responses and responses with missing token-usage data
Open question: does yoyo count output from errored turns?
6
Codex 0.167 / older Codex #2183
Standalone web-search failures must be returned to the model; truncation messages must appear in tool results so incomplete output is not mistaken for complete
Partially violated by yoyo's truncation path
The only confirmed open gap in this pass is the truncation-fidelity issue tracked as yoyo #1159, item 1.
Permission semantics (rows 1–2)
In Claude Code, a session-scoped exact Bash ask/allow rule is refused rather than asked when a broader deny rule covers the same command. The generalized rule: a narrower allow never overrides a broader deny. yoyo satisfies this because PermissionConfig::check_with_suspended (src/config.rs:23) evaluates deny before allow.
The related Claude Code behavior, permissions.denyInsteadOfPrompting, denies any call that would otherwise wait on a prompt and lets the turn proceed. This matters for yoyo's -p / non-tty behavior, where a prompt has no human to answer it; yoyo has not yet been probed for this case.
Status fidelity (row 3)
Claude Code's /rename reported plain success when the underlying save failed. This is the same success-over-failed-write bug class yoyo keeps finding in its own commands, and is worth a standing test oracle: any command whose persistence step fails must report failure.
Output bounding and truncation (rows 4, 6)
Claude Code's /context keeps the 10 largest rows per list so the table the model reads stays short — a simple, transferable bounding rule for any model-facing listing.
Codex 0.167 returns standalone web-search failures to the model (#58147), and older Codex (#2183) included the truncation message inside tool results so the model is not confused by incomplete output. The principle: failures and truncations are part of the tool result contract, not side-channel noise.
yoyo's concrete gap (#1159, item 1):StreamingBashTool (src/tools.rs:537-546) keeps only the head past 256 KB, emitting the marker ... (output truncated) with no byte count. The outer with_truncation wrapper (format/output.rs, truncate_tool_output) does head+tail by lines — but it operates on text already head-cut by the inner tool, so the real tail (the cargo test verdict) is already gone. yoagent 0.27.5's bash, by contrast, keeps head+tail and emits ... (output truncated: N bytes omitted) ....
Rendering diagram…
The fix direction implied by the sources is to perform head+tail retention inside the streaming tool, before any wrapper runs, and to have the marker report the omitted byte count.
Context accounting (row 5)
Codex 0.167 changed automatic compaction to account for retained output from failed responses and from responses with missing token-usage data. The transferable check for yoyo: whether its context accounting includes output from errored turns, and whether it degrades gracefully when usage data is absent.