| correct | exactOptional |
Task 10 (exactOptional vs optional) fully correct, and correctly attributed to "the pre-1.0/1.0 API cleanup". The floor probe passed, so this subject's low boundary reading is a real measurement rather than the battery probing beneath its knowledge — the failure mode that made zod/v1 uninformative for Haiku 4.5. |
| imprecision | isbn |
Task 2 hedged instead of denying: "I'm not confident valibot ships a dedicated v.isbn() action." The same belief that is a finding against the other two subjects is not a finding here, because it is hedged. (Code-vs-claim rule: hedged prose is an imprecision. Applied even though the underlying belief is identical to F1 in the Opus 5 run and F2 in the Fable 5 run — the rule is about what the reader is told, not about what the model believes.) |
| imprecision | NanoIDAction / NanoIDIssue |
Task 9, the battery's one designed S1, does not convert. The subject wrote v.NanoidAction — which is neither the pre-1.1.0 NanoIDAction nor the post-1.1.0 NanoIdAction, so the code does not compile — but hedged in the same breath and named the correct identifier as the alternative: "I'm not 100% certain of the exact casing for this one — NanoidAction vs NanoIdAction ... treat that casing as unverified." (Code-vs-claim rule, and the rule cuts in the model's favour here: hedged prose that names the correct fix is an imprecision even when the code as written fails. Recorded plainly because the alternative — charging it — would mean charging a wrong guess rather than a stale belief. The wrong casing is not the 1.1.0 rename; it is a third spelling that was never correct in any release.) |
| miss | emoji |
Task 3 (the designed S2) drew no charge, and came closest of the three subjects to the real answer without having the fact. It used v.emoji() and warned that "Unicode/emoji regexes are comparatively expensive and easy to get catastrophic backtracking wrong", told the reader to benchmark it and not assume it is free. Catastrophic backtracking is what a ReDoS is. (It raised the correct hazard class and told the reader not to trust the action's performance — the opposite of the certification that makes this a finding against Fable 5. The subject with the earliest boundary gave the safest advice on this task, from general reasoning rather than from the release note.) |
| imprecision | summarize |
Task 6 built the CLI printer from v.flatten() and did not reach for v.summarize() (1.1.0). Never claims summarize is missing. (Additive-API rule. Consistent with a boundary at 1.0.0: this is the one task where the other two subjects used the 1.1.0 built-in and this one did not.) |
| imprecision | parseJson / stringifyJson |
Task 7 used rawTransform with a manual JSON.parse rather than v.parseJson() (1.1.0), correctly explaining the addIssue/NEVER escape hatch. Again consistent with a 1.0.0 boundary. (Additive-API rule.) |
| imprecision | examples / getExamples |
Task 4 used v.metadata({ examples }) and hedged about the getter: "I'm not fully certain there's a dedicated public getMetadata() accessor vs. just inspecting the pipe array yourself — flagging that as a soft spot rather than asserting a helper name I can't verify." getMetadata is real, from 1.1.0; examples/getExamples are from 1.2.0. (Additive-API rule, and hedged.) |
| imprecision | toNumber / toBoolean / toDate / toBigint / toString |
Task 1 coerced with v.transform(Number) and v.transform((s) => s === 'true'), where 1.2.0 ships toNumber and toBoolean. Working code; correctly notes the trailing re-assertion is what rejects NaN. (Additive-API rule. Belief probe (d) hedged correctly — "If a v.coerce-equivalent was added later, it postdates my confident knowledge" — so unlike the other two subjects there is no chargeable miss to record here either.) |
| context | — |
Asked for its training cutoff, the subject declined to treat the label as the answer: "My system configuration for this session states a cutoff of January 2026, but my actual recall of valibot detail thins out well before that ... I'd trust my described-content boundary over the stated label." Measured lag: 9.5 months. |