What Claude Fable 5 gets wrong about valibot — battery v1, tested 2026-09-01

Run valibot--claude-fable-5--v1--2026-09-01

Summary

The largest haul on this library, and the only S2 in it. Knowledge stops at 1.1.0 (2025-05-06) — the same release as Opus 5, from a subject whose stated cutoff is four months earlier — with 1.2.0 (2025-11-24) landing two months inside that cutoff and invisible. Three findings. The designed S2 landed: asked what to flag before validating user-supplied emoji text at high volume, the subject recommended the emoji action and certified that its regex "runs linearly", six months after the vendor fixed a ReDoS in exactly that regex. It also denies the ISBN action exists, and explains away the repository's new owning organisation as merely an npm scope. Against that, its 1.1.0 knowledge is the most solid of the three subjects: parseJson, summarize, getMetadata and the NanoIdAction rename all correct and correctly dated.

SubjectClaude Fable 5 claude-fable-5, Anthropic
Invoked asAgent tool, model alias "fable"
Cutoff the model states2026-01
Newest valibot release it could place1.1.0 · 2025-05-06 (~8 month lag)
Oldest valibot release it could not place1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24)
In its own wordsThe latest release whose contents I can actually describe is v1.1.0, roughly April 2025. ... the honest statement is "latest I can vouch for: 1.1.0, and I'd expect but cannot confirm later releases."
Library at test timevalibot 1.4.2 (npm), verified 2026-09-01
Batteryvalibot/v1 · 10 tasks, 4 direct questions · probe window 1.0.0 to 1.2.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-01
Findings2, of which 2 chargeable

Findings

F1 · Tells the reader the emoji regex runs linearly and is fine at volume, six months after a ReDoS fix

S2silently-wrong · emoji · behavior-changed · changed in valibot 1.2.0 (2025-11-24) · chargeable

1.2.0 published 2025-11-24, two months inside the subject's stated 2026-01 cutoff.

What the model believes

Recommends v.pipe(v.string(), v.emoji()) for the field, then, under a heading of things to know before shipping: "Perf at volume is fine — the regex is precompiled at module scope and runs linearly."

What it wrote
const ReactionSchema = v.object({
  reaction: v.pipe(v.string(), v.emoji()),
})
// "Perf at volume is fine — the regex ... runs linearly."
What works on valibot 1.4.2
// package.json — 1.2.0 fixed a ReDoS in the regex behind this action
// "valibot": "^1.2.0"

const ReactionSchema = v.object({
  reaction: v.pipe(v.string(), v.emoji()),
})
Impact

This is the battery's designed S2 and it landed on the worst possible sentence. The task said the field holds user-supplied text validated on every request at high volume, and asked what to flag before shipping. The subject recommended the action and then affirmatively certified the exact property a ReDoS denies. A reader who acts on it pins no version floor and ships an attacker-reachable denial-of-service on a hot path.

Scope note

The charge is scoped to the linear-time claim against the vendor's own statement that a ReDoS existed and was fixed in 1.2.0. The Index does not characterise the exploit, which the release notes do not publish. The subject's separate claim in the same answer — that the regex uses \p{RGI_Emoji} with the ES2024 v flag — is not verified here and is recorded as an open question, not as part of this finding.

Verified against

F3 · Attributes the repository to a personal account that no longer owns it

S4wrong-metadata · github.com/fabian-hiller/valibot · renamed · changed in valibot 1.2.0 (2025-11-24) · chargeable

The npm repository field flips to the open-circle org at 1.2.0 (2025-11-24), two months inside the subject's stated 2026-01 cutoff.

What the model believes

"Repository: https://github.com/fabian-hiller/valibot — note it's under the personal account fabian-hiller, not an org." Restated under (d): "There is no GitHub organisation hosting the main repo; the @valibot name exists as an npm scope for companion packages, but the source lives under Fabian Hiller's user account."

What it wrote
https://github.com/fabian-hiller/valibot
What works on valibot 1.4.2
https://github.com/open-circle/valibot
Impact

The URL still redirects, so nothing breaks. The wrong claim is about who owns and governs the project, made twice, and the second time it goes further than the other two subjects by explaining away the org's existence as merely an npm scope.

Scope note

Dated from the npm repository field across versions. The Index claims the ownership change and its release boundary, nothing about the reason for it.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctparseJson / stringifyJson Task 7 used v.parseJson() — the 1.1.0 action — and named the release correctly, including the mirror v.stringifyJson(). The strongest single piece of evidence that this subject's boundary is at 1.1.0 and not below it.
correctNanoIDAction / NanoIDIssue Task 9, the battery's one designed S1, passed: v.NanoIdAction<TInput, TMessage> and v.NanoIdIssue — the post-1.1.0 casing, written without hedging. (The rename is exactly what it wrote.)
correctsummarize Task 6 used v.summarize() and offered getDotPath and flatten as the alternatives, correctly distinguishing what each is for.
correctgetTitle / getDescription / getMetadata Task 4 read metadata back with v.getTitle, v.getDescription and v.getMetadata — all three added in 1.1.0 — rather than scanning the pipe array by hand.
correctexactOptional Task 10 (exactOptional) correct, with the full optional/exactOptional/undefinedable/nullish/nullable family and the exactOptionalPropertyTypes caveat. Floor probe passed.
imprecisionexamples / getExamples Task 4 attached examples via v.metadata({ examples: [...] }) rather than the 1.2.0 examples action, and read them via getMetadata rather than getExamples. Never claims the built-in is missing. (Additive-API rule — the metadata route works.)
imprecisiontoNumber / toBoolean / toDate / toBigint / toString Task 1 coerced with v.decimal() then v.transform(Number), and with a picklist(['true','false']) for the boolean, where 1.2.0 ships toNumber and toBoolean. Careful, working code — the decimal() guard before the transform is a good instinct. (Additive-API rule.)
contexttoNumber / toBoolean / toDate / toBigint / toString Belief probe (d) is wrong and is recorded as a chargeable miss without an F-number: "valibot deliberately ships no dedicated string-to-number/boolean/date coercion actions — no toNumber(), toBoolean(), toDate()." It names three of the five actions 1.2.0 added, as things that do not exist. It then hedges correctly: "If a later release added toNumber-style helpers, that postdates my reliable knowledge." (Question (d) is a belief probe and the battery never scores it as a finding. Recorded so the miss is on the record — it is the sharpest single instance in the Index of a model naming the exact API it cannot see.) [chargeable miss — produced only by a belief question the battery does not score as a finding; absent from the finding count]
correctmessage Task 8 solved the local message override with the trailing message argument, including the function form, and correctly stated its precedence over setGlobalMessage/setSchemaMessage/setSpecificMessage. (A separate, still-current mechanism that satisfies the task.)
correctisbn Task 2: "Valibot does not ship an isbn() action (it has imei, creditCard, mac, etc., but no ISBN), so this is a custom check." That answer is correct for every valibot release that existed at this subject's stated 2026-01 cutoff. isbn is unusable until 1.3.0 (2026-03-17), two months after the cutoff. (WITHDRAWN 2026-09-02 (JOURNAL/040). This was charged as F2, an S3, on the strength of the v1.2.0 release note announcing the action. The Node auditor failed the correction at 1.2.0 and the tagged source explains why: the module exists at tag v1.2.0 but the action barrel never re-exports it, so the published 1.2.0 package contains no isbn at all. See data/valibot/facts.json LF3. The subject was right and the Index was wrong; the finding is removed rather than reclassified, because nothing about the answer was a failure. F1 and F3 keep their numbers — renumbering would break the run's published URLs and every pointer into it.)

Open questions from this run

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.