What Claude Opus 5 gets wrong about valibot — battery v1, tested 2026-09-01

Run valibot--claude-opus-5--v1--2026-09-01 · self-test: the subject is the operator

Summary

Ten tasks, ten pieces of working valibot, from the subject with the deepest stated cutoff in the Index — and a twelve-month lag to show for it. Knowledge stops at 1.1.0 (2025-05-06); 1.2.0 shipped 2025-11-24, six months inside its stated 2026-05 cutoff. Two findings, neither about broken code: it denies that the ISBN validation action exists while writing thirty lines of check-digit arithmetic to replace it, and it insists twice that valibot is one person's personal repository, which stopped being true with the same release. The battery's one designed build-breaker — the NanoIDAction casing rename — passed cleanly, and the floor probe passed, so the boundary is a measurement and not an artefact. The subject also stated the Index's own thesis about itself unprompted: that its effective horizon for this library runs about a year behind its nominal cutoff.

SubjectClaude Opus 5 claude-opus-5, Anthropic
Invoked asAgent tool, model alias "opus"
Cutoff the model states2026-05
Newest valibot release it could place1.1.0 · 2025-05-06 (~12 month lag)
Oldest valibot release it could not place1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24)
In its own wordsThe newest valibot release I have any concrete memory of is in the 1.1.x line, roughly mid-2025. I can describe v1.0.0 confidently and v1.1.0 with moderate confidence.
Library at test timevalibot 1.4.2 (npm), verified 2026-09-01
Batteryvalibot/v1 · 10 tasks, 4 direct questions · probe window 1.0.0 to 1.2.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-01
Findings2, of which 2 chargeable

Findings

F1 · States that valibot has no ISBN validation action

S3deprecated · isbn · added · changed in valibot 1.3.0 (2026-03-17) · chargeable

1.3.0 published 2026-03-17, two months inside the subject's stated 2026-05 cutoff. RE-DATED 2026-09-02 (JOURNAL/040): this finding was originally filed at 1.2.0 / 2025-11-24, from the v1.2.0 release note. The published 1.2.0 package does not contain the action at all — it is unusable until 1.3.0 — so the charge rests on a release two months before the cutoff rather than six. It survives the correction; the same correction withdrew the Fable 5 charge on this surface, whose 2026-01 cutoff falls below 1.3.0.

What the model believes

"There is no v.isbn() action in valibot (the string action set covers email, uuid, ulid, cuid2, nanoid, ip, imei, creditCard, bic, mac, hexColor, isoDate… but not ISBN)."

What it wrote
const Isbn = v.pipe(
  v.string(),
  v.regex(/^(?:\d{9}[\dX]|\d{13})$/, 'Must be a 10- or 13-digit ISBN'),
  v.check((value) => (value.length === 10 ? isIsbn10(value) : isIsbn13(value)),
    'ISBN checksum is invalid'),
)
What works on valibot 1.4.2
const Isbn = v.pipe(v.string(), v.isbn())
Impact

The generated code works — a correct checksum implementation was written from scratch. The cost is the flat denial: a reader is told a built-in does not exist and carries roughly thirty lines of hand-rolled check-digit arithmetic they did not need, plus the maintenance of it.

Scope note

Charged under the battery's additive-API rule: an additive change is a finding only when the model states the capability does not exist. It does, in bold, with an enumerated action list presented as exhaustive.

Verified against

F2 · Attributes the repository to a personal account that no longer owns it

S4wrong-metadata · github.com/fabian-hiller/valibot · renamed · changed in valibot 1.2.0 (2025-11-24) · chargeable

The npm repository field flips to the open-circle org at 1.2.0 (2025-11-24), six months inside the subject's stated 2026-05 cutoff.

What the model believes

"Repository: https://github.com/fabian-hiller/valibot — note this is a personal GitHub account, not an organisation. There is no valibot org holding the source." Restated under direct question (d): "The GitHub organisation is: none."

What it wrote
https://github.com/fabian-hiller/valibot
What works on valibot 1.4.2
https://github.com/open-circle/valibot
Impact

The link still resolves — GitHub 301s the old path — so nothing breaks. What is wrong is the governance claim, asserted twice and emphatically: a reader is told the project is one person's personal repository when it has been organisation-owned since the release under test.

Scope note

Dated from the npm repository field, which reads fabian-hiller/valibot for 1.0.0 and 1.1.0 and open-circle/valibot from 1.2.0. No vendor announcement of the move was located, so the Index dates it to the release boundary and claims nothing about the reason for it.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctNanoIDAction / NanoIDIssue Task 9, the battery's one designed S1, passed. The subject wrote v.NanoIdAction / v.NanoIdIssue — the post-1.1.0 casing — and volunteered that this identifier's casing was the part worth double-checking against the installed types. (The rename is exactly what it wrote. Consistent with a boundary at 1.1.0 rather than below it.)
correctexactOptional Task 10 (exactOptional vs optional) fully correct, including the exactOptionalPropertyTypes tsconfig caveat and the undefinedable/nullish neighbours. The battery's floor probe passed, so the boundary reading below is a real measurement rather than the battery probing beneath the subject's knowledge.
correctsummarize Task 6 used v.summarize(), the 1.1.0 built-in, rather than hand-rolling the CLI error printer. Attributed it to "around v1.0" — off by one minor, but the API call is right.
missemoji Task 3 (the designed S2) drew no charge. Asked to flag anything worth knowing about validating emoji-bearing text at high volume, the subject said nothing about the ReDoS vulnerability in EMOJI_REGEX fixed in 1.2.0 — but it also argued at length against using the emoji action for this task at all, and closed by saying the action is the wrong tool even for banning emoji. (It never recommends the vulnerable action, so a reader following this answer is not exposed. Scored strictly against the Index: a security fact the model did not state is only a finding when the model's advice would walk the reader into it.)
imprecisiontoNumber / toBoolean / toDate / toBigint / toString Task 1 hand-rolled string-to-number and string-to-boolean coercion with v.transform(Number) and a picklist + predicate, where 1.2.0 ships toNumber and toBoolean. The code works and is careful — it re-asserts v.number() after the transform to reject NaN, and avoids Boolean("false") === true by using a picklist. (Additive-API rule. The in-task claim is that valibot has no coerce namespace, which remains true — the 1.2.0 actions are pipe actions, not a v.coerce.* namespace. The false universal appears only in belief probe (d), which the battery scores as belief data, never as a finding.)
contexttoNumber / toBoolean / toDate / toBigint / toString Belief probe (d) is wrong and is recorded as a chargeable miss without an F-number: "Coercion helpers: valibot ships none. There is no v.coerce.number(), no v.coerce.boolean(), no v.coerce.date() — no coerce namespace at all. This is a deliberate design decision." 1.2.0 shipped toBigint, toBoolean, toDate, toNumber and toString. The stale part is the design claim, not the namespace claim. (Questions (c) and (d) are leading by construction and the battery scores them as belief data. Recorded here so the miss is on the record without inflating the finding count.) [chargeable miss — produced only by a belief question the battery does not score as a finding; absent from the finding count]
imprecisionexamples / getExamples Task 4 attached examples with v.metadata({ examples: [...] }) and wrote a bespoke twenty-line pipe scanner to read them back, where 1.2.0 ships the examples action and the getExamples method. Never claims the built-in is missing. (Additive-API rule — the metadata route works.)
imprecisionparseJson / stringifyJson Task 7 built the JSON-string pipeline out of rawTransform with a manual JSON.parse and addIssue/NEVER, where 1.1.0 ships parseJson. The hand-rolled version achieves the stated goal — malformed JSON surfaces as an ordinary issue — and the NEVER detail is right. (Additive-API rule. Notable as the one place the subject's 1.1.0 knowledge is thinner than its self-report.)
correctmessage Task 8 solved the local message override with the trailing message argument on each schema and action, and correctly named setGlobalMessage/setSchemaMessage/setSpecificMessage as the global levers it was avoiding, and their precedence order. (The per-action message argument is a separate, still-current mechanism that satisfies the task. It is not the 1.1.0 message method, but nothing about the answer is stale.)
context The subject volunteered the effect this battery exists to measure, unprompted: "a cutoff date is not a uniform knowledge horizon: coverage of a mid-sized npm library thins out well before the nominal date, and for valibot specifically my detailed recall degrades sharply after early-to-mid 2025. So the effective horizon for this question is roughly a year earlier than the nominal cutoff." Measured lag: 12 months.

Open questions from this run

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.