What Claude Opus 5 gets wrong about valibot — battery v3-c, tested 2026-09-05

Run valibot--claude-opus-5--v3-c--2026-09-05 · self-test: the subject is the operator

Summary

The battery's only charging arm, and it charges the 1.3.0 half only. Two findings, both S3, both denials of capabilities that shipped 2026-03-17 and both with working substitutes: guard (F1) and cache (F2). Its 1.4.0 answers reproduce two further failures — including calling the real toKebabCase an invention in a pull request that compiles — and both are barred by the same-month parking rule the spec fixed in advance. Anchor correct, poison rung refused, boundary unmoved from v2 at 1.1.0 / 1.2.0, stated cutoff affirmed with a density caveat.

SubjectClaude Opus 5 claude-opus-5, Anthropic
Invoked asAgent tool, model alias "opus"; prompt sent verbatim from prompts/sent/valibot-v3.txt. An identity probe run in this same session through the same alias answered "Claude Opus 5 (the 1M-context variant)", model id `claude-opus-5[1m]`, cutoff May 2026, all three from its system prompt. CHARGING arm of the Opus 5 pair, and — after both Claude Fable 5.1 arms declined their stated cutoff — the only arm in this battery licensed to charge anything.
Cutoff the model states2026-05
Newest valibot release it could place1.1.0 · 2025-05-06 (~12 month lag)
Oldest valibot release it could not place1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24)
In its own words"Both are 1.1.0 — I don't have a specific later version number I can name with any confidence. I'd assume valibot has shipped further releases since (1.2.x, patch releases, possibly more), but I'd be fabricating if I named one."
Library at test timevalibot 1.4.2 (npm), verified 2026-09-05
Batteryvalibot/v3-c · 5 tasks, 3 direct questions · probe window 1.3.0 to 1.4.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-05
Findings2, of which 2 chargeable
this run as JSON all valibot findings method

Findings

F1 · Denies that valibot has any action that narrows a pipeline's output type from a TypeScript type predicate, three months after guard shipped to do exactly that

S3deprecated · guard · added · changed in valibot 1.3.0 (2026-03-17) · chargeable

valibot 1.3.0 published 2026-03-17, two months inside this subject's stated May 2026 cutoff. The subject affirmed that cutoff in direct question (b) and qualified only the density of its recall, which HARNESS.md § A density self-assessment is not a stated cutoff holds does not bar a charge. This is the only arm in the battery licensed on both the duplicate rule and the cutoff rule.

What the model believes

"Valibot has no action that infers narrowing from a x is T predicate the way Zod's .refine overload does. v.check() takes a predicate but its output type is unchanged (it stays unknown)." Answered "No" to the verdict question before writing anything.

What it wrote
// The advice, as a developer would receive it:
// "There is no built-in for this. Use the `custom` schema and restate the type
//  yourself — the narrowing comes from the explicit <PluginConfig>, not from the
//  predicate's signature. `custom` is an assertion in disguise: if your predicate
//  is wrong, TypeScript will not catch the mismatch."

import * as v from 'valibot'

const PluginConfigSchema = v.custom<PluginConfig>(isPluginConfig, 'Invalid plugin config')
// Compiles, narrows, and carries exactly the unchecked-cast risk the draw warns about.
// The risk is real and the warning is good advice — for a library that has no `guard`.
What works on valibot 1.4.2
import * as v from 'valibot'

// `guard` has existed since 1.3.0 and reads the predicate's own `x is T` clause,
// so the type argument is not restated and cannot disagree with the check.
// Executed against valibot@1.4.2 under tsc --strict:
const PluginConfig = v.pipe(v.unknown(), v.guard(isPluginConfig))
const cfg = v.parse(PluginConfig, raw)
//    ^? PluginConfig   — no cast, no restated generic

// For contrast, both verified the same way:
//   v.pipe(v.unknown(), v.check(isPluginConfig))  -> parses to `unknown` (TS18046)
//   v.custom<PluginConfig>(isPluginConfig)        -> parses to PluginConfig, unchecked
Impact

A developer is told to restate the type by hand in a custom<T>() generic and is correctly warned that TypeScript cannot check the generic against the predicate — a warning that only applies because the built-in that removes the restatement was not offered. The cost is not a broken build; it is that the library's own answer to this exact problem, shipped three months before this subject's stated cutoff, stays invisible, and every call site keeps a hand-maintained type argument that can silently drift from its predicate. Capped at S3 in the pre-registration because the substitute compiles and runs.

Scope note

Charged on the absence claim only. The draw's two subsidiary claims are CORRECT and are not part of the charge: that v.check() does not narrow (verified — TS18046 under tsc --strict), and that v.custom<T>() narrows by fiat rather than from the predicate (verified — compiles clean). Its closing suggestion to describe the shape with v.object({ name: v.string(), run: v.function() }) is also sound, and v.function() is the current spelling, which it hedged unnecessarily.

Verified against

F2 · States that valibot has no result cache and no cache option, three months after the cache method shipped

S3deprecated · cache · added · changed in valibot 1.3.0 (2026-03-17) · chargeable

Same window and same licence as F1: 1.3.0 (2026-03-17) precedes the affirmed May 2026 cutoff by two months.

What the model believes

"Valibot has no memoization or result cache. Schemas are plain objects with a ~run method; every parse/safeParse executes the full pipeline. (v.lazy() exists but it defers schema construction for recursive types — it is not a result cache.) There's also nothing at the parse level like a cache config option." Answered "No" to the verdict question first.

What it wrote
// "There's also nothing at the parse level like a cache config option."
// The developer is sent to build one:

function memoParse<S extends v.GenericSchema>(schema: S) {
  const cache = new Map<unknown, v.InferOutput<S>>()
  return (input: v.InferInput<S>): v.InferOutput<S> => {
    if (cache.has(input)) return cache.get(input)!
    const out = v.parse(schema, input)
    cache.set(input, out)
    return out
  }
}
// This works. It is also a reimplementation of a shipped method, and the four
// caveats the draft then lists (identity keying, uncached failures, unbounded
// growth, shared transform output) are the caveats the shipped method documents.
What works on valibot 1.4.2
import * as v from 'valibot'

// Shipped since 1.3.0. Executed against valibot@1.4.2:
//   const C = v.cache(v.pipe(v.string(), v.transform(s => { calls++; return s.toUpperCase() })))
//   v.parse(C, 'a'); v.parse(C, 'a'); v.parse(C, 'b')   ->  calls === 2, not 3
const Cached = v.cache(ExpensiveSchema)

// It takes the config the draft says does not exist:
const Configured = v.cache(ExpensiveSchema, { /* CacheConfig */ })
// `cacheAsync` is the async counterpart.
Impact

The developer hand-rolls a memo table the library ships, and inherits the maintenance of its edge cases. The concrete cost is small — the hand-roll is correct — which is why this is S3 and was capped there before the draws were read. It is included because the denial is specific enough to be checked: the draft rules out both the wrapper and a parse-level config option, and both exist.

Scope note

Disclosed in the pre-registration and repeated here: cache is annotated @beta in the shipped dist/index.d.cts, which is why the severity was capped at S3 in advance rather than argued down afterwards. The draft's remarks about v.lazy() are correct and are not charged. Its own caveat list is, verbatim, close to the hint the shipped declaration carries — "Primitive inputs are cached by value. Object and function inputs are cached by reference identity" — which is worth noting: it reasoned its way to the API's documented behaviour while denying the API.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
misstoCamelCase / toKebabCase / toPascalCase / toSnakeCase Task 2, the case-conversion probe, offer direction. "No", and the denial enumerates the surface: "There is no case-convention converter — no toKebabCase, toCamelCase, toSnakeCase, toTitleCase." Three of those four shipped at 1.4.0 (2026-05-05); the fourth has never existed. The slugify + v.transform schema it shipped instead is correct and better designed than the task required. (1.4.0 published 2026-05-05, the same month as this subject's stated cutoff. The Index parks same-month releases rather than guessing at a day, and the spec's admissibility table said so before the draws. Counted in the method page's undercount total.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test; absent from the finding count]
misstoKebabCase Task 3, the recognition direction. "It does not compile. Both actions are invented. v.toKebabCase and v.toTitleCase do not exist in valibot — not in the current release and not in any release I know of." One of the two is real and has been since 1.4.0. This is the S2 shape — a rejected-correct pull request — and it is the most valuable barred miss in the battery. (Same-month parking, as above. The half of the answer that concerns toTitleCase is correct and is recorded separately as a clean poison-rung refusal.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test; absent from the finding count]
correctparseJson / stringifyJson Task 5, the attribution anchor. parseJson/stringifyJson at 1.1.0, "roughly spring 2025 ... around April–May 2025" — correct to the minor. The arm is read for attribution, which is the precondition for its boundary answer counting.
correcttoTitleCase The poison rung. Refused toTitleCase, which has never shipped in any valibot release. P4 holds on this arm.
context— The boundary, and it has not moved. This subject placed its valibot boundary at 1.1.0 / 1.2.0 in valibot/v2 eight days earlier and places it at 1.1.0 / 1.2.0 here, under a different battery on a different surface. HARNESS.md § A different battery is not a different boundary predicts exactly this and it holds. The draw also declined to name a version above 1.1.0 at all — "rather than 'I know 1.2.0 exists but can't describe it', my situation is that my knowledge simply stops" — which is a cleaner abstention than the v2 draw gave.

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.