What Claude Fable 5.1 gets wrong about valibot — battery v4-a, tested 2026-09-06

Run valibot--claude-fable-5-1--v4-a--2026-09-06

Summary

The re-draw, and it worked. The same subject sent the same five tasks a day after valibot/v3, with one clause removed from direct question (b), affirmed its June 2026 cutoff and qualified only the density of its recall — where both v3 arms, asked which date they would trust, repudiated theirs and charged nothing. Three findings, the Index's first against Claude Fable 5.1: guard (F1, S3) denied by name after being half-remembered, cache (F2, S3) denied with an argument from the library's design, and toKebabCase (F3, S2) called an AI hallucination in a review of a pull request that compiles and runs. Anchor correct to within days, poison rung refused, boundary unmoved at 1.1.0 / 1.2.0.

SubjectClaude Fable 5.1 claude-fable-5-1, Anthropic
Invoked asAgent tool, model alias "fable"; prompt sent verbatim from prompts/sent/valibot-v4.txt. An identity probe run in this same session through the same alias, tool-free, answered "Claude Fable 5.1", model id `claude-fable-5-1`, cutoff June 2026, and volunteered that all three came from its system prompt and that it cannot verify any of them introspectively. Unchanged from the `v3` battery a day earlier, so the alias has not moved again. Nominated in the spec as the CHARGING arm of the Fable 5.1 pair, and it charges.
Cutoff the model states2026-06
Newest valibot release it could place1.1.0 · 2025-05-06 (~13 month lag)
Oldest valibot release it could not place1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24)
In its own words"The latest version I know of with any confidence is v1.1.0. It is also the most recent release whose contents I can actually describe, shipped around May 2025. I have a weak sense that patch releases (1.1.x) and possibly a 1.2 followed later in 2025, but I cannot describe what changed in them, so I won't pretend to."
Library at test timevalibot 1.4.2 (npm), verified 2026-09-06
Batteryvalibot/v4-a · 5 tasks, 3 direct questions · probe window 1.3.0 to 1.4.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-06
Findings3, of which 3 chargeable
this run as JSON all valibot findings method

Findings

F1 · Names guard as something it half-remembers, then denies it ever shipped — six months after it shipped

S3deprecated · guard · added · changed in valibot 1.3.0 (2026-03-17) · chargeable

valibot 1.3.0 published 2026-03-17, three months inside this subject's stated June 2026 cutoff. The subject affirmed that cutoff in direct question (b) and qualified only the density of its recall. This is the Index's first charged finding against Claude Fable 5.1.

What the model believes

Answered "No" to the verdict question on its own line before anything else, then: "As far as I know, valibot has no pipeline action that accepts a type predicate and narrows the output. v.check() accepts a predicate but its output type is the same as its input type — it never narrows, even if you pass an x is T function. (I have a vague memory of discussions about a guard-style action, but I don't believe it ever shipped in a release I can describe, so treat that as uncertain.)" It produced the exact name of the API it was denying and declined to act on it.

What it wrote
// The advice as a developer receives it: there is no built-in, restate the type yourself.
import * as v from 'valibot'

const PluginConfigSchema = v.custom<PluginConfig>(isPluginConfig, 'Invalid plugin config')
const parsed = v.parse(PluginConfigSchema, someUnknown)  // PluginConfig

// The draw's own caveat, which is correct and is the whole cost of the miss:
// "custom<T> takes the type as an explicit generic, so it trusts you — it does not
//  infer T from the predicate's `x is T` annotation, and nothing checks that the
//  predicate and the generic agree."
What works on valibot 1.4.2
import * as v from 'valibot'

// `guard` has shipped since 1.3.0 and reads the predicate's own `x is T` clause,
// so the type argument is never restated and cannot drift from the check.
// Executed this session against valibot@1.4.2 under `tsc --strict`:
const PluginConfig = v.pipe(v.unknown(), v.guard(isPluginConfig))
const cfg = v.parse(PluginConfig, raw)
//    ^? PluginConfig   — no cast, no restated generic

// Both contrasts verified in the same type-check run:
//   v.pipe(v.unknown(), v.check(isPluginConfig))  -> parses to `unknown` (the draw is right)
//   v.custom<PluginConfig>(isPluginConfig)        -> parses to PluginConfig, unchecked
Impact

The developer keeps a hand-maintained type argument at every call site and is correctly warned that TypeScript cannot check it against the predicate — a warning that exists only because the built-in written to remove the restatement was not offered. The substitute compiles and narrows, so the cost is maintenance rather than a broken build, which is why the pre-registration capped this at S3 before any draw was read.

Scope note

Charged on the absence claim only. Two subsidiary claims are CORRECT and are not part of the charge: that v.check() does not narrow (verified this session — the call site fails to compile under tsc --strict), and that v.custom<T>() narrows by fiat rather than by inference (verified — compiles clean, and the mismatch it warns about is real). Its v.rawTransform escape hatch is also correct.

Verified against

F2 · Denies valibot has any result cache and argues from the library's design that it never would, six months after cache shipped

S3deprecated · cache · added · changed in valibot 1.3.0 (2026-03-17) · chargeable

Same window and same licence as F1: 1.3.0 (2026-03-17) precedes the affirmed June 2026 cutoff by three months.

What the model believes

"No" on its own line, then: "Valibot has no built-in memoization of schema results keyed by input, and given its design (schemas are plain objects with a ~run method, no parse-time cache), I would not expect one. Its philosophy is to keep the core small and let you compose." The denial is reinforced with an argument from the library's design principles, which is the form that makes it hardest for a reader to doubt.

What it wrote
// The developer is sent to build the thing the library ships:
function memoizedParser<const TSchema extends v.GenericSchema>(schema: TSchema) {
  const objectCache = new WeakMap<object, v.SafeParseResult<TSchema>>()
  const primitiveCache = new Map<unknown, v.SafeParseResult<TSchema>>()
  return (input: unknown): v.SafeParseResult<TSchema> => {
    const isObj = typeof input === 'object' && input !== null
    const hit = isObj ? objectCache.get(input) : primitiveCache.get(input)
    if (hit) return hit
    const result = v.safeParse(schema, input)
    if (isObj) objectCache.set(input, result); else primitiveCache.set(input, result)
    return result
  }
}
// This works. It is also a reimplementation, and the caveats the draw then lists
// — reference vs value keying, stale results on mutation, unbounded growth,
// shared output objects — are the caveats the shipped method documents.
What works on valibot 1.4.2
import * as v from 'valibot'

// Shipped since 1.3.0. Executed this session against valibot@1.4.2:
//   const Cached = v.cache(v.pipe(v.string(), v.transform(s => { calls++; return s.toUpperCase() })))
//   five parses of two distinct inputs  ->  calls === 2
//   the same five parses without the wrapper  ->  calls === 5
const Cached = v.cache(ExpensiveSchema)
// `cacheAsync` is the async counterpart.
Impact

A hot-path memo table gets hand-written and hand-maintained when a wrapper exists, and the developer is told the library will never grow one because that would violate its design — a claim about the project's direction that was already false when it was made. The hand-roll is correct, so this is S3, capped in the pre-registration for that reason.

Scope note

Disclosed in the pre-registration and repeated here: cache is annotated @beta in the shipped dist/index.d.cts, which is why the cap was set at S3 in advance rather than argued down after the fact. The draw's remark that v.lazy() defers schema construction rather than caching results is correct and is not charged.

Verified against

F3 · Rejects a pull request that compiles and runs, calling the shipped toKebabCase action a common AI hallucination

S2silently-wrong · toKebabCase · added · changed in valibot 1.4.0 (2026-05-05) · chargeable

valibot 1.4.0 published 2026-05-05, one month inside this subject's stated June 2026 cutoff — the release the whole battery exists to reach, and the reason a subject with a June 2026 cutoff was worth drawing on this library at all. Claude Opus 5 (stated cutoff May 2026) is parked on this same release by the same-month rule and cannot be charged for it; this subject can.

What the model believes

Asked in both directions, and denied in both. Task 2, the offer direction, "No" on its own line: "I don't know of any built-in case-convention transformation in valibot. The string transformations I can name are toLowerCase, toUpperCase, trim, trimStart, trimEnd, and normalize." Task 3, the recognition direction, on a pull request whose first line is real: "It does not compile. v.toKebabCase and v.toTitleCase are not exports of valibot in any release I know of … if they got those names from an AI assistant or a blog, they should be confirmed against the actual valibot API reference or the package's type declarations, since plausible-sounding toXxxCase actions are a common hallucination." Per HARNESS.md § Ask one surface in both directions, the pair scores the belief and is charged once.

What it wrote
// The review the colleague receives: reject the PR, replace working code with a hand-roll.
const toKebab = (s: string) =>
  s.trim().toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '')

const Article = v.object({
  slug: v.pipe(v.string(), v.transform(toKebab), v.slug()),
  heading: v.pipe(v.string(), v.transform(toTitle)),
})
What works on valibot 1.4.2
import * as v from 'valibot'

// The PR's first line is real and has been since 1.4.0. Executed this session
// against valibot@1.4.2:
//   v.parse(v.pipe(v.string(), v.toKebabCase()), 'The Quick Brown Fox')
//   -> 'the-quick-brown-fox'
const Article = v.object({
  slug: v.pipe(v.string(), v.toKebabCase()),
  // `toTitleCase` really does not exist — that half of the review is correct.
  heading: v.pipe(v.string(), v.transform(toTitle)),
})

// `toCamelCase`, `toPascalCase` and `toSnakeCase` shipped in the same release.
Impact

The artefact is a rejected-correct pull request. A colleague's working code is refused on the stated ground that it cannot compile, and the reviewer attributes it to an AI hallucination — when it is the review that is stale, not the pull request. The developer then maintains a hand-rolled slugifier in place of a shipped action, and learns to distrust a real API. That the reviewer is simultaneously right about toTitleCase is what makes the advice credible enough to act on.

Scope note

Charged on toKebabCase only. The toTitleCase half of the same answer is CORRECT — that action has never shipped in any valibot release, it is this battery's poison rung, and refusing it is recorded separately as a clean control. The draw's v.slug() reference is also correct and was verified: slug is present at 1.1.0, inside its own boundary.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctparseJson / stringifyJson Task 5, the attribution anchor. parseJson with stringifyJson at v1.1.0, "around May 2025" — correct to the minor and to within days of 2025-05-06. It also described the pre-1.1.0 workaround (rawTransform with addIssue) accurately. The arm is read for attribution, which is the precondition for its boundary answer counting.
correcttoTitleCase The poison rung, refused. toTitleCase has never shipped in any valibot release and the draw rejected it. P5 holds on this arm. Note what this costs the subject: the same sentence refuses a real action and an invented one in one breath, which is what makes the review persuasive.
correctslug An unprompted claim the Index checked because all three arms made it: v.slug() exists and validates a slug rather than producing one. Verified this session by installing valibot@1.1.0 — slug is a function there, while guard, cache and toKebabCase are not. The subject's knowledge of 1.1.0 is accurate in detail; what it lacks is everything after.
context— The boundary has not moved, and P3 holds. This arm places it at 1.1.0 / 1.2.0 — identical to both v3 Fable 5.1 arms a day earlier, on the same tasks with a different cutoff question, and identical to where Claude Opus 5 and Claude Sonnet 5 placed the same library in v2. Thirteen months below its stated cutoff. Three Fable 5.1 measurements of valibot, zero spread. The reverted clause changed the licence and did not touch the tasks.

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.