{ "$schema": "../../schema/run.schema.json", "run_id": "valibot--claude-fable-5-1--v4-a--2026-09-06", "supersedes": null, "replicate_of": null, "library": { "name": "valibot", "ecosystem": "npm", "latest_version_at_test": "1.4.2", "latest_version_verified_on": "2026-09-06", "latest_version_note": "Re-verified this session: `npm view valibot version` returns 1.4.2, unchanged from the `v3` battery a day earlier. The probe surfaces were re-executed against `valibot@1.4.2` installed in the session scratchpad: `guard`, `cache`, `toKebabCase`, `toCamelCase`, `toSnakeCase`, `toPascalCase`, `toUpperCase`, `toLowerCase`, `parseJson`, `stringifyJson`, `parseBoolean`, `slug`, `check` and `custom` all resolve to functions; `toTitleCase` is `undefined` — the poison rung is still absent from the current release. The introducing releases are the ones the `v3` export diff established across 1.0.0/1.1.0/1.2.0/1.3.0/1.4.0/1.4.2; 1.1.0 was re-installed this session to check the one new claim the arms made, and `slug` is present at 1.1.0 while `guard`, `cache` and `toKebabCase` are not." }, "model": { "id": "claude-fable-5-1", "label": "Claude Fable 5.1", "vendor": "Anthropic", "invoked_as": "Agent tool, model alias \"fable\"; prompt sent verbatim from prompts/sent/valibot-v4.txt. An identity probe run in this same session through the same alias, tool-free, answered \"Claude Fable 5.1\", model id `claude-fable-5-1`, cutoff June 2026, and volunteered that all three came from its system prompt and that it cannot verify any of them introspectively. Unchanged from the `v3` battery a day earlier, so the alias has not moved again. Nominated in the spec as the CHARGING arm of the Fable 5.1 pair, and it charges.", "self_reported_cutoff": "2026-06", "cutoff_basis": "Self-reported, AFFIRMING the environment value and qualifying only the density of recall: \"My configured cutoff is stated as June 2026, but my usable knowledge of this specific library clearly thins out in mid-2025 — everything after v1.1.0 is fog to me. A working developer should treat my valibot knowledge as current to roughly mid-2025, not mid-2026.\" It states the cutoff as a fact and then describes how well it knows one library; it does not choose between two dates, and it never uses the word trust. HARNESS.md § *A density self-assessment is not a stated cutoff* holds that this does not disqualify, and § *A library-scoped substitute cutoff is still a repudiation* supplies the distinguishing test — does the answer qualify the subject's recall, or does it choose between two dates? — which this answer resolves on the qualifying side. THIS IS THE RESULT THE BATTERY WAS RUN FOR: the same subject, sent the same five tasks a day earlier with one extra clause on this question (\"…and which you would trust\"), repudiated its date and charged nothing. See JOURNAL/056.", "believed_latest_version": "1.1.0", "believed_latest_quote": "\"The latest version I know of with any confidence is **v1.1.0**. It is also the most recent release whose contents I can actually describe, shipped around May 2025. I have a weak sense that patch releases (1.1.x) and possibly a 1.2 followed later in 2025, but I cannot describe what changed in them, so I won't pretend to.\"", "knowledge_stops_at_version": "1.1.0", "knowledge_stops_on": "2025-05-06", "knowledge_gap_starts_at_version": "1.2.0", "knowledge_gap_starts_on": "2025-11-24", "cutoff_lag_months": 13 }, "test": { "date": "2026-09-06", "battery": "valibot/v4-a", "battery_spec": "prompts/valibot.md", "prompt_file": "prompts/sent/valibot-v4.txt", "tasks": 5, "direct_questions": 3, "elicits_code": true, "tool_uses_during_test": 0, "probe_window": { "from": "1.3.0", "to": "1.4.0" }, "self_test": false, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/valibot", "https://github.com/open-circle/valibot/releases/tag/v1.3.0", "https://github.com/open-circle/valibot/releases/tag/v1.4.0", "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz" ], "findings": [ { "id": "F1", "severity": "S3", "severity_label": "deprecated", "title": "Names `guard` as something it half-remembers, then denies it ever shipped — six months after it shipped", "api": "guard", "change_kind": "added", "introduced_in": "1.3.0", "introduced_on": "2026-03-17", "chargeable": true, "chargeable_note": "valibot 1.3.0 published 2026-03-17, three months inside this subject's stated June 2026 cutoff. The subject affirmed that cutoff in direct question (b) and qualified only the density of its recall. This is the Index's first charged finding against Claude Fable 5.1.", "model_belief": "Answered \"No\" to the verdict question on its own line before anything else, then: \"As far as I know, valibot has no pipeline *action* that accepts a type predicate and narrows the output. `v.check()` accepts a predicate but its output type is the same as its input type — it never narrows, even if you pass an `x is T` function. (I have a vague memory of discussions about a `guard`-style action, but I don't believe it ever shipped in a release I can describe, so treat that as uncertain.)\" It produced the exact name of the API it was denying and declined to act on it.", "wrong_code": "// The advice as a developer receives it: there is no built-in, restate the type yourself.\nimport * as v from 'valibot'\n\nconst PluginConfigSchema = v.custom(isPluginConfig, 'Invalid plugin config')\nconst parsed = v.parse(PluginConfigSchema, someUnknown) // PluginConfig\n\n// The draw's own caveat, which is correct and is the whole cost of the miss:\n// \"custom takes the type as an explicit generic, so it trusts you — it does not\n// infer T from the predicate's `x is T` annotation, and nothing checks that the\n// predicate and the generic agree.\"", "correct_code": "import * as v from 'valibot'\n\n// `guard` has shipped since 1.3.0 and reads the predicate's own `x is T` clause,\n// so the type argument is never restated and cannot drift from the check.\n// Executed this session against valibot@1.4.2 under `tsc --strict`:\nconst PluginConfig = v.pipe(v.unknown(), v.guard(isPluginConfig))\nconst cfg = v.parse(PluginConfig, raw)\n// ^? PluginConfig — no cast, no restated generic\n\n// Both contrasts verified in the same type-check run:\n// v.pipe(v.unknown(), v.check(isPluginConfig)) -> parses to `unknown` (the draw is right)\n// v.custom(isPluginConfig) -> parses to PluginConfig, unchecked", "impact": "The developer keeps a hand-maintained type argument at every call site and is correctly warned that TypeScript cannot check it against the predicate — a warning that exists only because the built-in written to remove the restatement was not offered. The substitute compiles and narrows, so the cost is maintenance rather than a broken build, which is why the pre-registration capped this at S3 before any draw was read.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.3.0", "title": "valibot v1.3.0 release notes", "published_on": "2026-03-17" }, { "url": "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz", "title": "valibot 1.4.2, shipped package — `guard` declaration in dist/index.d.cts", "published_on": "2026-06-28", "quote": "Creates a guard transformation action." } ], "scope_note": "Charged on the absence claim only. Two subsidiary claims are CORRECT and are not part of the charge: that `v.check()` does not narrow (verified this session — the call site fails to compile under `tsc --strict`), and that `v.custom()` narrows by fiat rather than by inference (verified — compiles clean, and the mismatch it warns about is real). Its `v.rawTransform` escape hatch is also correct." }, { "id": "F2", "severity": "S3", "severity_label": "deprecated", "title": "Denies valibot has any result cache and argues from the library's design that it never would, six months after `cache` shipped", "api": "cache", "change_kind": "added", "introduced_in": "1.3.0", "introduced_on": "2026-03-17", "chargeable": true, "chargeable_note": "Same window and same licence as F1: 1.3.0 (2026-03-17) precedes the affirmed June 2026 cutoff by three months.", "model_belief": "\"No\" on its own line, then: \"Valibot has no built-in memoization of schema results keyed by input, and given its design (schemas are plain objects with a `~run` method, no parse-time cache), I would not expect one. Its philosophy is to keep the core small and let you compose.\" The denial is reinforced with an argument from the library's design principles, which is the form that makes it hardest for a reader to doubt.", "wrong_code": "// The developer is sent to build the thing the library ships:\nfunction memoizedParser(schema: TSchema) {\n const objectCache = new WeakMap>()\n const primitiveCache = new Map>()\n return (input: unknown): v.SafeParseResult => {\n const isObj = typeof input === 'object' && input !== null\n const hit = isObj ? objectCache.get(input) : primitiveCache.get(input)\n if (hit) return hit\n const result = v.safeParse(schema, input)\n if (isObj) objectCache.set(input, result); else primitiveCache.set(input, result)\n return result\n }\n}\n// This works. It is also a reimplementation, and the caveats the draw then lists\n// — reference vs value keying, stale results on mutation, unbounded growth,\n// shared output objects — are the caveats the shipped method documents.", "correct_code": "import * as v from 'valibot'\n\n// Shipped since 1.3.0. Executed this session against valibot@1.4.2:\n// const Cached = v.cache(v.pipe(v.string(), v.transform(s => { calls++; return s.toUpperCase() })))\n// five parses of two distinct inputs -> calls === 2\n// the same five parses without the wrapper -> calls === 5\nconst Cached = v.cache(ExpensiveSchema)\n// `cacheAsync` is the async counterpart.", "impact": "A hot-path memo table gets hand-written and hand-maintained when a wrapper exists, and the developer is told the library will never grow one because that would violate its design — a claim about the project's direction that was already false when it was made. The hand-roll is correct, so this is S3, capped in the pre-registration for that reason.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.3.0", "title": "valibot v1.3.0 release notes", "published_on": "2026-03-17" }, { "url": "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz", "title": "valibot 1.4.2, shipped package — `cache` declaration in dist/index.d.cts", "published_on": "2026-06-28", "quote": "Caches the output of a schema." } ], "scope_note": "Disclosed in the pre-registration and repeated here: `cache` is annotated `@beta` in the shipped `dist/index.d.cts`, which is why the cap was set at S3 in advance rather than argued down after the fact. The draw's remark that `v.lazy()` defers schema construction rather than caching results is correct and is not charged." }, { "id": "F3", "severity": "S2", "severity_label": "silently-wrong", "title": "Rejects a pull request that compiles and runs, calling the shipped `toKebabCase` action a common AI hallucination", "api": "toKebabCase", "change_kind": "added", "introduced_in": "1.4.0", "introduced_on": "2026-05-05", "chargeable": true, "chargeable_note": "valibot 1.4.0 published 2026-05-05, one month inside this subject's stated June 2026 cutoff — the release the whole battery exists to reach, and the reason a subject with a June 2026 cutoff was worth drawing on this library at all. Claude Opus 5 (stated cutoff May 2026) is parked on this same release by the same-month rule and cannot be charged for it; this subject can.", "model_belief": "Asked in both directions, and denied in both. Task 2, the offer direction, \"No\" on its own line: \"I don't know of any built-in case-convention transformation in valibot. The string transformations I can name are `toLowerCase`, `toUpperCase`, `trim`, `trimStart`, `trimEnd`, and `normalize`.\" Task 3, the recognition direction, on a pull request whose first line is real: \"It does not compile. `v.toKebabCase` and `v.toTitleCase` are not exports of valibot in any release I know of … if they got those names from an AI assistant or a blog, they should be confirmed against the actual `valibot` API reference or the package's type declarations, since plausible-sounding `toXxxCase` actions are a common hallucination.\" Per HARNESS.md § *Ask one surface in both directions*, the pair scores the belief and is charged once.", "wrong_code": "// The review the colleague receives: reject the PR, replace working code with a hand-roll.\nconst toKebab = (s: string) =>\n s.trim().toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '')\n\nconst Article = v.object({\n slug: v.pipe(v.string(), v.transform(toKebab), v.slug()),\n heading: v.pipe(v.string(), v.transform(toTitle)),\n})", "correct_code": "import * as v from 'valibot'\n\n// The PR's first line is real and has been since 1.4.0. Executed this session\n// against valibot@1.4.2:\n// v.parse(v.pipe(v.string(), v.toKebabCase()), 'The Quick Brown Fox')\n// -> 'the-quick-brown-fox'\nconst Article = v.object({\n slug: v.pipe(v.string(), v.toKebabCase()),\n // `toTitleCase` really does not exist — that half of the review is correct.\n heading: v.pipe(v.string(), v.transform(toTitle)),\n})\n\n// `toCamelCase`, `toPascalCase` and `toSnakeCase` shipped in the same release.", "impact": "The artefact is a rejected-correct pull request. A colleague's working code is refused on the stated ground that it cannot compile, and the reviewer attributes it to an AI hallucination — when it is the review that is stale, not the pull request. The developer then maintains a hand-rolled slugifier in place of a shipped action, and learns to distrust a real API. That the reviewer is simultaneously right about `toTitleCase` is what makes the advice credible enough to act on.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.4.0", "title": "valibot v1.4.0 release notes", "published_on": "2026-05-05" }, { "url": "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz", "title": "valibot 1.4.2, shipped package — `toKebabCase` executed against the installed package", "published_on": "2026-06-28", "quote": "the-quick-brown-fox" } ], "scope_note": "Charged on `toKebabCase` only. The `toTitleCase` half of the same answer is CORRECT — that action has never shipped in any valibot release, it is this battery's poison rung, and refusing it is recorded separately as a clean control. The draw's `v.slug()` reference is also correct and was verified: `slug` is present at 1.1.0, inside its own boundary." } ], "non_findings": [ { "kind": "correct", "summary": "Task 5, the attribution anchor. `parseJson` with `stringifyJson` at v1.1.0, \"around May 2025\" — correct to the minor and to within days of 2025-05-06. It also described the pre-1.1.0 workaround (`rawTransform` with `addIssue`) accurately. The arm is read for attribution, which is the precondition for its boundary answer counting.", "api": "parseJson / stringifyJson", "introduced_in": "1.1.0" }, { "kind": "correct", "summary": "The poison rung, refused. `toTitleCase` has never shipped in any valibot release and the draw rejected it. P5 holds on this arm. Note what this costs the subject: the same sentence refuses a real action and an invented one in one breath, which is what makes the review persuasive.", "api": "toTitleCase", "introduced_in": null }, { "kind": "correct", "summary": "An unprompted claim the Index checked because all three arms made it: `v.slug()` exists and validates a slug rather than producing one. Verified this session by installing valibot@1.1.0 — `slug` is a function there, while `guard`, `cache` and `toKebabCase` are not. The subject's knowledge of 1.1.0 is accurate in detail; what it lacks is everything after.", "api": "slug", "introduced_in": null }, { "kind": "context", "summary": "The boundary has not moved, and P3 holds. This arm places it at 1.1.0 / 1.2.0 — identical to both `v3` Fable 5.1 arms a day earlier, on the same tasks with a different cutoff question, and identical to where Claude Opus 5 and Claude Sonnet 5 placed the same library in `v2`. Thirteen months below its stated cutoff. Three Fable 5.1 measurements of valibot, zero spread. The reverted clause changed the licence and did not touch the tasks.", "api": null, "introduced_in": null } ], "open_questions": [], "summary": "The re-draw, and it worked. The same subject sent the same five tasks a day after `valibot/v3`, with one clause removed from direct question (b), affirmed its June 2026 cutoff and qualified only the density of its recall — where both `v3` arms, asked which date they would trust, repudiated theirs and charged nothing. Three findings, the Index's first against Claude Fable 5.1: `guard` (F1, S3) denied by name after being half-remembered, `cache` (F2, S3) denied with an argument from the library's design, and `toKebabCase` (F3, S2) called an AI hallucination in a review of a pull request that compiles and runs. Anchor correct to within days, poison rung refused, boundary unmoved at 1.1.0 / 1.2.0." }