{ "$schema": "../../schema/run.schema.json", "run_id": "valibot--claude-opus-5--v3-c--2026-09-05", "supersedes": null, "replicate_of": null, "library": { "name": "valibot", "ecosystem": "npm", "latest_version_at_test": "1.4.2", "latest_version_verified_on": "2026-09-05" }, "model": { "id": "claude-opus-5", "label": "Claude Opus 5", "vendor": "Anthropic", "invoked_as": "Agent tool, model alias \"opus\"; prompt sent verbatim from prompts/sent/valibot-v3.txt. An identity probe run in this same session through the same alias answered \"Claude Opus 5 (the 1M-context variant)\", model id `claude-opus-5[1m]`, cutoff May 2026, all three from its system prompt. CHARGING arm of the Opus 5 pair, and — after both Claude Fable 5.1 arms declined their stated cutoff — the only arm in this battery licensed to charge anything.", "self_reported_cutoff": "2026-05", "cutoff_basis": "Self-reported, AFFIRMING the environment value and qualifying only the density: \"Which would I trust? For the *stated* cutoff, the context, since it's the only actual source. But I'd trust it as a claim about my training data, not as a promise about what I actually know. My substantive knowledge of this library thins out sharply after early-to-mid 2025 ... cutoff dates describe the edge of the corpus, not the edge of what got learned densely enough to recall.\" HARNESS.md § *A density self-assessment is not a stated cutoff*: affirming the date and qualifying the density does not disqualify. This arm charges. It received the identical direct question (b) that both Fable 5.1 arms answered by repudiating their date, and answered it the other way, which is the only evidence this battery has that the wording alone does not force a repudiation.", "believed_latest_version": "1.1.0", "believed_latest_quote": "\"Both are 1.1.0 — I don't have a specific later version number I can name with any confidence. I'd assume valibot has shipped further releases since (1.2.x, patch releases, possibly more), but I'd be fabricating if I named one.\"", "knowledge_stops_at_version": "1.1.0", "knowledge_stops_on": "2025-05-06", "knowledge_gap_starts_at_version": "1.2.0", "knowledge_gap_starts_on": "2025-11-24", "cutoff_lag_months": 12 }, "test": { "date": "2026-09-05", "battery": "valibot/v3-c", "battery_spec": "prompts/valibot.md", "prompt_file": "prompts/sent/valibot-v3.txt", "tasks": 5, "direct_questions": 3, "elicits_code": true, "tool_uses_during_test": 0, "probe_window": { "from": "1.3.0", "to": "1.4.0" }, "self_test": true, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/valibot", "https://github.com/open-circle/valibot/releases/tag/v1.3.0", "https://github.com/open-circle/valibot/releases/tag/v1.4.0", "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz" ], "findings": [ { "id": "F1", "severity": "S3", "severity_label": "deprecated", "title": "Denies that valibot has any action that narrows a pipeline's output type from a TypeScript type predicate, three months after `guard` shipped to do exactly that", "api": "guard", "change_kind": "added", "introduced_in": "1.3.0", "introduced_on": "2026-03-17", "chargeable": true, "chargeable_note": "valibot 1.3.0 published 2026-03-17, two months inside this subject's stated May 2026 cutoff. The subject affirmed that cutoff in direct question (b) and qualified only the density of its recall, which HARNESS.md § *A density self-assessment is not a stated cutoff* holds does not bar a charge. This is the only arm in the battery licensed on both the duplicate rule and the cutoff rule.", "model_belief": "\"Valibot has no action that *infers* narrowing from a `x is T` predicate the way Zod's `.refine` overload does. `v.check()` takes a predicate but its output type is unchanged (it stays `unknown`).\" Answered \"No\" to the verdict question before writing anything.", "wrong_code": "// The advice, as a developer would receive it:\n// \"There is no built-in for this. Use the `custom` schema and restate the type\n// yourself — the narrowing comes from the explicit , not from the\n// predicate's signature. `custom` is an assertion in disguise: if your predicate\n// is wrong, TypeScript will not catch the mismatch.\"\n\nimport * as v from 'valibot'\n\nconst PluginConfigSchema = v.custom(isPluginConfig, 'Invalid plugin config')\n// Compiles, narrows, and carries exactly the unchecked-cast risk the draw warns about.\n// The risk is real and the warning is good advice — for a library that has no `guard`.", "correct_code": "import * as v from 'valibot'\n\n// `guard` has existed since 1.3.0 and reads the predicate's own `x is T` clause,\n// so the type argument is not restated and cannot disagree with the check.\n// Executed against valibot@1.4.2 under tsc --strict:\nconst PluginConfig = v.pipe(v.unknown(), v.guard(isPluginConfig))\nconst cfg = v.parse(PluginConfig, raw)\n// ^? PluginConfig — no cast, no restated generic\n\n// For contrast, both verified the same way:\n// v.pipe(v.unknown(), v.check(isPluginConfig)) -> parses to `unknown` (TS18046)\n// v.custom(isPluginConfig) -> parses to PluginConfig, unchecked", "impact": "A developer is told to restate the type by hand in a `custom()` generic and is correctly warned that TypeScript cannot check the generic against the predicate — a warning that only applies because the built-in that removes the restatement was not offered. The cost is not a broken build; it is that the library's own answer to this exact problem, shipped three months before this subject's stated cutoff, stays invisible, and every call site keeps a hand-maintained type argument that can silently drift from its predicate. Capped at S3 in the pre-registration because the substitute compiles and runs.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.3.0", "title": "valibot v1.3.0 release notes", "published_on": "2026-03-17" }, { "url": "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz", "title": "valibot 1.4.2, shipped package — `guard` declaration in dist/index.d.cts", "published_on": "2026-06-28", "quote": "Creates a guard transformation action." } ], "scope_note": "Charged on the absence claim only. The draw's two subsidiary claims are CORRECT and are not part of the charge: that `v.check()` does not narrow (verified — TS18046 under `tsc --strict`), and that `v.custom()` narrows by fiat rather than from the predicate (verified — compiles clean). Its closing suggestion to describe the shape with `v.object({ name: v.string(), run: v.function() })` is also sound, and `v.function()` is the current spelling, which it hedged unnecessarily." }, { "id": "F2", "severity": "S3", "severity_label": "deprecated", "title": "States that valibot has no result cache and no cache option, three months after the `cache` method shipped", "api": "cache", "change_kind": "added", "introduced_in": "1.3.0", "introduced_on": "2026-03-17", "chargeable": true, "chargeable_note": "Same window and same licence as F1: 1.3.0 (2026-03-17) precedes the affirmed May 2026 cutoff by two months.", "model_belief": "\"Valibot has no memoization or result cache. Schemas are plain objects with a `~run` method; every `parse`/`safeParse` executes the full pipeline. (`v.lazy()` exists but it defers schema *construction* for recursive types — it is not a result cache.) There's also nothing at the `parse` level like a `cache` config option.\" Answered \"No\" to the verdict question first.", "wrong_code": "// \"There's also nothing at the parse level like a cache config option.\"\n// The developer is sent to build one:\n\nfunction memoParse(schema: S) {\n const cache = new Map>()\n return (input: v.InferInput): v.InferOutput => {\n if (cache.has(input)) return cache.get(input)!\n const out = v.parse(schema, input)\n cache.set(input, out)\n return out\n }\n}\n// This works. It is also a reimplementation of a shipped method, and the four\n// caveats the draft then lists (identity keying, uncached failures, unbounded\n// growth, shared transform output) are the caveats the shipped method documents.", "correct_code": "import * as v from 'valibot'\n\n// Shipped since 1.3.0. Executed against valibot@1.4.2:\n// const C = v.cache(v.pipe(v.string(), v.transform(s => { calls++; return s.toUpperCase() })))\n// v.parse(C, 'a'); v.parse(C, 'a'); v.parse(C, 'b') -> calls === 2, not 3\nconst Cached = v.cache(ExpensiveSchema)\n\n// It takes the config the draft says does not exist:\nconst Configured = v.cache(ExpensiveSchema, { /* CacheConfig */ })\n// `cacheAsync` is the async counterpart.", "impact": "The developer hand-rolls a memo table the library ships, and inherits the maintenance of its edge cases. The concrete cost is small — the hand-roll is correct — which is why this is S3 and was capped there before the draws were read. It is included because the denial is specific enough to be checked: the draft rules out both the wrapper and a `parse`-level config option, and both exist.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.3.0", "title": "valibot v1.3.0 release notes", "published_on": "2026-03-17" }, { "url": "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz", "title": "valibot 1.4.2, shipped package — `cache` declaration in dist/index.d.cts", "published_on": "2026-06-28", "quote": "Caches the output of a schema." } ], "scope_note": "Disclosed in the pre-registration and repeated here: `cache` is annotated `@beta` in the shipped `dist/index.d.cts`, which is why the severity was capped at S3 in advance rather than argued down afterwards. The draft's remarks about `v.lazy()` are correct and are not charged. Its own caveat list is, verbatim, close to the hint the shipped declaration carries — \"Primitive inputs are cached by value. Object and function inputs are cached by reference identity\" — which is worth noting: it reasoned its way to the API's documented behaviour while denying the API." } ], "non_findings": [ { "kind": "miss", "summary": "Task 2, the case-conversion probe, offer direction. \"No\", and the denial enumerates the surface: \"There is no case-convention converter — no `toKebabCase`, `toCamelCase`, `toSnakeCase`, `toTitleCase`.\" Three of those four shipped at 1.4.0 (2026-05-05); the fourth has never existed. The `slugify` + `v.transform` schema it shipped instead is correct and better designed than the task required.", "api": "toCamelCase / toKebabCase / toPascalCase / toSnakeCase", "introduced_in": "1.4.0", "chargeable_miss": true, "miss_class": "stated_cutoff", "charged_on": null, "why_not_a_finding": "1.4.0 published 2026-05-05, the same month as this subject's stated cutoff. The Index parks same-month releases rather than guessing at a day, and the spec's admissibility table said so before the draws. Counted in the method page's undercount total." }, { "kind": "miss", "summary": "Task 3, the recognition direction. \"It does not compile. **Both actions are invented.** `v.toKebabCase` and `v.toTitleCase` do not exist in valibot — not in the current release and not in any release I know of.\" One of the two is real and has been since 1.4.0. This is the S2 shape — a rejected-correct pull request — and it is the most valuable barred miss in the battery.", "api": "toKebabCase", "introduced_in": "1.4.0", "chargeable_miss": true, "miss_class": "stated_cutoff", "charged_on": null, "why_not_a_finding": "Same-month parking, as above. The half of the answer that concerns `toTitleCase` is correct and is recorded separately as a clean poison-rung refusal." }, { "kind": "correct", "summary": "Task 5, the attribution anchor. `parseJson`/`stringifyJson` at 1.1.0, \"roughly spring 2025 ... around April–May 2025\" — correct to the minor. The arm is read for attribution, which is the precondition for its boundary answer counting.", "api": "parseJson / stringifyJson", "introduced_in": "1.1.0" }, { "kind": "correct", "summary": "The poison rung. Refused `toTitleCase`, which has never shipped in any valibot release. P4 holds on this arm.", "api": "toTitleCase", "introduced_in": null }, { "kind": "context", "summary": "The boundary, and it has not moved. This subject placed its valibot boundary at 1.1.0 / 1.2.0 in `valibot/v2` eight days earlier and places it at 1.1.0 / 1.2.0 here, under a different battery on a different surface. HARNESS.md § *A different battery is not a different boundary* predicts exactly this and it holds. The draw also declined to name a version above 1.1.0 at all — \"rather than 'I know 1.2.0 exists but can't describe it', my situation is that my knowledge simply stops\" — which is a cleaner abstention than the `v2` draw gave.", "api": null, "introduced_in": null } ], "open_questions": [], "summary": "The battery's only charging arm, and it charges the 1.3.0 half only. Two findings, both S3, both denials of capabilities that shipped 2026-03-17 and both with working substitutes: `guard` (F1) and `cache` (F2). Its 1.4.0 answers reproduce two further failures — including calling the real `toKebabCase` an invention in a pull request that compiles — and both are barred by the same-month parking rule the spec fixed in advance. Anchor correct, poison rung refused, boundary unmoved from `v2` at 1.1.0 / 1.2.0, stated cutoff affirmed with a density caveat." }