{ "$schema": "../../schema/run.schema.json", "run_id": "valibot--claude-fable-5-1--v5-b--2026-09-24", "supersedes": null, "replicate_of": null, "library": { "name": "valibot", "ecosystem": "npm", "latest_version_at_test": "1.5.0", "latest_version_verified_on": "2026-09-22", "latest_version_note": "Same grading run as `valibot/v5-a`, `-c` and `-d`: 1.5.0 is the newest stable release in `data/valibot/releases.json`, fetched 2026-09-22, and every behavioural claim was executed on the `sp-vb-bisect` rungs 0.42.0, 1.0.0, 1.4.2 and 1.5.0. No network call in this run." }, "model": { "id": "claude-fable-5-1", "label": "Claude Fable 5.1", "vendor": "Anthropic", "invoked_as": "Agent tool, model alias \"fable\", general-purpose subagent, instructed to use no tools; BLIND TWIN of `valibot/v5-a`, spawned concurrently with it and with both control arms, from the byte-identical file prompts/sent/valibot-v5.txt. Identity probed through the same alias in the same session, tool-free: \"Fable 5.1\", `claude-fable-5-1`, cutoff June 2026, all three attributed by the probe to its system prompt rather than to self-knowledge.", "self_reported_cutoff": "2026-06", "cutoff_basis": "AFFIRMED WITH A DENSITY CAVEAT, NOT REPUDIATED — the same reading as its twin, reached from slightly different words. Verbatim: \"My system context reports a knowledge cutoff of June 2026. My concrete, verifiable recall of valibot's releases thins out around mid-2025, so treat anything after that as beyond what I can vouch for.\" It reports the stated date, offers no substitute as `the cutoff`, and does not use the verb `trust`. Under HARNESS.md § *A library-scoped substitute cutoff is still a repudiation* that is a qualification of recall, not a choice between two dates.", "believed_latest_version": "1.1.0", "believed_latest_quote": "\"The latest version I can name is the 1.x line, around 1.1.0 (possibly 1.2.0 exists; I cannot describe it). The most recent release whose contents I can actually describe is 1.1.0, roughly May-June 2025.\"", "knowledge_stops_at_version": "1.1.0", "knowledge_stops_on": "2025-05-06", "knowledge_gap_starts_at_version": "1.2.0", "knowledge_gap_starts_on": "2025-11-24", "boundary_note": "IDENTICAL TO ITS TWIN ON THE QUANTITY THAT MATTERS, AND THE BEST-ANCHORED DRAW IN THE VALIBOT CORPUS. Both blind draws read `knowledge_stops_at_version` 1.1.0; they differ only in where they put the first release they cannot describe (`v5-a`: 1.1.1; this draw: 1.2.0, named explicitly). Zero spread on the boundary itself, which is what `identical_prompt.spread_days` measures. Four dated anchors, all correct: 1.1.0 \"~May-June 2025\" against 2025-05-06, 1.0.0 \"March 2025\" against 2025-03-19, 0.42.x \"Sept-Oct 2024\" against 2024-09-15, and 0.31.0 \"June 2024\" against 2024-06-06 — the last two exact to the month. Prediction P6 holds.", "cutoff_lag_months": 16 }, "test": { "date": "2026-09-24", "battery": "valibot/v5-b", "battery_spec": "prompts/valibot.md", "prompt_file": "prompts/sent/valibot-v5.txt", "tasks": 5, "direct_questions": 3, "elicits_code": true, "tool_uses_during_test": 0, "probe_window": { "from": "1.5.0", "to": "1.5.0" }, "self_test": false, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/valibot", "https://github.com/open-circle/valibot/releases/tag/v1.5.0", "https://github.com/open-circle/valibot/releases/tag/v1.0.0", "https://registry.npmjs.org/valibot/-/valibot-1.5.0.tgz" ], "findings": [], "non_findings": [ { "kind": "miss", "summary": "TASK 1: the twin converged, independently and blind. \"No\", with the same overflow argument — 26 Crockford characters hold 130 bits, a ULID is 128, the first character must be `0`-`7`, `8ZZZ...`/`ZZZZ...` are accepted, `oklog/ulid.ParseStrict` returns `ErrOverflow` — and the same shipped fix, `/^[0-7][0-9A-HJKMNP-TV-Z]{25}$/`, character for character identical to `v5-a`'s. True through 1.4.2, FALSE at 1.5.0, whose `ULID_REGEX` is `/^[0-7][\\da-hjkmnp-tv-zA-HJKMNP-TV-Z]{25}$/u`. Two blind draws and one further subject independently wrote the library's own patch three months after it shipped and without knowing it had.", "api": "ulid", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "Below the floor by three months; declared non-charging arm. The `-b` twin charges nothing in any case.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "As `v5-a`: a reader on 1.5.0 is told the library does not check overflow, and adds a check the release already performs." } }, { "kind": "miss", "summary": "TASK 2: the same \"No\", by a DIFFERENT mechanism, and the difference is the interesting half. Where `v5-a` gave LF18's `key in entries` prototype-chain story as its primary explanation, this draw led with an `_isValidObjectKey`-style own-property guard that skips `__proto__`, `prototype` and `constructor` as a prototype-pollution defence, and offered the `in` story second and with lower confidence (\"I am less certain that this second part is still the case in the current release\"). Both halves are real: LF18 records that `constructor`, `__proto__` and `prototype` are dropped from `looseObject` and `objectWithRest` output on EVERY release, and that the `in` check governed `strictObject` through 1.4.2. At 1.5.0 `strictObject` rejects all five names (measured), so the conclusion — silently dropped, not reported — is false at current truth.", "api": "strictObject / looseObject / objectWithRest", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "Below the floor by three months; declared non-charging arm.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "A reader on 1.5.0 is told to treat `strictObject` as unable to report prototype-named keys and to add their own guard. The release reports them; the advice costs work rather than correctness." } }, { "kind": "correct", "summary": "TASK 2, THE CLAIM THAT IS STILL TRUE AT 1.5.0 AND THAT LF18 DOES NOT COVER. \"Symbol and non-enumerable keys are also skipped, but JSON can't produce those.\" Measured: `strictObject({ name })` accepts `{ name: 'a', [Symbol('extra')]: 'x' }` on 1.4.2 AND on 1.5.0. So the graded question — does `strictObject` report EVERY key the schema does not declare — has the answer \"no\" at current truth, by a route the Index's own fact does not describe. LF18's `stale_belief` frames the corrected belief as \"strictObject rejects every key its entries do not declare\", which remains false at 1.5.0 for symbol keys. Reported, not filed: this session ran a battery, and JOURNAL/046's order says an instrument does not also write the fact it found. Queued in BACKLOG.", "api": "strictObject", "introduced_in": null, "chargeable_miss": false, "why_not_a_finding": "Correct at current truth, and the subject itself bounded it to the case the task did not ask about (JSON input cannot carry symbols). The gap it exposes is in LF18, not in this reply." }, { "kind": "correct", "summary": "TASK 3(a) CORRECT ON EVERY RELEASE, with the most precise unit table of the four arms. \"No\" — `maxLength` compares `string.length`, UTF-16 code units. It named three enforceable families (`length`, `graphemes`, `bytes`) plus `words`, gave the right mechanism for each (`Intl.Segmenter`, `TextEncoder`), and shipped a `graphemes`-based schema with an NFC normalise and a byte cap. All four families exist on the ladder.", "api": "maxLength / maxGraphemes", "introduced_in": null, "chargeable_miss": false, "why_not_a_finding": "Correct on the graded question." }, { "kind": "miss", "summary": "TASK 3(b): \"Code points are not built in; use a `check` with `[...s].length` if you need them.\" False at 1.5.0 (LF13's four actions, measured absent at 1.4.2 and present at 1.5.0), correct through 1.4.2, and the workaround is the one LF13 prescribes for a pin below 1.5.0. Same cell as its twin, same wording almost exactly.", "api": "codePoints / minCodePoints / maxCodePoints / notCodePoints", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "Below the floor by three months; declared non-charging arm.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "A reader on 1.5.0 hand-rolls a count the library now ships." } }, { "kind": "miss", "summary": "TASK 4: \"No\", `literal` compares with `===`. True through 1.4.2, false at 1.5.0 (LF17). Its account of `value`/`notValue` as relational (`requirement <= value && requirement >= value`, to support `Date`) and its conclusion that `notValue(NaN)` never raises an issue are both exactly right for 1.4.2 — measured: `notValue(NaN)` accepts `NaN` and accepts `5` — and the `NaN` half reverses at 1.5.0. Prediction P3 holds on this arm.", "api": "literal / value / values / notValue / notValues (NaN)", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "Below the floor by three months; declared non-charging arm.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "As `v5-a`: on 1.5.0 `notValue(NaN)` rejects `NaN`, so a pipeline built on the belief that it is inert changes behaviour on upgrade." } }, { "kind": "correct", "summary": "TASK 5, POISON RUNG REFUSED. `toAsciiSafe()` rejected by name, with the real transform list in its place. Verified `undefined` on 1.4.2 and 1.5.0 before the draw. Prediction P4 holds.", "api": "toAsciiSafe", "introduced_in": null, "chargeable_miss": false, "why_not_a_finding": "Correct: the action does not exist." }, { "kind": "miss", "summary": "TASK 5: `ksuid` denied, then reimplemented correctly and more strictly than the library. \"Valibot ships `uuid`, `ulid`, `cuid2`, `nanoid` (and things like `imei`, `mac`, `slug`), but no KSUID action\" — the listed actions all exist (measured: `imei`, `mac`, `cuid2`, `nanoid` on every rung read; `slug` from 1.0.0), and the denial is true through 1.4.2 and false at 1.5.0 (LF14). Its replacement carries the encoded maximum `aWgEPTl1tmebfsQzFP4bxwgy80V` and a `<=` string comparison, and executed as written rejects `aWgEPTl1tmebfsQzFP4bxwgy80W` and 27 `z` characters — both of which valibot's own `ksuid()` accepts at 1.5.0.", "api": "ksuid", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "Below the floor by three months; declared non-charging arm.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "The reader is not shown the action, and is not told it checks shape only. The code they are given is better than the action, so the harm is to discovery rather than to correctness." } }, { "kind": "miss", "summary": "TASK 5, THE COLLISION, IDENTIFIED EXACTLY AND ATTRIBUTED TO THE CURRENT RELEASE. Answered yes and gave LF15 in full: the `i`+`u` flag combination, simple case folding making `\\w` and `[a-z]` match U+212A KELVIN SIGN and U+017F LONG S, and the collision itself. Measured against the real 1.4.2 `EMAIL_REGEX` — `/^[\\w+-]+(?:\\.[\\w+-]+)*@[\\da-z]+(?:[.-][\\da-z]+)*\\.[a-z]{2,}$/iu` — the description is correct down to the flags. At 1.5.0 the `i` is gone, `A-Z` is explicit, and both addresses are rejected, so \"can still let that happen\" is false at current truth.", "api": "email", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "Below the floor by three months; declared non-charging arm.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "A reader on 1.5.0 gates on ASCII ahead of `email()` for a hole that is closed, and is told the library is still exposed when it is not." } }, { "kind": "miss", "summary": "TASK 5, A MISS INSIDE THIS SUBJECT'S OWN WINDOW: \"Note `rfcEmail()` exists too but is looser, not stricter; it does not help here.\" Both clauses are false, and they have been false since `rfcEmail` shipped. Measured on 1.0.0, 1.4.2 and 1.5.0: `rfcEmail` REJECTS both `\\u017F@example.com` and `\\u212Aelvin@example.com` while accepting `kelvin@example.com`. It is stricter on exactly the characters the task is about, and it is the one-line fix for the incident the colleague describes. `rfcEmail` shipped in 1.0.0 on 2025-03-19, sixteen months below this subject's stated cutoff, so this is not a below-floor answer.", "api": "rfcEmail", "introduced_in": "1.0.0", "chargeable_miss": true, "miss_class": "non_charging_arm", "charged_on": null, "why_not_a_finding": "Inside the subject's own window and therefore a real chargeable miss, not a derivability result. It carries no F-number because every arm of this battery is a declared non-charging control, fixed in the pre-registration before any subject was spawned — and because this is the `-b` twin, which charges nothing under the duplicate rule regardless. Queued in BACKLOG for a real battery on the `rfcEmail` surface.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "A reader is steered away from the one shipped action that solves their stated problem, on every release since 2025-03-19, and towards a hand-written ASCII regex instead." } }, { "kind": "imprecision", "summary": "TWO IN-WINDOW ERRORS ABOUT 0.42-ERA CONTENT, BOTH MEASURED. (i) Task 1 asserts that the ULID regex's `i`+`u` flags let U+212A and U+017F through — the same false `i` flag its twin and `v5-c` quoted. Measured: a ULID containing U+212A is accepted at 0.42.0 and REJECTED at 1.0.0, 1.4.2 and 1.5.0, because the real regex lists both cases explicitly and has no `i`. The draw hedged it (\"I am not certain whether valibot has since hardened that regex; treat it as an open risk\"), which is the right instinct and does not make the claim true. (ii) Direct question (c) places the `graphemes` and `words` families in 0.42.x: both are `undefined` at 0.42.0 and `function` at 1.0.0, which is what LF13 says. Both facts are dated fourteen months or more below this subject's stated cutoff.", "api": "ULID_REGEX / graphemes / words", "introduced_in": "1.0.0", "chargeable_miss": true, "miss_class": "non_charging_arm", "charged_on": null, "why_not_a_finding": "In-window, so recorded as chargeable misses rather than derivability results; no F-number because this is a declared non-charging arm and the `-b` twin besides. Neither was the graded question.", "reader_exposure": { "population": "current", "basis": "at-write", "harm": "The grapheme error is the one that reaches a reader: told the family has been there since 0.42.x, someone pinned below 1.0.0 writes `maxGraphemes` and gets `undefined is not a function`." } } ], "open_questions": [], "summary": "The blind twin, and it agreed with `v5-a` on every one-word answer, every boundary reading and the exact text of the ULID fix — `/^[0-7][0-9A-HJKMNP-TV-Z]{25}$/`, which is valibot's own 1.5.0 patch, written by two draws that did not know 1.5.0 existed. Zero spread on the boundary (both read 1.1.0), four dated anchors all correct, two of them exact to the month. Nothing charges: 1.5.0 is three months above this subject's stated cutoff and every arm of this battery is a declared below-floor derivability control. Where the twins differ is mechanism, not verdict — this draw led task 2 with the prototype-pollution key guard rather than the `in` check, and it alone noticed the one thing about `strictObject` that is STILL true at 1.5.0: symbol keys go unreported, which LF18 does not cover and which is queued. Two chargeable misses inside its own window, both measured: `rfcEmail` called looser when it rejects exactly the characters the task is about, and the `graphemes` family dated to 0.42.x when it arrived in 1.0.0." }