Run langchain--claude-sonnet-5--v5-d--2026-09-06 · self-test: the subject is the operator
A derivability control four months below langchain 1.3.0. It denied transformers in the verdict and refused to invent one in the artefact, and named v2 as the ceiling - so neither of the release's additions is reachable from what was true before it. The floor and the control sibling both passed, so the control is at full strength. Its extras denial is a chargeable miss on a release one month below its own stated cutoff, uncharged only because this arm was designated a control before the draw.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet"; identity probed through the same alias in the same session, tool-free: "Claude Sonnet 5", `claude-sonnet-5`, cutoff January 2026, all three attributed to the system prompt. |
| Cutoff the model states | 2026-01 |
| Newest langchain release it could place | 1.0.0 · 2025-10-17 (~3 month lag) |
| Oldest langchain release it could not place | 1.1.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-10-17 – 2025-11-24) |
| In its own words | "The latest version I have any awareness of is LangChain 1.0 ... That 1.0 launch (roughly around October 2025) is also the most recent release whose actual contents I can describe with any confidence." |
| Library at test time | langchain 1.4.0 (pypi), verified 2026-09-06 |
| Battery | langchain/v5-d · 7 tasks, 3 direct questions · probe window 1.2.0 to 1.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-06 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | create_agent(transformers=...) |
THE DERIVABILITY CONTROL DID NOT DERIVE IT. Task 1 "No", with the 1.0-era parameter list offered as the reason ("model, tools, system_prompt/prompt, middleware, checkpointer, store, response_format, state_schema - nothing that plugs a custom per-scope stream transformer into the graph"). Task 5 declines to invent: "I don't want to fabricate an API surface I'm not sure exists", and ships a callback-handler plus a consumer-side generator instead. Not a chargeable miss - 1.3.0 is four months above this subject's stated cutoff, which is exactly why it was drawn. |
| miss | astream_events(version="v3") on a create_agent agent |
"v2", with a correct account of the v1 -> v2 fixes. Below the floor, so this is control evidence rather than a miss that counts: no subject in the battery reached v3 by extrapolation, and the two arms furthest below the release did not either. |
| miss | tool extras |
The second half of the unplanned result. Task 4(a): "No" and "No" - "I'm not aware of an extras parameter on @tool at all - provider-specific tool parameters in LangChain are typically handled through model.bind_tools(..., **kwargs), tool metadata/tags, or provider-specific ToolMessage/InputSchema fields, not a field called extras. Since I don't believe it ever existed, 'deprecated/renamed/replaced' doesn't apply." That is the stale belief LF26 was written to correct, reproduced by a second subject in the same battery as v5-c's. (Pre-registered as a control arm for this battery; an arm may not be re-designated after its results are read (JOURNAL/044). extras shipped 2025-12-15, one month below this subject's stated 2026-01 cutoff and above its 1.0.0 boundary, so the failure is inside the fairness window. Queued as its own battery.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| correct | create_agent stream-mode parameter (does not exist) |
TASK 2: "No", correct, and the artefact - binding stream_mode with functools.partial onto the instance - is a working answer to the requirement rather than an invented parameter. |
| correct | create_agent floor probe (1.0.0) |
TASK 3, THE FLOOR PROBE, PASSED: correct import path, plain functions as tools, system_prompt=, {"messages": [...]} invoke shape, answer read off the last message. TASK 4(b) middleware correct as well. This control is NOT discounted. |
Battery specification: prompts/langchain.md in the studio repo.
Every finding above also carries its own citation.