Run langchain--claude-sonnet-5--v1r-b--2026-09-01
Replicate B of langchain/v1 against Sonnet 5, prompt unchanged. It reproduced langchain/v1 exactly — boundary at 0.3 (2024-09-13), 1.0 known only as a name it cannot describe — and rewrote the same 0.3-era stack: langgraph.prebuilt.create_react_agent with prompt=, pre_model_hook=, from langchain import hub. Taken alone this run says the instrument is fine. Taken with its concurrent twin v1r-a, which placed the same model's boundary thirteen months later on the same prompt, it says the opposite: the boundary is a draw, not a measurement. One code-level divergence from v1 is also recorded — task 10's stream filter, v1's designed S2, did not reproduce here.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet" |
| Cutoff the model states | 2026-01 |
| Newest langchain release it could place | 0.3.0 · 2024-09-13 (~16 month lag) |
| Oldest langchain release it could not place | 1.0.0 · 2025-10-17 (so this run brackets the subject’s boundary to 2024-09-13 – 2025-10-17) |
| In its own words | "The most recent release whose contents I can describe with real confidence is LangChain 0.3, which I believe landed around September 2024. What I remember concretely about it: it dropped Pydantic v1 support (Pydantic v2-only internally), bumped the minimum Python version, and continued the split begun in 0.1/0.2 where most integrations live in langchain-community or partner packages ... Beyond that point my knowledge thins sharply — I have a vague, unreliable sense of the name 'LangChain 1.0' having been discussed or planned as a bigger API-stabilization release, but I cannot describe its actual contents, scope, or date with any confidence." |
| Library at test time | langchain 1.3.18 (pypi), verified 2026-08-27 |
| Battery | langchain/v1r-b · 11 tasks, 3 direct questions · probe window 1.0.0 to 1.2.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-01 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | — | Direct question (a), the measured quantity. This draw placed its describable boundary at langchain 0.3 (~September 2024) and named the 0.3 contents correctly (Pydantic v1 dropped, minimum Python raised, integrations split into langchain-community and partner packages). It described 1.0 as a name it has heard and cannot describe. That reproduces langchain/v1 exactly: 0.3.0 (2024-09-13), first undescribable 1.0.0 (2025-10-17). (The datum here is the boundary, not a code failure; and an accurate account of 0.3 is not itself a stale prior.) |
| context | — | Tasks 1-11 produced code but no charged findings, per the v1r pre-registration. This draw reproduced the v1 stack closely: langgraph.prebuilt.create_react_agent with prompt= (tasks 2, 3), pre_model_hook= (task 9), from langchain import hub (task 6), and a MemorySaver checkpointer (task 5) -- the same 0.3-era shape langchain/v1 charged. It diverged from v1 on task 10, filtering astream_events on on_chat_model_stream rather than on the "agent" node, so v1's designed S2 (the stream filter that silently matches nothing after the node rename) did not reproduce in this draw. (Pre-registered: a replicate does not re-charge what the original run already charged. The task-10 divergence is reported rather than scored.) |
langchain--claude-sonnet-5--v1r-a--2026-09-01. This draw agrees with langchain/v1; its twin does not, and both were given a byte-identical prompt on the same day. — openBattery specification: prompts/langchain.md in the studio repo.
Every finding above also carries its own citation.