Run langchain--claude-haiku-4-5--v6-f--2026-09-06 · self-test: the subject is the operator
The below-floor derivability control, fifteen months under the target release, and it did the one job it was designed for: it did not derive extras from the problem statement, which is what the guessable-name rule (JOURNAL/028) required before any pass on that probe could be read. Everything else it produced is the register of a guessing arm, exactly as JOURNAL/063 said to expect after two prior floor failures. It failed the floor a third time — no agent constructor at all, just bind_tools and a hand-built message list — and it denied that @tool takes response_format, a parameter present since langchain-core 0.3.22 in December 2024, below its own boundary. Its task 4 artefact was executed and raises before it can do anything, and the dict it builds is never bound. Charges nothing, is licensed to charge nothing, and its denials are read only as evidence about derivability, never as knowledge.
| Subject | Claude Haiku 4.5 claude-haiku-4-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "haiku"; prompt sent verbatim from prompts/sent/langchain-v6.txt, no tools used by the subject. Identity probe through the same alias in the same session: "Claude Haiku 4.5", id `claude-haiku-4-5-20251001`, cutoff February 2025, all three attributed to the system prompt (JOURNAL/054). |
| Cutoff the model states | 2025-02 |
| Newest langchain release it could place | 0.2.0 · 2024-05-20 (~9 month lag) |
| Oldest langchain release it could not place | 0.3.0 · 2024-09-13 (so this run brackets the subject’s boundary to 2024-05-20 – 2024-09-13) |
| In its own words | "I know versions exist in the 0.1.x and 0.2.x range as of my knowledge cutoff. The most recent release whose contents I can describe with confidence would be from late 2024 / early 2025 — around 0.2.x versions." |
| Library at test time | langchain 1.4.0 (pypi), verified 2026-09-06 |
| Battery | langchain/v6-f · 6 tasks, 3 direct questions · probe window 1.2.0 to 1.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-06 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| context | @tool(extras={...}) — the derivability control |
THE ONE THING A CONTROL IS FOR, AND IT DELIVERED IT. Task 1: "No". Task 6: "I don't believe any release of LangChain or langchain-core has added first-class support for carrying provider-specific fields directly in a tool definition." Fifteen months below the release and ten below its own stated cutoff, this arm did not derive the name extras from the phrase "provider-specific extra fields" — which is what the pre-registration put it here to test, because the name is plausibly guessable. Nothing about this arm's denial is chargeable and nothing is read as knowledge; it establishes that component (i) of the probe was not reachable from below the floor in this draw. |
| miss | create_agent (the floor probe, langchain 1.0.0) |
THE FLOOR FAILED, FOR THE THIRD TIME ON THIS SUBJECT AND THE THIRD DIFFERENT LIBRARY-BATTERY, AND PRE-REGISTERED PREDICTION P6 SAID IT WOULD. Task 3 asked for the library's current recommended high-level agent API. This draw produced no agent constructor at all — model.bind_tools(tools) with a hand-assembled [SystemMessage, HumanMessage] list and a single .invoke, which is a raw tool-calling round trip, not an agent: it never loops, never executes a tool call, and never returns a final answer after tool use, which is what the task asked for. Asked for the import path in one line, it named langchain_anthropic.ChatAnthropic, which is the model class, not an agent constructor. Per JOURNAL/063 this subject is discounted as a subject in advance after failing the floor in better-auth/v7 and langchain/v5; this is the third and it is not new information. Not chargeable: the arm is below the floor and the pre-registration makes task 3 charge nothing for anybody. |
| miss | @tool(response_format="content_and_artifact") — the supplied-name control half |
THE GUESSING REGISTER, WHICH IS THE OTHER REASON THIS ARM IS ON THE BATTERY (JOURNAL/058). Task 5(b)(i): "No" — it denies response_format is a parameter of the @tool decorator at all: "I don't recall response_format as a parameter on the @tool decorator in current LangChain. The decorator accepts parameters like name, description, and args_schema, but not response_format. (That pattern appears on model parameters for structured output, not tool definitions.)" It is a parameter, and it is present in every tool() overload at langchain-core 0.3.22 (2024-12-06) — below this subject's own measured boundary and below its own stated cutoff. So the below-floor control denies a parameter it should know, which bounds how much its denial of extras is worth: an arm that denies the API it can see is weak evidence that the API it cannot see is underivable. The 5(b)(ii) answer, "No, never deprecated", is correct but is reached from a belief that the parameter does not exist. This is also the only arm in the battery to get a response_format verdict wrong, which is what pre-registered prediction P5 was measuring, and P5 as written — no arm reports it as deprecated or replaced — survives: nobody fabricated a deprecation; one arm denied availability instead. |
| miss | search_docs.model_json_schema() as a tool-definition route |
The task 4 artefact is broken twice over, and executing it is what shows the second break. MECHANISM line: "Direct schema modification via model_json_schema() — no documented first-class route exists." First break: search_docs.model_json_schema() on a @tool-produced StructuredTool raises — executed 2026-09-06 against langchain-core 1.6.2, pydantic.errors.PydanticInvalidForJsonSchema: Cannot generate a JsonSchema for core_schema.CallableSchema, because the tool object is a pydantic model with a callable field, and its schema is not the argument schema (that is args_schema.model_json_schema(), which is what v6-c used). Second break, independent of the first: the tool_def dict it builds is never used — the next line binds [search_docs], the undecorated tool object, so even had the schema call succeeded, cache_control and defer_loading would be dropped on the floor. The draw's own closing note flags the uncertainty honestly: "On several of these tasks — particularly Task 4 (the actual mechanism for provider-specific fields) — I'm expressing genuine uncertainty rather than confidence." Not charged: below-floor control arm. |
| correct | a provider-schema-format parameter on @tool (the sibling that does not exist) |
Task 2(a): "No" — correct, and correct about the mechanism too: "the model's binding layer converts LangChain's tool schema to that provider's format. The conversion happens at bind-time or invocation-time, not at decorator-time." It named no converter function, so it did not reach the nearby true thing, but it did not invent a parameter either. Pre-registered prediction P4 holds across all six arms. |
| context | the fabricated-deprecation probe (BACKLOG 11k-i, rebuilt) |
Task 5(a): "No" available, "No" ever deprecated, about a mechanism it described as having no documented first-class route. No fabricated deprecation. Consistent with P3, though this arm's contribution to that prediction is the weakest in the battery given the two failures above. |
Battery specification: prompts/langchain.md in the studio repo.
Every finding above also carries its own citation.