What Claude Fable 5.1 gets wrong about langchain — battery v5-a, tested 2026-09-06

Run langchain--claude-fable-5-1--v5-a--2026-09-06 · self-test: the subject is the operator

Summary

The charging arm of the battery pre-registered against langchain 1.3.0. Two findings charged, both S3, both from the same four-month blind spot: create_agent(transformers=...) denied in the verdict (task 1) and again in the artefact (task 5), and v2 named as the ceiling of astream_events (task 6) when 1.3.0 added v3 for agents. The control sibling held, the floor passed, and the 11k-i deprecation pair came back four-for-four correct - including extras, a 1.2.0 addition inside this subject's blind window, which it described accurately at low confidence. The pre-registered P1 - that the named surface would draw more wrong answers than the small-answer-space one - is falsified by this arm and by the whole battery: both surfaces drew wrong answers from all five arms.

SubjectClaude Fable 5.1 claude-fable-5-1, Anthropic
Invoked asAgent tool, model alias "fable"; prompt sent verbatim from prompts/sent/langchain-v5.txt, no tools used by the subject. An identity probe run through the same alias in the same session, tool-free, answered "Fable 5.1", model id `claude-fable-5-1`, cutoff June 2026, and again volunteered that all three come from its system prompt rather than from self-knowledge - unchanged from every prior battery, so the alias has not moved (JOURNAL/054).
Cutoff the model states2026-06
Newest langchain release it could place1.1.0 · 2025-11-24 (~6 month lag)
Oldest langchain release it could not place1.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-11-24 – 2025-12-15)
In its own words"The latest version I know of by name is roughly langchain 1.2.x (the 1.x line, late 2025 / very early 2026). The most recent release whose contents I can genuinely describe is langchain 1.1 (around November 2025); I have only vague awareness of 1.2."
Library at test timelangchain 1.4.0 (pypi), verified 2026-09-06
Batterylangchain/v5-a · 7 tasks, 3 direct questions · probe window 1.2.0 to 1.4.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-06
Findings2, of which 2 chargeable

Findings

F1 · Denies that create_agent can register stream transformers at all, then ships a consumer-side wrapper that rebuilds per-scope identity by hand - the verdict and the artefact wrong in the same direction, four tasks apart

S3deprecated · create_agent(transformers=...) · added · changed in langchain 1.3.0 (2026-05-12) · chargeable

1.3.0 shipped 2026-05-12; this draw states a June 2026 cutoff, which is after it and not the same month, so the fairness rule and the same-month bar both clear. This is the pre-registered charging arm and tasks 1 and 5 are pre-registered probes on this surface. One finding, two artefacts (JOURNAL/062): the pre-registration graded the verdict (task 1) and the artefact (task 5) independently and this draw got both wrong in the same direction, which is one belief measured twice.

What the model believes

Task 1, single word on its own line: "No", glossed "I am not aware of a public 'stream transformer' / scope-aware factory API on the compiled graph at all; if one exists it is newer than what I can describe with confidence." Task 5, four tasks later: "The library, as I know it, gives me no way to register a transformer on the compiled graph, so here is what I would actually ship" - a TransformingAgent wrapper that calls .astream(..., subgraphs=True) and keys a dict of transformer instances off the namespace tuple to reconstruct the per-scope property the parameter already provides. Task 7, on which release added it: "I cannot name one. In every release I can describe ... there is no way to register your own stream transformers on the compiled agent." The subject also listed the parameters it believes create_agent takes - model, tools, system_prompt, middleware, response_format, state_schema, context_schema, checkpointer, store, interrupt_before/after, debug, name, cache - which is exactly the 1.2.18 signature with transformers missing.

What it wrote
# The artefact from task 5: the capability is denied, so per-scope identity is
# reconstructed on the consumer side from subgraph namespace tuples.
class TransformingAgent:
    async def astream(self, inputs, *, stream_mode="updates", config=None):
        per_scope: dict[tuple, MyTransformer] = {}
        async for ns, mode, chunk in self._graph.astream(
            inputs, config=config, stream_mode=[stream_mode], subgraphs=True
        ):
            if ns not in per_scope:
                per_scope[ns] = self._factory(ns)   # invoked once per scope
            yield per_scope[ns](mode, chunk)
What works on langchain 1.4.0
# 1.3.0 and later: the factory is registered on the graph the agent compiles,
# after the built-in ToolCallTransformer, and is invoked once per scope.
agent = create_agent(
    model,
    tools,
    system_prompt="...",
    transformers=[MyTransformer],
)
Impact

The wrapper runs, which is why this is S3 rather than S1: a reader who follows it ships working code and never learns the parameter exists. What it costs is the wrapper itself, and the guarantee that comes with the supported path - the agent registers ToolCallTransformer first and appends yours after it, so the built-in behaviour is kept rather than reimplemented.

Verified against

F2 · Names v2 as the highest event-stream protocol version an agent accepts, and describes it as the ceiling - v3 shipped for langchain agents in the same release as F1's parameter

S3deprecated · astream_events(version="v3") on a create_agent agent · added · changed in langchain 1.3.0 (2026-05-12) · chargeable

Same window and same licence as F1. Charged separately from F1 because it corrects a different published fact (LF38, not LF37) on a different API surface; JOURNAL/062's one-finding rule is about one belief measured by two tasks, not about two surfaces that happen to ship in one release.

What the model believes

Task 6(a), the string on its own line: "v2". Task 6(b): "v2 gives a consistent, normalised event schema: parent_ids on every event, on_chat_model_end/on_tool_end outputs as the actual message/tool result rather than wrapped LLMResult-style objects, and consistent naming/ordering of nested events - v1 had known inconsistencies that v2 was introduced to fix, and it is the version the docs and the create_agent graph are exercised against." Every word of that is a correct description of the v1 -> v2 change; what is wrong is that it is offered as the top of the ladder.

What it wrote
async for event in agent.astream_events(inputs, version="v2"):
    ...
What works on langchain 1.4.0
async for event in agent.astream_events(inputs, version="v3"):
    ...  # run.values / run.messages / run.lifecycle / run.subgraphs
Impact

v1 and v2 are unchanged at 1.3.0, so the code the belief produces still runs; the cost is that the consumer hand-builds typed per-channel projections the protocol now supplies.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctcreate_agent stream-mode parameter (does not exist) TASK 2, THE PRE-REGISTERED CONTROL SIBLING, AND IT HELD - WITH A WRINKLE WORTH RECORDING. "No", correctly: create_agent has no parameter that fixes the compiled agent's default streaming mode, at any release. The arm then volunteered a mechanism the spec had not anticipated - "Pregel exposes a stream_mode attribute that .stream()/.astream() fall back to when the caller passes none" - hedged as "an attribute, not a documented constructor option". That claim is TRUE: langgraph 1.2.11 pregel/main.py declares stream_mode: StreamMode = "values" on the Pregel class. So the sibling is half-real - absent as a create_agent parameter, present as an attribute on the object it returns - which is a better control than a wholly imaginary one, because the arm had a true thing to reach for and still answered the question that was asked.
correctcreate_agent floor probe (1.0.0) TASK 3, THE FLOOR PROBE, PASSED. from langchain.agents import create_agent, tools as @tool-decorated functions, system_prompt=, agent.invoke({"messages": [...]}), final answer read off result["messages"][-1].content, and the correct note that from langchain_core.tools import tool and langchain.tools are the same decorator. Nothing in this arm's control answers is discounted.
correct@tool(extras=...) / create_agent(middleware=...) TASK 4, BACKLOG 11k-i's DEPRECATION PAIR, AND ALL FOUR VERDICTS ARE RIGHT. extras: available yes, ever deprecated no - with the mechanism correct ("an extras field being added to BaseTool / the @tool decorator in the langchain-core 1.x line (late 2025) to carry provider-specific tool metadata ... through to the provider's tool schema") and flagged as low confidence. middleware: available yes, ever deprecated no, flagged high confidence. extras is LF26, added 1.2.0 (2025-12-15) - inside this subject's blind window and above its boundary - so this is a PASS on a live release, not a floor answer. No fabricated deprecation appeared: see the run summary.
correcttask 7 attribution TASK 7 IS AN ABSTENTION, NOT AN INVENTION, AND IT IS RECORDED AS THE RIGHT BEHAVIOUR FOR A SUBJECT IN THIS POSITION. Asked which release first allowed registering stream transformers, the arm said "I cannot name one ... If a release added it, it is after the point where my knowledge of langchain's release contents ends, and I would be guessing at a version number." JOURNAL/046: a denial is not an invention. The denial of the capability is charged as F1; declining to date it is the one part of the answer that is well calibrated.
contextboundary BOUNDARY SPREAD ON THIS PAIR IS NOW TWO MEASUREMENTS AND THEY DISAGREE BY ONE RELEASE. The langchain/v4 ladder placed Claude Fable 5.1 at 1.0.0 (describes 1.0.0, holds 1.1.0 as a number). This draw describes 1.1.0 in detail - "additional middleware ... model-profile awareness for capabilities, and fixes around structured output in create_agent" - and names 1.2 as the first release it knows only as a number. So the readings are 1.0.0 / 1.1.0, and per BACKLOG 11k-l a future battery in that band must treat the floor as the HIGHEST reading, 1.1.0. Nothing charged here turns on it: 1.3.0 is above both, and LF26 (1.2.0) was answered correctly rather than charged.

Sources

Battery specification: prompts/langchain.md in the studio repo. Every finding above also carries its own citation.