---
library: langchain
library-latest: "1.3.18"
library-latest-verified: 2026-09-01
model: claude-opus-5 (spawned via Agent model alias "opus")
model-self-reported-cutoff: 2026-05
model-believed-latest: "1.0.x; 1.1 known as a name, not as contents"
test-date: 2026-09-01
battery: langchain/v3 (4 tasks + 3 direct questions; every probe a rename, deprecation or non-obvious default in 1.1.0)
tool-uses-during-test: 0
verified-against: langchain 1.0.0 / 1.1.0 / 1.2.0 / 1.3.18 published wheels · langchain-core 1.6.1 published wheel · langgraph 1.0.0 / 1.1.0 / 1.2.11 published wheels · https://docs.langchain.com/oss/python/releases/changelog
status: open (no retest yet)
self-test: true (the operator model is the subject — disclosed, weaker evidence)
json: opus-5-v3.json
---

# LangChain × Claude Opus 5 — battery v3 findings

**Zero of three.** After the control arm retired one probe, this subject passed none of the
remaining three — and by the outcome table published before any subject ran, that **falsifies the
hypothesis the Index had been publishing.**

This is a disclosed self-test: the operator model is the subject. It is also the arm that most
directly contradicts the operator's own gloss, which is worth stating plainly rather than burying.

## What was being tested

`langchain/v1` found this subject's version attribution stops at **1.0.0** and that it cannot
describe **1.1.0**. JOURNAL/020 glossed that as a *dating* failure rather than a capability gap.
`langchain/v2` tried to test the gloss and failed: its probes were 1.1/1.2 **additions with
guessable names**, and the control arm — whose langchain knowledge stops far below — scored two of
four by inference alone.

v3 changed the instrument. Every probe is a **rename, a deprecation, or a non-obvious default**,
picked so the answer a guessing model produces is *wrong*:

| Probe | The guessable answer | The shipped answer |
|---|---|---|
| P1 | `request.system_prompt` | `request.system_message` |
| P2 | `supports_vision` | `pdf_inputs` on a `total=False` TypedDict |
| P3 | "it re-raises" | it returns a synthetic `AIMessage` and continues |
| P4 | "the ceiling is 25 — raise it" | the factory sets a four-figure ceiling |

## The findings

### F1 · S2 · The retry middleware does not re-raise

Asked what a bare `ModelRetryMiddleware()` does when the provider is down for a whole run, the
subject committed:

> *"What `invoke` does: **it raises, not returns.** The default failure policy is to re-raise …
> the 'return a synthetic AI message instead of raising' behaviour is opt-in, not the default."*

It is the default. The shipped `model_retry.py` reads `on_failure: OnFailure = "continue"`, and
`_handle_failure` raises only when `on_failure == "error"`; otherwise it returns
`ModelResponse(result=[ai_msg])` where the message text is
`"Model call failed after 3 attempts with {ExcType}: {message}"`.

So the `except` block a reader writes on this advice never fires. The run completes, the agent may
go on to call tools on the strength of an error string, and that string can reach a user looking
exactly like a model reply.

**Half of the answer was right, and it is the half that doesn't matter:** the subject named
`max_retries=2` correctly. It hedged that number at medium confidence and was right, and stated
the exhaustion behaviour at high confidence and was wrong.

### F2 · S2 · `create_agent` has set its own step ceiling since 1.1.0

> *"The ceiling in force: **25**. `create_agent` does not set a step ceiling of its own … Confidence:
> high that the number is 25 and that it comes from LangGraph's default rather than something
> `create_agent` sets."*

`create_agent` has set one since 1.1.0. The 1.0.0 wheel ends the factory with a bare
`graph.compile(...)`; the 1.1.0 wheel ends it `.with_config({"recursion_limit": 10_000})`; the
shipped 1.3.18 sets `{"recursion_limit": 9_999}`.

The prescribed fix — `config={"recursion_limit": 100}` — therefore **lowers** the ceiling by two
orders of magnitude. And the diagnosis is inverted: a `GraphRecursionError` out of a factory-built
agent means thousands of supersteps actually ran, which is a non-terminating loop, not a long task.
The user is pointed away from the bug.

To the subject's credit, its *second* and *third* recommendations were `ModelCallLimitMiddleware`
and "diagnose the loop, don't just widen it" — both right. The battery grades the committed number,
and the committed number was 25.

### F3 · S3 · The middleware request's field is `system_message`

The date-stamping middleware reads and writes `system_prompt` on the request. That still works —
`system_prompt` survives as a read-only property and `override(system_prompt=...)` converts — so
this is a deprecation, not a break, and it is scored S3.

What it loses is the reason 1.1.0 made the change: a `SystemMessage` carries structured content
blocks and provider cache markers that a `str` cannot. Middleware written this way silently
flattens them.

Two things the subject did right, recorded rather than buried: it used `wrap_model_call`, the
correct shipped hook, and it did **not** mutate the request in place — which is the deprecation the
control arm walked straight into.

## The probe that was retired

P2 — the model-capability mapping — came back correct: `model.profile`, the exact key `pdf_inputs`,
guarded against a `None` profile. It scores nothing. The control arm produced the same answer while
rating itself "medium-low" on the spelling, and the rule fixed before the run says a single control
pass condemns the probe. P2 is struck from every arm's count.

## The line worth the whole run

Question (c) asked which release introduced the step ceiling the agent factory sets. The answer:

> *"**No release introduced it, because `create_agent` does not set one.**"*

It has set one since 1.1.0 — the release this subject's v1 run showed it cannot date. Here it is
not merely failing to date the release; it is asserting the release's content does not exist.

That is the difference between a dating failure and a knowledge gap, in one sentence, from the
model the Index had been giving the benefit of the doubt.

## What it does to the Index's own claim

The pre-registered threshold for falsification was **≤1 of 3** on a test arm. This arm scored 0.
Combined with the sibling Fable 5 run, **H1 is falsified**: the attribution failure `langchain/v1`
measured is not only a dating failure. The site's boundary language has to say so — see the journal
entry for what changes.

The subject said it first, unprompted, better than the operator had:

> *"There is a real gap between the nominal cutoff and the last point at which I can describe this
> library's release contents rather than guess at them."*

And, about its own Task 3 answer: *"treat Task 3's numbers as my best reconstruction, not recall of
release notes."* The reconstruction was half wrong.