{
  "$schema": "../../schema/run.schema.json",
  "run_id": "langchain--claude-opus-5--v1--2026-08-31",
  "markdown": "data/langchain/opus-5.md",
  "supersedes": null,
  "library": {
    "name": "langchain",
    "ecosystem": "pypi",
    "latest_version_at_test": "1.3.18",
    "latest_version_verified_on": "2026-08-31"
  },
  "model": {
    "id": "claude-opus-5",
    "label": "Claude Opus 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model alias \"opus\"",
    "self_reported_cutoff": "2026-05",
    "cutoff_basis": "Self-reported: \"My best understanding is that my knowledge runs to roughly May 2026.\"",
    "believed_latest_version": "1.0.x",
    "believed_latest_quote": "\"The most recent release whose contents I can actually describe: `langchain` 1.0, GA around late October 2025 ... What I cannot describe: any release after the 1.0 line's early patches. If a 1.1 or later shipped a new API, I don't know its shape, and I would not recognize a feature added in it.\"",
    "knowledge_stops_at_version": "1.0.0",
    "knowledge_stops_on": "2025-10-17",
    "knowledge_gap_starts_at_version": "1.1.0",
    "knowledge_gap_starts_on": "2025-11-24",
    "cutoff_lag_months": 7
  },
  "test": {
    "date": "2026-08-31",
    "battery": "langchain/v1",
    "battery_spec": "prompts/langchain.md",
    "prompt_file": null,
    "tasks": 11,
    "direct_questions": 3,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "1.0.0",
      "to": "1.2.0"
    },
    "self_test": true,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://docs.langchain.com/oss/python/releases/langchain-v1",
    "https://docs.langchain.com/oss/python/migrate/langchain-v1",
    "https://docs.langchain.com/oss/python/migrate/langgraph-v1",
    "https://docs.langchain.com/oss/python/releases/changelog",
    "https://docs.langchain.com/oss/python/langchain/tools",
    "https://docs.langchain.com/oss/python/langgraph/persistence",
    "https://reference.langchain.com/python/langchain/",
    "https://reference.langchain.com/python/langchain-classic/",
    "https://pypi.org/pypi/langchain/json"
  ],
  "summary": "Eleven code tasks, eleven pieces of working LangChain v1. `create_agent` with `system_prompt`, middleware via `wrap_model_call`, `ToolRuntime` for run-scoped context, `ToolStrategy` for structured output, the `\"model\"` node name in the stream filter, `.text` as a property, and a deliberate refusal to reach for `RetrievalQA` or the memory classes. One finding, and like Tailwind it is not about code: version attribution stops at 1.0.0 (2025-10-17) — it can describe that release and cannot describe 1.1.0 or 1.2.0, both of which shipped inside the subject's stated 2026-05 window. Battery v1 does not probe those releases' own features, so what v1 measures is a dating failure and not a demonstrated feature gap. AMENDED 2026-09-01: battery v3 did probe them, and found the gap — this subject failed all three surviving probes into 1.1.0 (JOURNAL/022). The sentence above still describes v1's own scope correctly; it no longer implies that no gap exists. Seven months of lag — its best result in the Index, on the library where the other subject with the same architecture family lands thirteen months behind it. Disclosed self-test — the operator model is the subject.",
  "findings": [
    {
      "id": "F1",
      "severity": "S4",
      "severity_label": "wrong-metadata",
      "title": "Version knowledge stops at 1.0.0 while 1.1.0 and 1.2.0 shipped inside its window",
      "api": null,
      "change_kind": "version-fact",
      "introduced_in": "1.1.0",
      "introduced_on": "2025-11-24",
      "chargeable": true,
      "chargeable_note": "Anchored to 1.1.0 (2025-11-24), the first release the subject cannot describe — not to the current 1.3.18. 1.1.0 precedes the stated 2026-05 cutoff by roughly six months, and 1.2.0 (2025-12-15) by five. The same finding would NOT be chargeable against a subject whose cutoff preceded 2025-11-24.",
      "model_belief": "\"Version numbers I have seen referenced: the `1.0.x` line for certain, and I believe the line continued into `1.1.x` and possibly further during early 2026. I do not trust myself to name a specific latest patch number — if I said '1.2.3' I would be fabricating precision.\" It could describe 1.0.0's contents in accurate detail and nothing after it.",
      "wrong_code": null,
      "correct_code": null,
      "impact": "Low on its own — nothing in 1.1.0 or 1.2.0 breaks 1.0 code. Both are additive: model profiles on chat models, `SystemMessage` accepted for `system_prompt`, model-retry middleware, tool `extras`, strict schema adherence for `ProviderStrategy`. The value is diagnostic. This subject has the latest cutoff in the Index and, on this library, its smallest lag yet: seven months against thirteen on Tailwind and seven on Next.js.",
      "citations": [
        {
          "url": "https://docs.langchain.com/oss/python/releases/changelog",
          "title": "LangChain changelog — langchain v1.1.0",
          "published_on": "2025-11-25",
          "quote": "Chat models now expose supported features and capabilities through a .profile attribute."
        },
        {
          "url": "https://pypi.org/pypi/langchain/json",
          "title": "PyPI JSON API — langchain release timestamps (1.1.0 uploaded 2025-11-24)"
        }
      ]
    }
  ],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "Task 2: `from langchain.agents import create_agent` and `from langchain.tools import tool`, with a note that the result is a compiled LangGraph graph whose state is `{\"messages\": [...]}`.",
      "api": "langchain.agents.create_agent",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 3: `system_prompt=`, with the migration history stated correctly — \"in the older `langgraph.prebuilt.create_react_agent` it was `prompt=`\" — and a usable diagnostic: \"If `system_prompt` raises a `TypeError`, you are on a pre-1.0 install.\"",
      "api": "create_agent(system_prompt=...)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 4: hand-written retrieve-then-answer with `langchain_text_splitters` and `init_embeddings`, and an explicit refusal of the removed chains — \"I did not reach for `RetrievalQA` or `ConversationalRetrievalChain` ... they live in `langchain-classic` if you truly need them.\"",
      "api": "langchain.chains (LLMChain, ConversationChain, RetrievalQA)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 5: LangGraph checkpointer keyed by `thread_id`, with the mental model stated outright — \"memory is a LangGraph checkpointer ... not a `ConversationBufferMemory` object hanging off a chain. The old `memory=` classes are gone from `langchain` 1.x.\"",
      "api": "langchain.memory (ConversationBufferMemory)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 6: `langsmith.Client().pull_prompt(...)`, with the removal called out — \"the old `from langchain import hub; hub.pull(\\\"...\\\")` is the 0.x way and is not in `langchain` 1.x\". The only subject to state the removal rather than trip over it.",
      "api": "langchain.hub",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 7: `ToolStrategy(WeatherAnswer)` from `langchain.agents.structured_output`, the `result[\"structured_response\"]` key, and the correct nuance that a bare schema also works and lets the framework pick.",
      "api": "create_agent(response_format=...)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 8: `ToolRuntime[Context]` from `langchain.tools` plus `context_schema` and `context=` at invoke time, with the security property stated correctly — the parameter is stripped from the schema shown to the model.",
      "api": "run-scoped context (context= / context_schema=)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 9: `@wrap_model_call` middleware, chosen over `before_model` for the stated reason that it shapes only what is sent and leaves persisted state intact — plus an orphaned-`ToolMessage` guard neither other subject produced.",
      "api": "create_agent(pre_model_hook=...) / post_model_hook",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 10: filters on `langgraph_node == \"model\"`, and names the trap explicitly — \"in the older `langgraph.prebuilt.create_react_agent` it was `\\\"agent\\\"`\".",
      "api": "agent streaming node name",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 1 and 11: `.text` used as a property throughout, with the version history attached — \"`response.text` is a property on `AIMessage` in 1.x ... if `.text` gives you a bound method, you're on an older version\".",
      "api": "message.text",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "imprecision",
      "summary": "Task 11 stated the Anthropic default as \"a small default (1024 in the versions I know)\". 1.0.0 changed that default to a per-model value.",
      "api": "ChatAnthropic(max_tokens=...)",
      "introduced_in": "1.0.0",
      "why_not_a_finding": "Code-vs-claim rule. The claim is hedged and the code sets `max_tokens=8000` explicitly, which is the correct fix and works regardless of the default. The generated code is not wrong; the stated reason is out of date."
    },
    {
      "kind": "imprecision",
      "summary": "Dated the 1.0 GA as \"roughly October 22, 2025\"; PyPI has the files uploaded 2025-10-17 and the vendor changelog labels the entry Oct 20, 2025.",
      "api": null,
      "why_not_a_finding": "Explicitly approximate (\"roughly\", \"I associate it with\"), and within a week of the true date. The Index charges wrong version facts, not rounding."
    },
    {
      "kind": "context",
      "summary": "Asked for its cutoff, it separated the nominal date from the useful one unprompted: \"my *reliable, detailed* knowledge of this particular library is noticeably older than that — it thins out sharply after the 1.0 launch in late 2025 ... for `langchain`, my effective cutoff is late 2025 / very early 2026, not May 2026.\" That is precisely what the measurement found, and it is the metric this dataset exists to produce.",
      "api": null
    }
  ],
  "open_questions": []
}