{
  "$schema": "../../schema/run.schema.json",
  "run_id": "langchain--claude-sonnet-5--v1--2026-08-31",
  "markdown": "data/langchain/sonnet-5.md",
  "supersedes": null,
  "library": {
    "name": "langchain",
    "ecosystem": "pypi",
    "latest_version_at_test": "1.3.18",
    "latest_version_verified_on": "2026-08-31"
  },
  "model": {
    "id": "claude-sonnet-5",
    "label": "Claude Sonnet 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model alias \"sonnet\"",
    "self_reported_cutoff": "2026-01",
    "cutoff_basis": "Self-reported: \"I'm configured/told my cutoff is January 2026, and I'll state that as the formal answer.\"",
    "believed_latest_version": "0.3.x",
    "believed_latest_quote": "\"0.3.x is the last release I can actually describe with real detail — specific module layout, the deprecation of `initialize_agent`/`AgentExecutor`, `with_structured_output` becoming standard. I'm aware, vaguely, of talk around a 1.0 release existing or being planned, but that's a version string I've encountered in passing, not a release whose contents I could walk through — I don't trust myself to state what changed in it.\"",
    "knowledge_stops_at_version": "0.3.0",
    "knowledge_stops_on": "2024-09-13",
    "knowledge_gap_starts_at_version": "1.0.0",
    "knowledge_gap_starts_on": "2025-10-17",
    "cutoff_lag_months": 16
  },
  "test": {
    "date": "2026-08-31",
    "battery": "langchain/v1",
    "battery_spec": "prompts/langchain.md",
    "prompt_file": null,
    "tasks": 11,
    "direct_questions": 3,
    "tool_uses_during_test": 0,
    "probe_window": { "from": "1.0.0", "to": "1.2.0" },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://docs.langchain.com/oss/python/releases/langchain-v1",
    "https://docs.langchain.com/oss/python/migrate/langchain-v1",
    "https://docs.langchain.com/oss/python/migrate/langgraph-v1",
    "https://docs.langchain.com/oss/python/releases/changelog",
    "https://reference.langchain.com/python/langchain/",
    "https://reference.langchain.com/python/langchain-classic/",
    "https://pypi.org/pypi/langchain/json"
  ],
  "summary": "The largest measured knowledge gap in the Index. This subject's usable LangChain knowledge stops at 0.3 (2024-09-13) — sixteen months before its own stated cutoff of 2026-01, and thirteen months earlier than the other two subjects on the same library. It writes the 0.3-era stack throughout: `langgraph.prebuilt.create_react_agent` with `prompt=`, `pre_model_hook=`, a stream filtered on the `\"agent\"` node, and `from langchain import hub`. Only the last of those is an outright ImportError, because LangGraph deprecated its prebuilt rather than deleting it — so the damage here is not a stack trace, it is a developer building today on the stack the framework moved off eleven months ago. Three findings, all chargeable. Notably, the subject diagnosed the gap itself: it volunteered that its \"sharp, specific knowledge\" of this library runs only to \"roughly mid-to-late 2024\", which is exactly where the measurement puts it.",
  "findings": [
    {
      "id": "F1",
      "severity": "S3",
      "severity_label": "deprecated",
      "title": "Builds agents with the deprecated LangGraph prebuilt instead of create_agent",
      "api": "langgraph.prebuilt.create_react_agent",
      "change_kind": "renamed",
      "introduced_in": "1.0.0",
      "introduced_on": "2025-10-17",
      "chargeable": true,
      "chargeable_note": "langchain 1.0.0 and LangGraph v1 both shipped 2025-10-17, roughly three months before this subject's stated 2026-01 cutoff.",
      "model_belief": "\"I reach for `create_react_agent` from `langgraph.prebuilt` here, not the older `langchain.agents.initialize_agent`/`AgentExecutor`, which I consider legacy at this point.\" Asked directly (question c), it named the same path: \"The recommended way to build an agent is LangGraph — either the prebuilt `create_react_agent` for a standard tool-calling loop, or a hand-built `StateGraph`.\"",
      "wrong_code": "from langgraph.prebuilt import create_react_agent\n\nagent = create_react_agent(llm, tools=[get_weather, calculate])\nagent = create_react_agent(llm, tools=[...], prompt=SYSTEM_PROMPT)",
      "correct_code": "from langchain.agents import create_agent\n\nagent = create_agent(llm, tools=[get_weather, calculate],\n                     system_prompt=SYSTEM_PROMPT)",
      "impact": "S3 rather than S1 because the prebuilt is deprecated, not deleted — this code still imports and runs. The cost is compounding: the model knows the *previous* migration (it explicitly rejects `initialize_agent`/`AgentExecutor` as legacy) and has no idea a second one happened. Everything it builds on top inherits the old surface — `prompt=` instead of `system_prompt=`, `pre_model_hook=` instead of middleware, and a stream filtered on the `\"agent\"` node. Those three are individually correct *for the deprecated prebuilt*, so they are not charged separately; but a developer who later takes the framework's own advice and swaps in `create_agent` will find the first two raise `TypeError` and the third silently stops matching.",
      "citations": [
        {
          "url": "https://docs.langchain.com/oss/python/migrate/langgraph-v1",
          "title": "LangGraph v1 migration guide — Deprecations",
          "quote": "LangGraph v1 deprecates the create_react_agent prebuilt. Use LangChain's create_agent, which runs on LangGraph and adds a flexible middleware system."
        },
        {
          "url": "https://docs.langchain.com/oss/python/releases/langchain-v1",
          "title": "What's new in LangChain v1 — create_agent",
          "quote": "The new standard for building agents in LangChain, replacing langgraph.prebuilt.create_react_agent."
        }
      ],
      "scope_note": "Verified that the prebuilt is deprecated rather than removed; the migration guide lists it under Deprecations with a replacement, and gives no removal version. The subject's own hedge on the parameter name (\"I've seen `state_modifier` and `prompt` both used\") is recorded as an imprecision, not charged."
    },
    {
      "id": "F2",
      "severity": "S1",
      "severity_label": "breaks-build",
      "title": "`from langchain import hub` — removed from the package in 1.0.0",
      "api": "langchain.hub",
      "change_kind": "removed",
      "introduced_in": "1.0.0",
      "introduced_on": "2025-10-17",
      "chargeable": true,
      "chargeable_note": "1.0.0 published 2025-10-17, inside the subject's stated 2026-01 window.",
      "model_belief": "Gave `pip install langchainhub langchain` and `from langchain import hub; prompt = hub.pull(\"hwchase17/react\")`. It flagged its own uncertainty — \"I'm not fully confident this call signature is still current — it's the kind of thing that shifted after my knowledge gets thin\" — but did not name the current path.",
      "wrong_code": "# pip install langchainhub langchain\nfrom langchain import hub\n\nprompt = hub.pull(\"hwchase17/react\")",
      "correct_code": "# pip install langchain-classic\nfrom langchain_classic import hub\nprompt = hub.pull(\"hwchase17/react\")\n\n# or, without the compatibility package:\n# pip install langsmith\nfrom langsmith import Client\nprompt = Client().pull_prompt(\"hwchase17/react\")",
      "impact": "`ImportError` on the second line of a fresh v1 environment, and the install line does not fix it — `langchain-classic` is a separate distribution that `pip install langchain` does not pull in. Charged under the code-vs-claim rule: the hedge admits uncertainty but names no correct alternative, so the code is the answer the developer gets.",
      "citations": [
        {
          "url": "https://docs.langchain.com/oss/python/migrate/langchain-v1",
          "title": "LangChain v1 migration guide — update your imports",
          "quote": "from langchain import hub  # [!code --]"
        },
        {
          "url": "https://reference.langchain.com/python/langchain-classic/hub/pull",
          "title": "API reference — langchain_classic.hub.pull"
        }
      ]
    },
    {
      "id": "F3",
      "severity": "S4",
      "severity_label": "wrong-metadata",
      "title": "Version knowledge stops at 0.3, sixteen months before its stated cutoff",
      "api": null,
      "change_kind": "version-fact",
      "introduced_in": "1.0.0",
      "introduced_on": "2025-10-17",
      "chargeable": true,
      "chargeable_note": "Anchored to 1.0.0 (2025-10-17), the first release the subject cannot describe, not to the current 1.3.18. 1.0.0 precedes the stated 2026-01 cutoff by roughly three months.",
      "model_belief": "\"0.3.x is the last release I can actually describe with real detail ... I'm aware, vaguely, of talk around a 1.0 release existing or being planned, but that's a version string I've encountered in passing, not a release whose contents I could walk through.\" Asked what `langchain` exports today, it answered: \"the top-level `langchain` package itself is fairly thin now — mostly legacy chains, `hub`, and some retrieval helpers\" — a description of 0.3, and the precise inverse of v1, where chains, hub and retrievers are the things that left.",
      "wrong_code": null,
      "correct_code": null,
      "impact": "Sixteen months of lag, the largest in the Index, on a library that shipped a namespace rewrite inside the gap. It is not a missing detail: the subject cannot describe the single release that determines whether any of its generated imports resolve. This is also the widest inter-model spread the Index has recorded — the other two subjects, on the same battery on the same day, both describe 1.0.0 accurately.",
      "citations": [
        {
          "url": "https://pypi.org/pypi/langchain/json",
          "title": "PyPI JSON API — langchain release timestamps (1.0.0 uploaded 2025-10-17)"
        },
        {
          "url": "https://docs.langchain.com/oss/python/releases/langchain-v1",
          "title": "What's new in LangChain v1"
        }
      ]
    }
  ],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "Task 1: a direct `ChatOpenAI` call with core message objects. Nothing in it depends on the reduced namespace, and it runs unchanged on v1.",
      "api": null
    },
    {
      "kind": "correct",
      "summary": "Task 4: the RAG pipeline uses `langchain_text_splitters`, `langchain_community` loaders, `langchain_chroma` and LCEL — all still valid distributions on v1. It does not reach for `RetrievalQA`, the chain that would have failed.",
      "api": "langchain.chains (LLMChain, ConversationChain, RetrievalQA)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 5: memory as a LangGraph checkpointer keyed by `thread_id`, explicitly rejecting the older `RunnableWithMessageHistory` wrapper for an agent. `MemorySaver` still exists alongside `InMemorySaver` in the current docs, so the older spelling is not charged.",
      "api": "langchain.memory (ConversationBufferMemory)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 7: `with_structured_output(Answer)` on a plain model — stable across 0.2, 0.3 and 1.x.",
      "api": null
    },
    {
      "kind": "correct",
      "summary": "Task 8: a `RunnableConfig`-typed tool parameter to inject the user id and keep it out of the tool schema. Still supported in v1; the migration guide states the `config[\"configurable\"]` route works for backward compatibility.",
      "api": "run-scoped context (context= / context_schema=)",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "imprecision",
      "summary": "Task 3 hedged the parameter name — \"this kwarg's name moved around across langgraph releases — I've seen `state_modifier` and `prompt` both used\". Both names belong to the deprecated prebuilt; the current answer, `system_prompt` on `create_agent`, is not among them.",
      "api": "create_agent(system_prompt=...)",
      "introduced_in": "1.0.0",
      "why_not_a_finding": "Folded into F1. `prompt=` is correct for the function this subject actually called, so charging it separately would double-count the same stale belief."
    },
    {
      "kind": "imprecision",
      "summary": "Task 9 used `pre_model_hook=` and task 10 filtered the stream on the `\"agent\"` node. Both are correct for `langgraph.prebuilt.create_react_agent` and wrong for `create_agent`, where the parameter does not exist and the node is named `\"model\"`.",
      "api": "agent streaming node name",
      "introduced_in": "1.0.0",
      "why_not_a_finding": "Same reason as above — consequences of F1, not independent beliefs. Recorded because they are exactly what breaks when a developer migrates the agent factory and nothing else."
    },
    {
      "kind": "context",
      "summary": "The subject volunteered the gap before it was measured: \"my confident, detailed recall of a fast-moving library like langchain thins out well before that date — realistically my sharp, specific knowledge ... is strongest through roughly mid-to-late 2024.\" The measurement puts its boundary at 0.3.0, 2024-09-13. All three subjects in this battery self-reported their own boundary accurately.",
      "api": null
    }
  ],
  "open_questions": [
    {
      "question": "Does the deprecated `langgraph.prebuilt.create_react_agent` still accept `pre_model_hook=`, and is its model node still named `\"agent\"`, on current LangGraph?",
      "status": "open",
      "resolution": "Not charged either way. The v1 migration guide documents the deprecation and its replacement but does not enumerate the prebuilt's surviving parameters, and no primary source was found that states them for the current release. Treated as unverified rather than assumed."
    }
  ]
}