| correct | langchain.agents.create_agent |
Task 2: from langchain.agents import create_agent and from langchain.tools import tool, with a note that the result is a compiled LangGraph graph whose state is {"messages": [...]}. |
| correct | create_agent(system_prompt=...) |
Task 3: system_prompt=, with the migration history stated correctly — "in the older langgraph.prebuilt.create_react_agent it was prompt=" — and a usable diagnostic: "If system_prompt raises a TypeError, you are on a pre-1.0 install." |
| correct | langchain.chains (LLMChain, ConversationChain, RetrievalQA) |
Task 4: hand-written retrieve-then-answer with langchain_text_splitters and init_embeddings, and an explicit refusal of the removed chains — "I did not reach for RetrievalQA or ConversationalRetrievalChain ... they live in langchain-classic if you truly need them." |
| correct | langchain.memory (ConversationBufferMemory) |
Task 5: LangGraph checkpointer keyed by thread_id, with the mental model stated outright — "memory is a LangGraph checkpointer ... not a ConversationBufferMemory object hanging off a chain. The old memory= classes are gone from langchain 1.x." |
| correct | langchain.hub |
Task 6: langsmith.Client().pull_prompt(...), with the removal called out — "the old from langchain import hub; hub.pull(\"...\") is the 0.x way and is not in langchain 1.x". The only subject to state the removal rather than trip over it. |
| correct | create_agent(response_format=...) |
Task 7: ToolStrategy(WeatherAnswer) from langchain.agents.structured_output, the result["structured_response"] key, and the correct nuance that a bare schema also works and lets the framework pick. |
| correct | run-scoped context (context= / context_schema=) |
Task 8: ToolRuntime[Context] from langchain.tools plus context_schema and context= at invoke time, with the security property stated correctly — the parameter is stripped from the schema shown to the model. |
| correct | create_agent(pre_model_hook=...) / post_model_hook |
Task 9: @wrap_model_call middleware, chosen over before_model for the stated reason that it shapes only what is sent and leaves persisted state intact — plus an orphaned-ToolMessage guard neither other subject produced. |
| correct | agent streaming node name |
Task 10: filters on langgraph_node == "model", and names the trap explicitly — "in the older langgraph.prebuilt.create_react_agent it was \"agent\"". |
| correct | message.text |
Task 1 and 11: .text used as a property throughout, with the version history attached — "response.text is a property on AIMessage in 1.x ... if .text gives you a bound method, you're on an older version". |
| imprecision | ChatAnthropic(max_tokens=...) |
Task 11 stated the Anthropic default as "a small default (1024 in the versions I know)". 1.0.0 changed that default to a per-model value. (Code-vs-claim rule. The claim is hedged and the code sets max_tokens=8000 explicitly, which is the correct fix and works regardless of the default. The generated code is not wrong; the stated reason is out of date.) |
| imprecision | — |
Dated the 1.0 GA as "roughly October 22, 2025"; PyPI has the files uploaded 2025-10-17 and the vendor changelog labels the entry Oct 20, 2025. (Explicitly approximate ("roughly", "I associate it with"), and within a week of the true date. The Index charges wrong version facts, not rounding.) |
| context | — |
Asked for its cutoff, it separated the nominal date from the useful one unprompted: "my reliable, detailed knowledge of this particular library is noticeably older than that — it thins out sharply after the 1.0 launch in late 2025 ... for langchain, my effective cutoff is late 2025 / very early 2026, not May 2026." That is precisely what the measurement found, and it is the metric this dataset exists to produce. |