Section 2.1's prompt fixed two of the four client replies. The other two, the raw vendor numbers and "And NVIDIA?", were never prompt problems. Decide what the model reads on each call, and have code put it there. This section is taught in the same session as section 2.1. The session 2 lecture recording and slides are on the chapter 2 page.
▶ Open in Colab ``` ```{admonition} Learning objectives :class: note - Say what the context is on one call, and how context engineering differs from writing a prompt. - Assemble the context for one call from instructions, a client card, a retrieved policy clause and the last exchange. - Count what each piece costs, and decide what belongs in the prompt, in the context per call, and in a code check after the answer. - Hand an independent job to a sub-agent with a clean desk, and say what that saves and what it costs. ``` ## 2.2.1 From prompts to context Section 2.1 ended on a list of what a prompt cannot do. It costs every call, it cannot hold facts, it cannot hold what changes, and a long one gets ignored in the middle. The two replies it did not fix need exactly those things: a policy clause the prompt never mentioned, and the memo from two hours ago. The **context** is everything the model reads on one call. In the loop from chapter 1 that is the messages list: the system prompt, any earlier turns, the tool results so far, and the question. **Context engineering** is deciding what goes in that list, in what order, at what cost, for every call, and having code do it rather than a person. ```{figure} _static/media/ch03-context-parts.png :alt: Venn diagram titled Context Engineering. One large circle labelled Context holds seven overlapping circles: Instructions / System Prompt, Long-Term Memory, State / History (short-term memory), Retrieved Information (RAG), User Prompt, Available Tools, and Structured Output, which sits inside Available Tools. :width: 640px What goes into the context on one call. The pieces overlap: a retrieved clause can sit in the system prompt, and a remembered fact can come back as retrieved text. ``` [Anthropic's engineering team](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) put it as the shift from finding the right words to finding the right *configuration of context*, and [Andrej Karpathy](https://x.com/karpathy/status/1937902205765607626) called it the delicate art of filling the window with just the right information for the next step. The picture this book uses is the analyst's desk. There is a fixed amount of room on it. Before each call, something has to decide what is on the desk and what stays in the filing cabinet. ```{raw} html :file: widgets/ch02-context-window.html ``` Tick *paste the whole handbook*, then swap it for *retrieve the one clause*. Same rule, same model: 6,000 tokens that push the question off the desk, or 30 that arrive only when they matter. Prompt engineering picks the words in one block. Context engineering picks the blocks. The prompt did not go away; it became the first block, and usually the smallest. ## 2.2.2 Context engineering at the firm Three client replies, three different desks. The only new code is `build_context`, which decides what comes out of the filing cabinet for each call. Pick a reply and read why each card is where it is: ```{raw} html :file: widgets/ch02-desk-assembler.html ``` Now run each desk for real. **The follow-up**, with and without the memo on the desk: ```{code-block} python :class: pyodide from agent import agent first = "Compare Deere's revenue growth with Caterpillar's last quarter" memo, log = agent(first, verbose=False) print("Without the last exchange on the desk:") agent("And NVIDIA?") print("\nWith it:") agent("And NVIDIA?", history=[("user", first), ("assistant", memo)]) ``` One lookup instead of three. The history tells the model what the question means, and it carries figures the model would otherwise fetch again. That is also why it is expensive, and why `build_context` keeps only the last exchange. **The vendor-data clause**, retrieved for this question only: ```{code-block} python :class: pyodide from agent import agent from context import build_context, describe_context q = "And the raw vendor numbers behind that?" system, history = build_context(q, client="meridian", history=[("user", first), ("assistant", memo)]) print("On the desk for this call:") describe_context(system, history, q) print() answer, log = agent(q, system=system, history=history) ``` The memo declines and cites `data-licensing-1`. Chapter 3 is about making that retrieval step reliable: how the handbook is chunked, how the search finds the right clause, and what to do when it finds the wrong one. **The client card**, with no history and no clause: ```{code-block} python :class: pyodide from agent import agent from context import build_context q = "What was Deere's revenue growth last quarter?" system, history = build_context(q, client="meridian") answer, log = agent(q, system=system) ``` A table, though nobody asked for one. That preference is held in the client's record, and code puts it on the desk. **Where does each thing live?** Sort what the firm needs the model to know: ```{raw} html :file: widgets/ch02-where-it-lives.html ``` **Checkpoint.** ```{raw} htmlCompliance updates the restricted list every Monday. The agent must never draft a note that recommends a restricted name. Where should the list live?