Agents, loops, and the words around them

A model wrapped in a loop that can call tools, observe the results, and decide its next action — not one prompt-to-text call, but a running process that decides what to do next. The line between "agent" and "workflow" is how much the model itself decides the next step.

The program running the loop around the model: it sends the prompt, parses tool calls out of the response, executes them, feeds results back, and decides when to stop. Claude Code, an IDE plugin, a CLI — all harnesses. The model provides judgment; the harness provides the loop, the tools, and the guardrails around both.

The instructions the harness prepends before the conversation starts — persona, rules, available tools, output format. Set once per session, invisible to the end user, and treated by the model as higher-authority than user turns. Not a magic override — a strong user instruction can still contradict it in the model's eyes.

The model outputs a structured request — a function name and arguments — instead of prose, and the harness executes it and returns the result as a new message. This is what turns a text generator into something that can act. The model never runs the tool itself; it only ever asks for it to be run.

The core control structure: the model reasons about what to do, calls a tool, sees the result, and repeats — with no fixed number of steps — until it decides the task is done or a stop condition is hit. Unbounded by design. Real harnesses cap it with a max-iterations or budget guard.

A control flow expressed as nodes (a call to the model, a tool, a check) and edges (which node runs next), often with conditional branches and cycles. LangGraph and similar frameworks compile agents this way instead of a single implicit loop. Trades the loop's flexibility for explicit, inspectable, resumable control flow.

A fixed sequence of steps — prompt A, then prompt B on its output, then a parser — with no branching and no loop. The simplest control flow, and often all a task needs. If you're not sure whether you need an agent, you probably need a chain.

One call produces a multi-step plan up front; a separate loop — often a simpler, cheaper one — executes each step and only calls back to the planner if a step fails or the plan needs revising. Cuts planning cost: you pay for one expensive reasoning pass, not one per step.

A cheap first step that classifies the incoming request and sends it to the right specialized prompt, tool set, or sub-agent — instead of one generalist prompt handling everything. Usually the highest-leverage place to put a small, fast, cheap model.

The total tokens the model can attend to in one call — system prompt, conversation history, tool results, everything — all competing for the same fixed budget. Bigger isn't free: relevant information buried in a huge context is still easy for the model to under-weight.

Deliberately deciding what to put into the context window and in what order — which files, which past turns, which tool results, summarized or verbatim — since the model can only act on what's actually there. The successor to "prompt engineering" once the hard part became curation, not phrasing.

Fetching relevant documents from an external store — usually by vector or keyword search — and inserting them into the context before generating an answer, grounding the model in facts it wasn't trained on. Only as good as the retrieval step: a great model on bad search results still answers wrong.

Short-term memory is just the current context window. Long-term memory is state written to external storage — a file, a database row, a vector store — deliberately saved and re-loaded across sessions. A model has no memory of its own; every "memory" is something the harness chose to persist and re-inject.

Reusing the model's internal computation for a prefix of the input that hasn't changed since the last call, so a long, stable system prompt or tool schema doesn't get reprocessed — and re-paid for — on every turn. Put stable content (system prompt, tool defs) first and volatile content (the latest user turn) last, so the prefix actually stays stable.

A named capability the harness exposes to the model with a schema — a description and typed parameters — that the model can invoke by name. Search the web, read a file, run SQL: each is one tool. The description in the schema matters as much as the code — it's the model's only way of knowing when to reach for it.

An open standard for exposing tools, data sources and prompts to any compatible agent over a common protocol, so a server built once — a database connector, a search API — plugs into many different harnesses without custom glue code. Think of it as a USB-C port for tools: one interface, many devices on either side.

Code the harness runs at fixed points in the loop — before a tool executes, after the model responds, when a session starts — independent of the model's own reasoning. Used to enforce policy, log, or block an action outright. Deterministic where the model is not: a hook always runs, whether or not the model "remembers" the rule.

A restricted environment — a container, a VM, a scoped filesystem — where an agent's actions (running code, writing files) are contained, so a mistaken or adversarial action can't reach the host system. The blast-radius question to ask before granting any tool: what's the worst this can do inside the box?

An agent that operates a real user interface — moving a cursor, clicking, typing, reading the screen — instead of calling structured APIs, for the many tasks that only exist as a GUI. Slower and less reliable than an API call; reach for it only when no API exists.

One agent (the orchestrator) breaks a task apart and delegates pieces to other agents (sub-agents), each with a narrower scope, then assembles their results. The orchestrator never does the work itself. Give sub-agents fresh, minimal context — don't just hand them the orchestrator's whole history.

Several agents working on the same problem, each with its own context, role and sometimes its own model, communicating through messages, a shared document, or a coordinator — instead of one agent doing everything. Adds coordination overhead and cost; only worth it when the roles genuinely benefit from separate context.

One agent explicitly passes control of the conversation to another, along with enough context to continue, instead of calling it as a sub-routine and waiting for a return value. The receiving agent takes over fully. Different from delegation: a handoff doesn't come back to the first agent unless handed off again.

Multiple agents read and write to one shared piece of state — a document, a scratchpad — instead of messaging each other directly, each contributing when it has something relevant to add. Simpler to add new agents to than a fixed message-passing topology; harder to reason about who changed what.

Checks that run on what goes into the model or comes out of it — a content filter, a schema validator, a policy check — that can block or rewrite a step before it takes effect. Distinct from a hook: guardrails specifically judge content, not just fire at a point in time. Put the expensive check on the output, the cheap one on the input — most bad inputs never reach the model at all.

A required pause for human approval before a consequential action executes — sending an email, deleting data, spending money — instead of letting the agent act fully autonomously. Scale the approval to the blast radius: read-only actions rarely need it, irreversible ones almost always should.

A test suite for a model or an agent's behaviour: a fixed set of inputs with a way to score the outputs, run before shipping a prompt or model change to see whether it actually made things better. Without evals, every prompt tweak is a guess — you're optimizing on vibes, not a number that can regress.

Tying a model's output to a verifiable external source — a retrieved document, a tool result, a citation — so a claim can be checked, instead of relying purely on what the model recalls from training. RAG is one way to ground; a calculator tool grounding an arithmetic answer is another.

The model producing a confident, fluent, and wrong statement — a fabricated citation, an API that doesn't exist — with no signal to the reader that it's unsupported. A generation-quality problem, not a bug in the traditional sense. Grounding reduces it; nothing eliminates it — always treat unverified model output as a claim, not a fact.

Continuing to train an already-trained model on a smaller, task-specific dataset, adjusting its weights so the behaviour shifts permanently — unlike a prompt, which only steers that one call. Expensive and slow to iterate on compared to a prompt; reach for it only after prompting has hit a real ceiling.

Training a model to prefer outputs that humans rate as better — humans rank candidate responses, a reward model learns from those rankings, and the model is optimized against that reward signal. What turns a raw text predictor into something that behaves like an assistant.

The model picking up a pattern from examples placed directly in the prompt — no weight updates, no training run — and applying it to a new input in the same call. Cheaper than fine-tuning and reversible; costs context-window space on every single call instead.

Training a smaller, cheaper model to imitate a larger one's outputs, so most of the capability survives at a fraction of the size and inference cost. The usual path from "we prototyped with the biggest model" to "we ship with a small one".

Prompting or training a model to write out intermediate reasoning steps before its final answer, which measurably improves accuracy on multi-step problems — and gives a human something to inspect for where the reasoning went wrong. Modern reasoning models generate long internal chains of thought automatically, often hidden from the final output.

Running a trained model forward to produce an output — as opposed to training, which adjusts its weights. Everything an agent does at runtime is inference; training already happened, long before, somewhere else.

The unit the model actually reads and writes — usually a few characters, not a whole word. Cost, context-window limits and rate limits are all measured in tokens, not words or characters. Rough rule of thumb: about 4 characters of English per token.

A parameter controlling how much randomness goes into picking the next token. Low (near 0) is nearly deterministic and repetitive; high pushes toward more varied, sometimes less coherent, output. Use low temperature for tool calls and structured output, higher only where creative variety is actually wanted.

Sending each token to the client as soon as it's generated, instead of waiting for the full response, so a user sees words appear progressively rather than staring at a blank screen until it's all done.

Time-to-first-token (TTFT) is the delay before the first token appears; tokens-per-second is the speed once it starts. Together they determine how an agent actually feels to wait on — a slow TTFT feels broken even if throughput is fine.