The Agent Loop, For Real: Termination, Budgets, Idempotency, and What Actually Breaks
- An agent loop is six boxes: assemble, call, parse, validate, execute, observe. Everything that makes it survive production is outside the loop.
- Retrying an LLM call with an unchanged context reproduces the same failure, because the model is near-deterministic given identical input. A retry must mutate the context or it is only a way to pay twice.
- Fifty tool schemas at roughly 150 tokens each add 7,500 tokens to every turn. Across a 20-turn run that is 150,000 tokens of catalogue before the agent has done any work.
What is an agent loop, mechanically?
Assemble context, call the model, parse the tool call, validate it against a schema, execute it, append the result, repeat. That is the whole loop, about thirty lines. Production correctness lives entirely outside it: a step budget, a token budget, a wall-clock budget, an idempotency key persisted before every side effect, and a run journal in your own database rather than the framework's.
One phrase in common use is actively misleading. The model does not call a tool. It emits a structured request that says it would like a tool called, and your harness decides whether to honour it. Everything interesting in agent engineering happens in that gap: validating the arguments, checking a budget, resolving a credential, enforcing an allowlist, and deciding whether this call has already happened. Teams who internalise that the harness is the agent write much better systems than teams who think the model is.
The loop below is deliberately unglamorous. No streaming, no parallel tool calls, no human-in-the-loop interrupts, no multi-tenancy. I have listed those omissions rather than hiding them, because a snippet that quietly leaves out the hard parts is worse than no snippet. What it does have is every guard from the diagram above, in the order they belong.
The layer this sits inside is layer four of the full AI product architecture reference, where the run and step tables referenced below are defined in full.
type Status =
| "succeeded" | "failed" | "cancelled"
| "budget_exceeded" | "timed_out" | "stalled";
export async function runAgent(ctx: RunCtx): Promise<Status> {
const deadline = Date.now() + ctx.budgets.wallClockMs;
let steps = 0;
let tokens = 0;
const callHashes: string[] = [];
while (true) {
if (Date.now() > deadline) return finish(ctx, "timed_out");
if (steps >= ctx.budgets.maxSteps) return finish(ctx, "budget_exceeded");
if (tokens >= ctx.budgets.maxTokens) return finish(ctx, "budget_exceeded");
if (await isCancelled(ctx.runId)) return finish(ctx, "cancelled");
// 1. assemble — stable parts first, so the cached prefix survives
const messages = assemble({
system: ctx.systemPrompt, // stable
tools: ctx.toolSubset, // stable FOR THE WHOLE RUN
fewShot: ctx.fewShot, // stable
task: ctx.task, // stable
history: ctx.history, // grows
});
// 2. call — the middleware records tokens, cache status, price snapshot
const res = await ctx.model.call({ messages, tools: ctx.toolSubset });
tokens += res.usage.inputTokens + res.usage.outputTokens;
await journal.writeModelStep(ctx, steps, res);
if (res.stopReason === "end_turn") return finish(ctx, "succeeded", res.text);
// 3+4. parse and validate — a schema failure is NOT a retry
const call = res.toolCall!;
const check = validate(call, ctx.schemas[call.name]);
if (!check.ok) {
ctx.history.push(toolError({
error: check.error,
expected_schema: ctx.schemas[call.name],
offending_field: check.path,
}));
steps++; // costs a step, deliberately
continue; // next turn, more information
}
// 5. execute — through the idempotency gate, never directly
const out = await executeTool(ctx, call);
// 6. observe — bounded at ingestion, full body in the journal
ctx.history.push(bounded(out, { maxTokens: 2000, keep: "head+tail" }));
steps++;
// no-progress detector
const h = hashCall(call.name, canonicalJson(call.args));
callHashes.push(h);
if (noProgress(callHashes)) return finish(ctx, "stalled");
}
}How does the loop know when to stop?
Seven ways, and only one is the model's opinion. The other six are the harness enforcing a limit. That asymmetry is the most useful thing to understand about agent termination: an agent without budgets has exactly one termination condition, and therefore no upper bound on cost.
Three of the budgets must be independent, because they fail differently. A step budget catches an agent taking many cheap actions. A token budget catches one taking few enormously expensive ones: a single tool returning a 200-kilobyte JSON blob can burn a run's entire allowance in one turn without incrementing the step count. A wall-clock budget catches the agent blocked on a slow external service that will never respond.
Terminate into a named, persisted status rather than throwing an exception. When a run ends as budget_exceeded rather than a stack trace, three good things happen: the user sees a partial result with a stated gap, the operator alerts on the rate rather than the instance, and nobody mistakes a working circuit breaker for a bug. That last one matters more than it sounds. I have watched teams alert on budget terminations as errors, get paged, and disable the budgets to stop the noise.
The subtlest terminator is the no-progress detector, and it exists because of a specific mechanism: each step looks different enough to the model to justify another attempt, so the run never converges. Hash the last K tool calls as tool name plus canonical arguments. If any hash repeats more than twice, or the last four steps oscillate between two hashes, terminate as stalled. This heuristic has false positives, a legitimate polling loop trips it, which is why the tool definition needs an opt-out flag.
| Termination | Who decides | Persisted status | What the user sees | Is it a bug? |
|---|---|---|---|---|
| Model emits a final answer | The model | succeeded | The answer | No |
| Step budget exhausted | The harness | budget_exceeded | Partial result plus a stated gap: what it found, what it did not reach | No — the system working |
| Token budget exhausted | The harness | budget_exceeded | Partial result; usually caused by one oversized tool response | No — but investigate the tool |
| Wall-clock exceeded | The harness | timed_out | Partial result; usually a slow external dependency | No — but check the dependency |
| No progress detected | The harness | stalled | Partial result plus the repeated action, named | Usually yes — a tool or prompt problem |
| Unrecoverable tool error | The harness | failed | An error with the failing tool named | Yes |
| Human cancel | The user | cancelled | Whatever completed before the cancel | No |
export type Budgets = {
maxSteps: number; // default 25
maxTokens: number; // default 400_000
wallClockMs: number; // default 600_000
maxRetries: number; // default 3, charged against maxTokens
};
export function noProgress(hashes: string[], allowRepeat = new Set<string>()): boolean {
const recent = hashes.slice(-6).filter(h => !allowRepeat.has(h));
if (recent.length < 4) return false;
// (a) the same call three times
const counts = new Map<string, number>();
for (const h of recent) counts.set(h, (counts.get(h) ?? 0) + 1);
for (const [, n] of counts) if (n >= 3) return true;
// (b) oscillation: A B A B over the last four steps
const last4 = recent.slice(-4);
if (last4.length === 4 &&
last4[0] === last4[2] &&
last4[1] === last4[3] &&
last4[0] !== last4[1]) return true;
return false;
}What goes in the context on each turn?
Five components: the system prompt, the tool catalogue, the task, the turn history, and any working state or retrieved context. Only one grows, and only one is quietly enormous. The tool catalogue is the number nobody publishes.
The arithmetic: fifty tools at roughly 150 tokens of schema each is 7,500 tokens on every turn. Across a twenty-turn run that is 150,000 tokens of pure catalogue before the agent has done anything useful. On a mid tier at $2 per million input tokens, that is thirty cents of a one-dollar run. It is both a cost line and an accuracy line, because a model choosing among fifty near-neighbour tools picks wrong more often than one choosing among seven.
Anthropic has now published numbers on exactly this. Their advanced tool-use release reports that deferring tool definitions and letting the model discover them on demand cut context consumption from 77,000 tokens to 8,700, an 85% reduction, while tool-selection accuracy on their internal evaluation rose from 79.5% to 88.1% on Claude Opus 4.5, and from 49% to 74% on Claude Opus 4. Their multi-agent guidance separately flags 15 to 20 tools as the point where attention starts to degrade. The fix and the full production pattern are in how to keep 50 MCP tools out of every turn's context.
The other budget killer is tool-output verbosity, and the fix is to truncate at ingestion, not assembly. Bound the response before it enters the history, a hard token cap with head-and-tail retention, never head-only, because the error is usually at the end, and store the full body in the journal with a pointer in the context. Six lines of code that prevent the most common context blowout.
| Component | Typical tokens | Stable across turns? | Cacheable? | Grows? | Lever to reduce it |
|---|---|---|---|---|---|
| System prompt | 800 | Yes | Yes — put it first | No | Trim; move examples to few-shot |
| Tool catalogue (50 tools) | 7,500 | Yes, if you resolve it once per run | Yes | No | Tool subsetting — the single largest lever |
| Few-shot examples | 700 | Yes | Yes | No | Delete the ones that never fire |
| Task / user input | 400 | Yes | Yes | No | Nothing to do |
| Turn history | +1,650 per completed turn | No | Yes, incrementally | Yes, quadratically in cost | Structured notes; bounded tool outputs |
| Tool outputs (unbounded) | 200 to 40,000 each | No | No | Yes, unpredictably | Truncate at ingestion, head+tail, pointer to journal |
| Retrieved context | 0 to 6,000 | Sometimes | Only if stable | No | Better retrieval; a reranker over a bigger k |
Should you compact the context?
Only when a named constraint forces you to. This is where I disagree with the most widely repeated guidance in the field, so let me state the other position fairly first. Anthropic's context-engineering work treats compaction as a first-class lever alongside structured note-taking and sub-agent isolation, and the argument is sound: model recall degrades as token count grows, well before the window is full, so a shorter context can be a more accurate one.
The counter-argument is arithmetic. Compaction discards a cached prefix. On a twenty-turn run where the prefix stays warm, cached input costs roughly a tenth of uncached input: cache reads bill at 0.10x base input on both major providers, and Anthropic's cache writes bill at 1.25x on a five-minute TTL or 2x on an hour. Summarising at turn twelve means paying for a summarisation call, then re-warming a prefix that had cost a tenth of list price, and doing so on every turn afterwards. On the token model here that turns a $0.24 cached run back into something closer to an uncached one.
So my position is narrower than either camp: compaction should answer a named constraint, not be a default. Three constraints genuinely name themselves. You are hitting a hard context window. You have measured recall degradation on your own eval as context grows. Or you have a latency ceiling that a large prompt breaks. Absent one of those three, keeping the full context behind a warm cache is usually cheaper, faster and more accurate than summarising it.
When compaction is correct, threshold-trigger it at around seventy per cent of the window rather than reacting at overflow, and prefer structured note-taking where you can. Writing durable findings to a scratchpad that is itself part of the stable prefix keeps the cache intact while bounding growth. That is a strictly better trade than summarising the conversation and throwing the cache away.
| Strategy | Input tokens on turn 15 | Cost of turn 15 (mid tier) | Accuracy risk | Use when |
|---|---|---|---|---|
| Keep full context, cache cold | 32,500 | $0.0650 | Context rot at high token counts | Never deliberately — this is the accidental default |
| Keep full context, cache warm | 32,500 (30,850 cached) | $0.0095 | Context rot at high token counts | The default. Requires a stable prefix ordering |
| Compact at 70% of window | ~9,000 after summary | $0.0180 + one summarisation call, then a cold prefix on every later turn | Lossy summary; the discarded detail is the detail you needed | A hard window limit, or measured recall degradation |
| Structured notes + trimmed history | ~14,000 (11,000 cached) | $0.0082 | Notes may omit what the model did not think mattered | Long horizons where the cache must survive |
| Sub-agent isolation | Orchestrator stays small; sub-agent contexts are separate | 3-10x tokens for equivalent work, per Anthropic's own testing | Handoff loses context; contradictions resolved silently | Genuinely breadth-parallel work only |
How do you retry an LLM call correctly?
By first deciding which of four error classes you are in, because they need genuinely different strategies and only two involve sending the same bytes again.
The rule that matters most, and the one almost never stated: a retry that does not mutate the context is a way to pay twice. At the temperatures production systems actually use, a model is close to deterministic given identical input. Retrying a malformed tool call against an unchanged context reproduces the malformed call, charges you for a second set of input tokens, and consumes a step. The retry has to carry new information or it is not a retry.
What to append on a validation failure is specific: the error itself, the schema the output should have conformed to, and the exact offending field path. Not the string please try again, which adds tokens and no information. The gap in recovery rate between the two is the difference between a system that self-corrects and one that burns its budget failing the same way six times.
Charge retries against the run's token budget rather than tracking them separately. A retry costs real tokens and real latency, and a system where the retry allowance is independent of the budget can spend three times its nominal ceiling while every counter reads green.
type ErrClass = "transport" | "rate_limit" | "invalid_output" | "tool_business";
export function classify(e: unknown): ErrClass {
if (isHttp(e) && e.status === 429) return "rate_limit";
if (isHttp(e) && e.status >= 500) return "transport";
if (e instanceof SchemaError) return "invalid_output";
return "tool_business";
}
export async function step(ctx: RunCtx, attempt = 0): Promise<StepResult> {
try {
return await callAndValidate(ctx);
} catch (e) {
switch (classify(e)) {
case "transport":
if (attempt >= ctx.budgets.maxRetries) throw e;
await sleep(backoffMs(attempt)); // same context: correct here
return step(ctx, attempt + 1);
case "rate_limit":
if (attempt >= 1) { // fall back fast, not slow
ctx.model = ctx.fallbackModel; // policy table decides this
await journal.note(ctx, "fallback_route", ctx.fallbackModel.id);
}
await sleep(backoffMs(attempt, { min: 1000 }));
return step(ctx, attempt + 1);
case "invalid_output":
// NOT a retry. The next turn, with more information.
ctx.history.push(toolError({
error: (e as SchemaError).message,
expected_schema: (e as SchemaError).schema,
offending_field: (e as SchemaError).path,
}));
return { kind: "continue" };
case "tool_business":
// A 404 is information for the model, not an exception for us.
ctx.history.push(toolResult({ ok: false, detail: String(e) }));
return { kind: "continue" };
}
}
}How do you stop a retry from sending two emails?
An idempotency key derived from the run identifier, the step index, the tool name and a canonical hash of the arguments, persisted before the side effect. Six lines of derivation, one unique index, and it closes the failure mode that costs the most credibility with customers.
Canonical matters. Two JSON objects with keys in different orders must hash identically or the whole scheme is decorative: key-sorted, no insignificant whitespace, stable number formatting. I have seen this exact bug, an idempotency table with millions of rows, none of which ever collided, because a language runtime reordered object keys between two serialisations.
The list of tools that need this is longer than teams assume. Any irreversible write qualifies: sending a message, creating a ticket, charging a card, provisioning a resource, publishing an event, writing to an external system of record. A read is exempt. A write that is genuinely idempotent at the provider, a PUT with a client-supplied identifier, is exempt. Almost nothing else is.
One more piece. The database key does not close the window between the successful external call and the response write. If the worker dies in that gap, the row sits in flight and nobody knows whether the email went out. You close that with the provider's own idempotency header where it is supported, and a reconciliation job that queries the provider by key and resolves stuck rows where it is not. Any explanation of this pattern that stops before the reconciler is incomplete. The same journal and the same guards appear inside a real product in that loop inside a production AI interview platform.
Where does the loop's state live?
In tables you own, with the framework's checkpointer as a convenience on top, not the source of truth. The acceptance test is a single query: what did run 8f3a do at step 7, what did it cost, and how long did it take. If your architecture cannot produce that with one SELECT, an on-call engineer cannot debug it at three in the morning.
This is not an argument against LangGraph checkpoints, Temporal histories or Restate journals. Those are good mechanisms, and the durable-execution pattern, journalled steps that replay deterministically after a crash, is genuinely valuable for long-running work with timers and human approvals. It is an argument about authority. A framework's persistence is optimised for resumption, not for the question an engineer asks during an incident, and the two are not the same schema.
Resumption after a worker crash has two modes; pick one deliberately. Replay re-executes the journalled steps to rebuild state, which requires that every step be deterministic or idempotent. Model calls are neither, so they must be journalled as results rather than replayed as calls. Restart discards partial work and begins again, which is simpler and correct for short runs and unacceptable for a run that already spent four dollars.
One practical detail that saves a week: lease your runs rather than assigning them. A worker takes a lease with a heartbeat; if the heartbeat stops, another worker picks the run up after the lease expires. Without this, a worker that OOMs takes its run with it and the only recovery is a human noticing.
| Option | Survives a worker crash? | Queryable with SQL? | Resumption semantics | Lock-in | Use when |
|---|---|---|---|---|---|
| In-memory only | No | No | None — the run is lost | None | Local development, never production |
| Framework checkpointer | Yes | Usually not usefully | Replay from the last checkpoint | Medium — schema is theirs | You already use the framework and add your own journal alongside |
| Your own run/step tables | Yes | Yes — this is the point | Replay or restart, your choice, per run kind | None | Always. This is the baseline, not the alternative |
| Durable execution engine | Yes | Via their UI, not your SQL | Deterministic replay with timers and signals | High — workflow code is theirs | Long horizons, human approvals, cross-day timers |
$ $ psql -c "select idx, kind, tool_name, cost_micros, ms, error from step_view where run_id='8f3a'"idx | kind | tool_name | cost_micros | ms | error-----+--------+----------------+-------------+--------+-------------------------0 | model | | 21400 | 1840 |1 | tool | search_docs | 0 | 312 |2 | model | | 25100 | 2010 |3 | tool | fetch_page | 0 | 14822 | null4 | model | | 61800 | 3190 |5 | tool | fetch_page | 0 | 290 | {"code":"schema_mismatch"}6 | model | | 64200 | 3260 |7 | tool | fetch_page | 0 | 301 | {"code":"schema_mismatch"}-- step 3 returned 38,412 tokens; steps 5 and 7 are the same call, hashed identical-- no-progress detector fired at step 9 -> status = stalled
What breaks in production?
Ten things, and the five most common are all context problems wearing different hats. Below is the matrix. The row I would tape to a monitor is the last one, where the honest time-to-detect for a cost failure is next month's invoice.
The most under-diagnosed failure is a tool that succeeds and returns an error string inside a 200 response. Your harness sees a success, appends a body that says something went wrong to the history, and the model reasons on it as though it were data. Nothing in your error rate moves. The fix is semantic validation of tool responses, not just JSON parsing.
The second most under-diagnosed is silent history truncation. When the assembled context exceeds the window, a naive harness drops the oldest turns and continues. The agent then re-does work it has already done, because the record of having done it is gone. The MAST taxonomy names this failure mode explicitly as loss of conversation history, and its annotated dataset places it within the largest of their three failure categories.
Every latency number in this section, and in any agent post worth reading, should be p95 and p99. The mean latency of an agent is meaningless because the distribution is bimodal: a cached single-turn resolution and a cold twelve-step run are not the same population, and averaging them produces a number that describes neither.
| Failure | Blast radius | Detection | Time to detect | Auto-recoverable? | Mitigation |
|---|---|---|---|---|---|
| Tool response schema drift after a server update | All runs using that tool | Validation failure rate by tool name | Minutes | No | Pin the schema hash; alert on change |
| A tool returns 40K tokens and blows the window | Single run, large cost | result_size p99 by tool | Seconds | Yes | Truncate at ingestion, head+tail, pointer to journal |
| Loop oscillates between two tools | Single run, large cost | No-progress detector; repeated call hashes | Immediate with the detector | Yes — terminate as stalled | Hash tool name + canonical args over a window |
| Duplicate side effect on resume | Externally visible — customer sees two | Duplicate idempotency_key insert attempts | Minutes to never | Yes | Key persisted before the side effect, plus a reconciler |
| Silent history truncation at the window | Single run, wrong answer | Context-length metric vs window; repeated work in the trace | Only via trajectory eval | No | Explicit compaction with a logged decision, never silent drop |
| Provider 529 mid-run | All traffic | Provider error rate by model | Seconds | Yes | Fallback route after one retry, not four |
| Args parse as JSON but fail semantically | Single run | Semantic validation failure rate | Seconds | Yes — mutated retry | Validate enums and ranges, not just shape |
| Tool succeeds but returns an error string in a 200 | Single run, wrong answer | Nothing, by default | Only via trajectory eval | No | Semantic response validation per tool |
| Step budget set high enough to be no budget | Cost | Distribution of steps per run vs the ceiling | Next month's invoice | No | Set the budget at p99 of successful runs, not at a round number |
| Cost regression from a prompt change | Cost | Cost per request by class, week over week | Next month's invoice without the alert | No | Alert on cost per request, never on total spend |
What does one agent run cost?
Between four cents and three and a half dollars on the model below, depending almost entirely on turn count, and the growth is superlinear because the entire history is re-sent every turn. Doubling a run from twenty turns to forty turns multiplies cost by 3.3, not 2. That single fact invalidates most agent budgets I have been shown.
The model: a 9,000-token static prefix (800 system, 7,500 catalogue for fifty tools, 700 few-shot), a 400-token task, history growing 1,650 tokens per completed turn, and 250 output tokens per turn, priced on a mid tier at $2 per million input and $10 per million output tokens at August 2026 list prices. Every number below follows from those five inputs, so substitute your own and re-derive them.
Two columns in this table are usually missing elsewhere and both change decisions. The catalogue tax is the cost of tool schemas alone, and at twenty turns it is 28.5% of the run. Cost per completed task is total cost divided by your measured success rate, the only unit in which two agent configurations are comparable: a forty-turn run at a 78% success rate can cost more per completed task than a twenty-turn run at 72%, and the raw cost table hides that entirely.
Prefix caching is the largest lever and the cheapest to implement, provided the prefix is genuinely stable. On this model, holding the cache warm takes a twenty-turn run from $1.05 to $0.24, a 77% reduction. The whole method, including the four common mistakes that silently set your hit rate to zero, is in prefix caching and what it does to the cost of a 20-turn run.
| Turns | Cumulative input | Output | Catalogue tax | Uncached | Cached | Per completed task (at 0.72) |
|---|---|---|---|---|---|---|
| 3 | 33,150 | 750 | 22,500 tok / $0.045 | $0.0738 | $0.0433 | $0.1025 / $0.0601 |
| 5 | 63,500 | 1,250 | 37,500 tok / $0.075 | $0.1395 | $0.0620 | $0.1938 / $0.0861 |
| 10 | 168,250 | 2,500 | 75,000 tok / $0.150 | $0.3615 | $0.1144 | $0.5021 / $0.1589 |
| 20 | 501,500 | 5,000 | 150,000 tok / $0.300 | $1.0530 | $0.2440 | $1.4625 / $0.3389 |
| 40 | 1,663,000 | 10,000 | 300,000 tok / $0.600 | $3.4260 | $0.6022 | $4.7583 / $0.8364 |
| Assumptions | 9,000-token static prefix + 400-token task; history +1,650/turn | 250/turn | 50 tools at ~150 tokens each | Mid tier $2/$10 per 1M, checked 24 Aug 2026 | Cache read 0.10x, write 1.25x, warm for the run | 0.72 is an illustrative rate — measure your own |
Do you need a framework for this?
No, and the honest recommendation is to write the loop yourself the first time, so you understand which guards you are choosing not to have. Frameworks are genuinely useful, but every one leaves the same list of things for you to build, and that list is where production correctness lives.
What they do give you is real. LangGraph reached 1.0 general availability in late 2025 and remains the strongest option for graph-structured control flow with built-in checkpointing and human-in-the-loop interrupts. The OpenAI Agents SDK is the simplest thing that works, built around explicit handoffs. The Claude Agent SDK gives you fine-grained lifecycle control. CrewAI has the broadest protocol support, shipping MCP and A2A natively. Google's ADK reached 2.0 in 2026 with first-class support for both MCP and A2A. Versions move monthly, so check the release page rather than trusting any comparison table, including this one.
What none of them gives you: your token and wall-clock budgets terminating into your statuses, your idempotency key derivation and reconciler, your journal schema in your database, your cost attribution with a price snapshot, your no-progress detector tuned to your tools, or your tool-output bounding policy. Every framework assumes you will bring those, and most teams discover the assumption after the incident.
So: use a framework if its control-flow model matches your problem, and write the six things above regardless. If you are choosing between building this in-house and bringing in help that has already made these mistakes, that is exactly what AI product development services are for.
| Control-flow model | Persistence | Protocols | Still yours to build | |
|---|---|---|---|---|
| LangGraph | Explicit graph, edge transitions | Built-in checkpointer + interrupts | MCP via adapters | Budgets, idempotency, cost attribution, journal |
| OpenAI Agents SDK | Handoffs between agents | Session-based, lightweight | Broad model coverage | Budgets, idempotency, cost attribution, journal |
| Claude Agent SDK | Lifecycle hooks around the loop | You supply it | MCP native | Budgets, idempotency, cost attribution, journal |
| CrewAI | Role-based crews and tasks | Built-in memory abstractions | MCP and A2A native | Budgets, idempotency, cost attribution, journal |
| Google ADK | Graph-based workflows | Built-in session services | MCP and A2A first-class | Budgets, idempotency, cost attribution, journal |
| Your own 30-line loop | Exactly your problem shape | Your Postgres tables | Whatever you need | All of it — which is the point of the column |
Agent loops: common questions
→What is an agent loop?
A loop that assembles context, calls a model, parses the tool call the model requests, validates it against a schema, executes it through the harness, appends the result as an observation, and repeats until a termination condition fires. The model does not call the tool; it emits a structured request and the harness decides whether to honour it, which is where validation, budgets, credentials and idempotency all live.
→How do you stop an AI agent from looping forever?
With three independent budgets and a no-progress detector, not with a prompt. A step budget catches many cheap actions, a token budget catches a few enormously expensive ones, and a wall-clock budget catches a blocked external call. Separately, hash the last few tool calls as name plus canonical arguments: if any hash repeats three times, or the last four steps oscillate between two hashes, terminate the run into a persisted stalled status.
→How many steps should an agent be allowed to take?
Set the ceiling at roughly the 99th percentile of steps taken by runs that succeeded, not at a round number. Twenty-five is a reasonable default to start from, because most successful agent tasks resolve well under it and a budget set far above your real distribution is not a budget. Review it monthly against the journal, because the right ceiling moves as your tools and prompts change.
→Should you retry a failed LLM tool call?
It depends on the error class. Transport errors and rate limits should be retried with backoff against the same context. Invalid model output should never be retried identically: at production temperatures the model is close to deterministic given identical input, so the same request reproduces the same failure and charges you twice. Append the validation error, the expected schema and the offending field, then take the next turn. A tool business error should not be retried at all; feed it back as an observation.
→How do you stop an agent from repeating a side effect on retry?
Derive an idempotency key from the run identifier, step index, tool name and a canonical, key-sorted hash of the arguments, and insert the step row carrying that key before performing the side effect. On a unique-constraint violation, read the existing row back and return its stored response. That does not cover a crash between the external call and the response write, which needs the provider's own idempotency header or a reconciliation job that queries the provider by key.
→Do you need a framework to build an agent?
No. LangGraph, the OpenAI Agents SDK, the Claude Agent SDK, CrewAI and Google's ADK all give you real things (graph control flow, handoffs, lifecycle hooks, protocol support, checkpointing), but none gives you your budgets, your idempotency keys, your journal schema, your cost attribution or your no-progress detector. Those are the parts that decide whether the system survives production, and you build them either way.