Back to Learnings

Learnings / Agent failure loops

When an agent does not stop.

One background session made 320 model calls without another human prompt. What we could reconstruct, what we could not prove, and why observing the work was not enough.

The reason

AI models are hard to control. Agents need circuit breakers.

A model can keep proposing actions while the software around it keeps executing them. Each request may look reasonable, yet the session can consume hours of work and shared capacity without finishing the task. Instructions alone are not an enforced limit.

A circuit breaker gives the surrounding system a way to intervene: pause the run when a configured boundary is crossed, record the reason and let an operator decide what happens next. It cannot guarantee good decisions, but it can limit how long a failing workflow continues.

01 / The story

Four requested features, nearly four hours of background work

The incident account published on NautGate on 24 August 2026 began with a request to implement four features on a small website. A background coding session continued alongside the visible conversation. After the initial instruction, none of its 320 model calls had another human prompt.

Across nearly four hours, that session consumed 46.55 million fresh input tokens. Of its 320 calls, 244 produced another tool call. Its recorded history grew from 19 messages to 662, and fresh input eventually reached 273,886 tokens on a single call.

This was not evidence of an agent becoming malicious. It was an automated process continuing beyond the resource boundaries we expected. The size of the website did not put a useful limit on the amount of work the session could generate.

02 / The problem

A successful request can still be part of a failing workflow

Checking for API errors alone would miss this kind of incident. A model can return a valid response, request another tool action and continue doing so. The individual step succeeds while the overall job fails to finish.

As the conversation and tool history expanded, more context went into subsequent requests. We observed no corresponding cache reads. Previously seen context therefore appeared repeatedly as fresh input, making each continuation more expensive in input capacity.

Those records show a traffic pattern, not the hidden reasoning inside the agent runtime. They do not prove why the background session continued or establish a single cause for the absent cache reuse. The distinction matters when deciding what to fix.

The damage was shared capacity. The published account records a five-hour limit event, a contribution to a second limit event and a large reduction in the available weekly allowance. A subscription request does not need a metered API charge to consume a resource other agents and their operator depend on.

03 / What we learned

Record the sequence, then give the system a stopping rule

Attribute consumption to the session

The requests passed through NautGate, which gave us the source application, session, timing and token records needed to reconstruct the run. We could examine context growth, cache behaviour and concurrent work instead of guessing from a limit notification.

That visibility only covers requests routed through the gateway. A tool that bypasses it also bypasses that evidence and any enforcement at that point.

Separate observation from intervention

At the time of the incident, the gateway could show what had happened but lacked a capacity-aware circuit breaker for subscription traffic. A dashboard cannot stop a session simply by displaying its consumption.

A guard needs an identity to act on, a policy to evaluate and a place where it can refuse the next request. The intervention should target the offending session where possible, rather than interrupting every agent using the same account.

Request arrives with session identity
  → Read usage and behaviour for that session
  → Evaluate the configured policy
      → Allow and record
      → Warn and record
      → Pause, record the reason and notify the operator

This is a conceptual decision path, not executable NautGate configuration. Useful policy inputs include fresh input in a rolling window, cache reads and writes, request counts and sustained tool activity. A long run can be legitimate, so its expected workload and limits need to be explicit.

A pause must survive the fallback logic

Moving a request to another provider does not necessarily solve the problem. It can continue the same runaway session through a different account or turn exhausted subscription capacity into metered spending.

The NautGate article describes Max Guard in v0.5.0, with observe, warn and pause policies. It tracks fresh input, cache reads and writes, output and requests, including rolling five-hour and weekly pressure. A paused Max request receives a non-retryable response and does not silently switch to metered Anthropic credits.

That is the gateway's boundary. The calling application must also handle the refusal correctly. A client that starts a new session or chooses another unguarded route can defeat the intent of a pause.

04 / Putting it into practice

Test the stopping behaviour before leaving an agent unattended

Use the current NautGate documentation for installation and configuration. The v0.5.0 account explains the response to this incident; it is not a substitute for the settings supported by the version you run.

  1. Identify the application and session on every request. Confirm that the intended traffic actually crosses the gateway.
  2. Observe a representative job. Record input growth, cache reuse and request counts before choosing limits.
  3. Exercise a pause with a bounded test. Check that the client stops, the operator receives an explanation and unrelated work remains available.
  4. Test the fallback path too. A blocked request should not reappear as an automatic request to a paid or unguarded provider.
  5. Define how work resumes. Inspect the session state and remaining task before approving more capacity; restarting the same loop is not recovery.

The lesson is not that every long-running agent is broken. It is that a continuing stream of valid requests is not proof of useful progress. We need evidence of what happened and a tested way to stop what should not continue.

Know when to stop the run.

The incident behind this article and the documentation for testing a stopping rule.