Chapter 1 · The Anatomy of an Operations Agent

Strip the marketing away and an operations agent is a loop. Quality, cost, and safety are all decided by what you put inside that loop.

1.1 The loop, precisely

An operations agent is software that runs one cycle over and over: assemble context, reason, pick an action, run a tool, observe the result, then continue, conclude, or escalate. Each pass is a step. An investigation is a trajectory, a sequence of steps. A remediation is a trajectory whose last steps change something and then verify the change. The Leadership Edition names the outer pipeline: Detect, Analyze, Resolve, Validate. This chapter is about what happens inside each stage.

loop (one step):
  if budget exceeded or loop detected:       # step, token, time budgets; checked first
      escalate with partial findings         # say what was checked and what wasn't
  context   = system_prompt + task + memory + retrieved_context + tool_results_so_far
  decision  = model.reason(context)          # hypothesis, plan, or tool selection
  if decision is tool_call:
      result = execute(tool, args)           # via policy + sandbox (Ch. 5-6)
      append result to trajectory            # feeds the next step
  elif decision is conclusion:
      emit finding | plan | verification     # with evidence attached
  elif decision is escalation:
      hand to a human with evidence so far   # low confidence, or needs approval

Three properties of this loop drive every design decision in the rest of the book:

Property	What it means	Covered in
Stateful across steps, stateless across incidents	The agent forgets the last incident unless you build memory on purpose	Chapter 2
Cost multiplies	Tokens per step × steps per task × tasks per day	Chapter 4
Untrusted input sits next to trusted instructions	Logs, tickets, and alert annotations come from the environment, and any of them can carry an attack	Chapter 5

1.2 What a step really costs

Think in unit economics from day one. A real investigation of a non-trivial incident runs many steps, and token use climbs fast once telemetry excerpts are in context. Independent benchmarks such as ITBench-AA publish turn, token, and cost curves next to accuracy for this reason (Appendix D). Use them to see the shape, then size your own cost from runs in your POC. Three consequences follow:

Context dominates cost. Tool results (log excerpts, metric series, config dumps) are almost always the largest token line, not the model’s own reasoning. Controlling what re-enters the loop is the first optimization: summarize, truncate, or pass a pointer instead of the payload.
Steps are your latency. Each step is a full model round-trip. A 40-step investigation on a slow frontier model can’t respond in real time. Parallel tool calls and delegation to specialists (Chapter 4) cut latency as much as they improve quality.
Budgets are guardrails. Production agents run with explicit step, token, time, and spend ceilings per task. They also need a loop detector: a check that notices the same tool being called with the same arguments to no new effect. An agent without budgets is a cost incident waiting for a quiet weekend.

When you evaluate a platform, ask: what are the per-task budgets, what happens when one is hit, and where can you see tokens and steps per run?

IN PRACTICE: BUDGETS IN CLOUDTHINKEREvery CloudThinker run has a fixed cap on model calls, and each subagent gets its own cap. When a run reaches it, the run stops and says it reached its model-call budget before it could finish. Send a follow-up message to continue from where it stopped. The cap isn’t a setting you change.Spend is capped per workspace with Credit limits: a monthly budget with Throttle or Notify only when it’s reached. To see where credits went, open Admin Settings → Usage and switch between By Feature, By User, and By Workspace (Usage and credits). Usage is shown per feature, user, and workspace, not per run.

1.3 The determinism boundary

The most useful design question inside the loop: which parts must be deterministic code, and which parts benefit from model judgment? Mature systems draw the line on purpose.

Layer	Deterministic (code)	Generative (model)
Sensing and triggering	Alert routing, dedup windows, severity mapping, policy filters	Deciding an anomaly deserves a deep investigation
Investigation	Tool execution, data collection, redaction, timeouts	Forming hypotheses, weighing evidence, choosing the next probe
Remediation	The runbook steps themselves, pre-flight checks, rollback mechanics	Choosing which fix fits this incident; drafting a plan when no runbook exists
Verification	Health checks, SLO probes, before/after state diffs	Judging whether the outcome matches intent; deciding to escalate

The pattern is deterministic rails, generative core. The model chooses; code executes. Every consequential side effect should go through versioned, testable, idempotent code paths, never through a shell string the model composed under pressure. Chapter 3 turns this into tool and runbook design. Chapter 6 turns it into policy.

IN PRACTICE: READ-ONLY VERSUS CHANGE-MAKING COMMANDSWhen CloudThinker uses one of your runbooks during a Resolve investigation, it treats read-only commands (like kubectl get pods) differently from commands that change something (like kubectl rollout restart). Read-only commands run without waiting. A change-making command goes through your Approvals. When it needs a person, it stops at Action requires approval until someone decides, and every attempt lands on the Execution history tab with its policy decision, approval, and outcome. For an uploaded runbook, an AI model reads the fenced code blocks at upload and records the commands that change something. If that step fails, a fixed list of look-only commands (such as kubectl get and kubectl logs) decides, and anything not on the list counts as change-making (Runbooks). Which actions need a person is set by your Approvals rules and mode, after the built-in Always blocked list refuses calls that never run. Ask the same question of any platform: where does reasoning end and execution begin, and who signs off on the crossing?

1.4 Verification is what makes it an agent

Automation executes and hopes. Agents execute and check. The validate stage is what makes graduated autonomy defensible: re-probe the symptom, diff state against intent, watch the SLO for a soak period, and roll back on failure. It’s the first thing to inspect in any demo. Two rules for verification:

Check independent signals. A cleared alert is not proof. The restart that masked the fault can also clear the alert, a failure mode documented in public benchmarks (Chapter 7). Verify against a signal the fix couldn’t have silenced, such as an SLO probe or a before/after state diff.
Give verification a deadline. Set an explicit soak window. When it ends, the system either declares success with evidence or escalates with a rollback already staged.

IN PRACTICE: VERIFYING A RESOLVE FIXEach Resolve investigation that reaches a disposition writes a recollection to incident memory. Its Verify-with: line names one scoped check, its target, and the expected result, for example “5xx rate on tg-checkout over 15 minutes; expected below 1%”. That line is the soak window and the independent signal.When CloudThinker applied a fix and recorded that check, it proposes a follow-up as a card in the conversation. Click Enable to schedule it. The check marks the fix Verified or Failed. A failed fix stays on record as a counter-example, so later investigations don’t repeat it. When the same symptom comes back, CloudThinker runs the old Verify-with: check first, and if it fails, investigates the current fault on its own terms.

KEY TAKEAWAYAn operations agent is a budgeted, observable loop with a deliberate determinism boundary and independent verification. Judge any platform by the loop’s internals (context handling, cost controls, execution rails, verification signals), not by how well its findings read.

Next steps

Chapter 2 · Context Engineering

The biggest lever on agent quality: what the agent is given to reason about.

Approvals

See how CloudThinker decides which actions run on their own and which wait for a person.

Learn

Workshops

Scenarios

Chapter 1 · The Anatomy of an Operations Agent

1.1 The loop, precisely

1.2 What a step really costs

1.3 The determinism boundary

1.4 Verification is what makes it an agent

Next steps

Chapter 2 · Context Engineering

Approvals

​1.1 The loop, precisely

​1.2 What a step really costs

​1.3 The determinism boundary

​1.4 Verification is what makes it an agent

​Next steps

Chapter 2 · Context Engineering

Approvals

1.1 The loop, precisely

1.2 What a step really costs

1.3 The determinism boundary

1.4 Verification is what makes it an agent

Next steps