Skip to main content
The pilot ends and the fleet stays. Day 2 is version management, model migrations, memory upkeep, and postmortems for a new kind of teammate, and this chapter gives you a working routine for each.

​
9.1 Everything is an artifact, and every change is a deploy

The unit you operate is a versioned bundle: Ship the bundle like a service release. Review it in a pull request, run it through the Chapter 7 eval gates, roll it out progressively, and roll it back as one unit. Hunt down invisible change: a prompt edited in a console, a model updated on the vendor’s side, a tool description tweaked in place. If a change can’t be diffed, gated, and reverted, it will eventually be the root cause you can’t find, in a system whose whole job is finding root causes.

​
9.2 The model migration playbook

Models change on someone else’s schedule: deprecations, upgrades, price moves. A model swap is the highest-variance change in the stack, so run it as a project:
  1. Pin versions explicitly. Never point production at “latest”.
  2. Run the full golden set on the candidate and diff results per incident class, not just the average.
  3. Canary on low-risk classes and watch online metrics: acceptance, rollback, step counts, and cost. A shift in step counts is the classic tell of changed behavior.
  4. Keep dual-run capability so sensitive classes can run old and new side by side and compare.
  5. Agree deprecation windows with vendors in advance, so a migration is planned work, not an emergency.
Some fleets mix models: small tuned models on routine classes, frontier models on novel investigations. Apply the same playbook per lane. You also gain resilience, because a price shock or regression in one model becomes a problem for one lane, not the whole fleet.

​
9.3 Memory and knowledge upkeep

Chapter 2 designed memory. On day 2 you maintain it. A monthly cycle that works:
  1. Review new memories written since the last cycle. Sample them, starting with high-impact incident classes.
  2. Settle contradictions where fresh evidence disputed a stored conclusion.
  3. Prune or expire memories that reference decommissioned systems.
  4. Re-index the knowledge layer after major changes to your estate.
Give the knowledge layer a named owner. It’s a production dependency, not a wiki with good intentions. And close the loop: every resolved incident’s postmortem should end with one explicit line, “What did we teach the system?“

​
9.4 Postmortems for agents

Agents will be wrong. The reflex you want is the one SRE culture already has: blameless postmortems that focus on mechanism, now applied to software teammates. When an agent makes a consequential mistake, rebuild the event from the trace and the audit record with four questions: Then name the mechanism, because each one has a different fix: “The AI made a mistake” isn’t a mechanism. Add every event to the golden set as a new scenario, so the same failure can’t ship twice. Hold the same line in public for people and agents: errors get engineered away with the same discipline, and neither gets a witch hunt. Nothing kills an agentic program faster than one bad incident treated as a betrayal instead of a defect.

​
9.5 The role, made concrete

The Field Guide names the agent operations engineer (§7): the person who runs the system that runs your estate. Here is a typical week:
IN PRACTICE: DAY 2 IN CLOUDTHINKERCloudThinker runs as a fleet by design: one built-in assistant, CloudThinker, spawns temporary subagents for specialist work, and they share the same workspace skills and memory. A skill fix therefore reaches every agent at once. Here is where each Day-2 habit lives:

​
Your first Day-2 hour

1

Put your custom skills in Git

Open workspace Settings, then Skills. Click Download on each custom skill and commit the .zip files, or their unzipped SKILL.md files, to a repository with a named owner.Success state: every custom skill in the workspace has a copy in Git you can diff.
2

Check what Dreaming changed

Open More → Workspace settings → Memory. In Run history, expand the latest Succeeded run and read its summary.Success state: you know which memory items and skills the last run wrote, merged, or deleted, and you’ve turned off skill deletion if any deleted skill mattered.
3

Review last week's scheduled work

Open Automations, then the Runs tab. Open any failed run, or any run that changed something, and read its conversation.Success state: each failed or write-making run from the last week has an owner and a next step.
KEY TAKEAWAYDay 2 is a discipline. Ship versioned bundles through eval gates, run model migrations as projects with per-class diffs, maintain memory on a schedule with a named owner, and run blameless postmortems that name the mechanism and grow the golden set. Give one engineer a week that makes all of it routine. Fleets that do this improve over time; fleets that don’t decay into the toil they replaced.

​
Next steps

Chapter 10 · The Vertical Pillars

Apply the same engineering to complexity, cost, and security.

Dreaming

Review nightly memory upkeep and its run history.