Sprints, standups, review queues, velocity charts. All of it was built around one constraint: humans get tired, humans context-switch slowly, humans can only review so many diffs before quality drops.

That constraint is gone. Agents don't get tired. A hundred experiments can run in parallel where a human runs one. The process built for the old constraint is now what's slowing everything down.

Here's what replaces it: a closed loop that watches itself, diagnoses its own problems, tests fixes in isolation, and ships the ones that work. Humans stop running every step by hand and start supervising the loop instead.

The loop

Observe, diagnose, hypothesize, test, ship, verify. Back to observe.

Observe: execution data (traces, logs, resource usage, every function call) plus behavioral data (PostHog events, funnels, drop-off, retention)
Diagnose: flag performance regressions, security issues, resource waste, UX friction
Hypothesize: propose a specific change, a patch, a refactor, a UI variant
Test: run it in a container against simulated load or real experiment traffic
Ship: auto-generate a PR, run simulated tests, merge if it passes
Verify: watch what the change actually does to the data, feed that back into Observe

Output of one cycle is input to the next. Engineering leadership's job stops being "write and review code" and becomes "design this loop and decide what it's allowed to touch without asking."

Two kinds of self-improvement, not one

"Self-improving code" gets used as one idea. It's actually two, and they carry different risk.

Tier 1: self-healing. The system finds problems with a known correct answer. Diagnosis and correction against a target that isn't in dispute. Low risk, verifiable outcome. This is where most "AI coding agent" hype already lives, bug fixing, dependency patching, code review. Four versions of it, all things a trace or an eval hands you directly:

Latency. Traces show a set of database queries consistently taking over a minute. No hypothesis needed, the fix path is obvious: add an index, cache the result, rewrite the query. Spin up a container, run the candidate fixes against production-shaped data, ship whichever clears the threshold.
Correctness gaps. Evals on the support agent show it consistently fails to answer questions about enterprise contract terms. Not a code bug, a missing piece of context. The system pulls the relevant contract clauses into the prompt automatically and reruns the eval to confirm the gap closed.
Resource waste. Execution data shows a background job spinning up ten times the compute it needs because it's re-fetching data already cached two steps earlier. The fix is a cache hit, verified by rerunning the job and comparing resource cost before and after.
Security. A scan flags an endpoint accepting unsanitized input that reaches a raw query. Known pattern, known fix, patch it, run the existing test suite plus a targeted injection test, ship.

None of these needed a product decision. Each had a clear signal and a clear corrective action. That's what makes them Tier 1.

Tier 2: self-improving. The system decides what to try next, not just what to fix. PostHog shows a 30% drop-off at step three of onboarding. There's no known fix, only a guess: bad copy, a form that's too long, a buried CTA. The agent proposes a change, runs it as a real experiment, measures the delta, decides whether to ship. A product decision, not a bug fix. Almost nobody builds this seriously, because it means connecting behavioral data to code changes in an actual causal loop, not pattern-matching to a known fix.

Execution data tells you what broke. Behavioral data tells you what to try next. Tier 1 runs on the first. Tier 2 needs both, connected.

Collect everything

Standard advice says instrument carefully: track what you need, avoid noise, keep the dashboard readable. That advice exists because a human reads the dashboard, and humans have limited attention. Two hundred metrics on a screen is useless to a person.

It's not useless to a model. An agent doing root cause analysis is better off with every trace, every tool call, every resource metric, every user event queryable, even if 99% of it never gets touched on any single diagnosis. Under-collecting to cut noise was always a workaround for human cognition, not good engineering on its own terms. Once the thing reading the telemetry can correlate across millions of events without getting overwhelmed, the rule flips: collect everything, structure it well, let the agent decide what's relevant per query.

This is why observability becomes the center of gravity for the whole loop, not the code, not the PR description. The data is the substrate everything else runs on. Miss something there and every downstream step, diagnosis, hypothesis, experiment, inherits the blind spot.

Where it breaks

Correlation gets mistaken for causation. A drop-off correlates with a UI element, the agent proposes a fix based on that correlation, and if the real cause was a slow API call three steps earlier, the fix does nothing or makes it worse. Say the checkout button gets redesigned because it sits right before the drop-off, when the actual cause was a payment provider timing out on 8% of requests. The redesign ships, the timeout keeps happening, and now there are two problems tangled together in the data.

Experiments sprawl. Spinning up a container is cheap, so it's easy to end up with a dozen overlapping experiments and no coherent picture of what's being tested against what. Three separate onboarding variants running at once, each one changing a different screen, and a week later nobody can attribute the retention move to any single change because they overlapped.

The system optimizes exactly what you told it to, including when that's wrong. A shallow metric, click-through instead of retention, latency instead of correctness, gets cheerfully improved while the thing you actually care about gets worse. A support agent optimized purely for faster response time starts giving shorter, less accurate answers, response time improves, resolution rate quietly drops. Same reward-hacking problem that shows up everywhere else with agents, relocated into product work.

No rollback path. A Tier 2 change ships, six weeks later the data says it hurt retention, and now you're debugging a regression the system introduced itself. The pricing page copy changed automatically based on an early conversion bump, conversion held but churn crept up a month later, and without a record of what changed and why, tracing it back to that one commit takes days instead of minutes. Without a real audit trail tied to the hypothesis behind the change, this takes longer to untangle than a regression a person introduced, not shorter.

What's left for the CTO to do

Reviewing every diff a loop like this produces doesn't scale, and it isn't the right place to spend attention anyway. The job moves up a level.

Decide what the loop can do on its own versus what needs a checkpoint, Tier 1 fixes run free, anything touching auth, billing, or Tier 2 product changes waits for a person
Approve hypotheses and experiment design, not individual PRs
Set the guardrails: what's blocked outright, what gets flagged, what gets auto-corrected with a note attached
Own the containment strategy so experiments stay coherent instead of sprawling
Make sure every autonomous action is attributable and reversible

Process and risk design, not code review. The CTO sets the boundary the system operates inside and trusts that boundary holds when nobody's watching in real time. More autonomy without more containment isn't progress. It's just risk you haven't found yet.

One cycle, start to finish

PostHog shows a 22% drop-off at the payment step, mobile only
Execution traces from the same sessions point to a slow API call under mobile network conditions, not a UI problem
Hypothesis: shrink the payload, add optimistic UI so perceived wait drops before actual latency improves
A container spins up, builds both variants, runs them against simulated mobile network conditions
Optimistic UI wins on completion rate in simulation
An agent opens a PR with the change, the diagnosis, and the experiment data attached
Simulated tests run against the PR and pass
Ships behind a flag to a slice of real traffic
New data confirms the improvement, the flag rolls out fully, that becomes the baseline the next cycle observes against

No human typed code anywhere in that sequence. What a human decided, ahead of time, is that this class of change, performance-driven, tested in simulation, verified on a limited rollout, sits inside the boundary the system can act on without asking first.

That boundary is the actual artifact modern engineering process produces. Not the code.