The short version: My overnight job runs a report-only agent over the homelabās records and ships a digest. On September 28 the sessions went from about 202 seconds to about 36, output fell from 38,000 tokens to 8,700, and the agent stopped trying to list directories it had no business in. Nothing in the repository had changed. The sonnet alias my dispatcher passes to the model had started resolving to a newer release.
Get the week's seven in one email, every Monday.
What went wrong
Nothing broke, which is the point. The dispatcherās default is --model sonnet, an alias that Anthropic moves to the current Sonnet release. From September 5 to September 27 the alias meant Sonnet 5. From the 28th it meant Sonnet 5.5, and the jobās character changed overnight without a commit, a deploy, or a line in any changelog I keep.
The old model spent its nights differently. Over twelve nights it made 49 attempts to run ls or find on projects outside its working directory, and the sandbox refused every one. Sessions ran 87 to 322 seconds. The new model made zero such attempts and finished in 26 to 46 seconds, with a quarter of the output.
Had the change gone the other way, I would have woken up to a slower, nosier agent with nothing to blame. My logs did not record which model actually ran each night. The transcripts did, because every message carries a model field, and I had never looked at it.
What we learned
- An alias is a deploy you didnāt schedule. It changes behaviour, cost and speed on someone elseās calendar.
- The sandbox did the work the prompt couldnāt. The prompt told the old model to stay in its lane, and 49 refusals say it didnāt listen. Permissions enforced outside the model kept it there anyway.
- Behaviour matters more than speed. The five-times speedup was pleasant. The drop from 49 out-of-lane attempts to zero is the change I would have wanted a release note for.
What went right
The evidence was there when I went looking: transcript timestamps for session length, per-message token counts, and the model name on every message. One afternoon of counting turned a vague sense that the nights felt faster into a dated chart with a visible step on September 28.
How to do it (as of October 2026)
- Pin the model in unattended jobs, or accept the alias on purpose and log the resolved model with every run. Either is fine, and not knowing is not.
- Keep refused tool calls as a metric. A change in that count is a change in the modelās behaviour, whatever the release notes say.
- When an unattended job changes character overnight, check the vendorās calendar before your own commits.
- Enforce boundaries outside the model. The prompt is a request, and the sandbox is a rule.
Status
Still true. The overnight job still names the alias, and nothing in its ledger records which model answered.