I Opened Up Effort and Effort Was Empty
I added per-job model and effort selection to the shared queue. When I had it do a self-review, the agent measured with 9 scratch sessions and found that the thinking I had turned off five days earlier was neutralizing effort. Why I split the requested value and the actual value into two columns, and how after deploy the docs and production disagreed in five places
The shared queue was running LLM jobs for several services fixed at Haiku and effort low. A news project I was building new was about to use the same queue and needed Sonnet for its summaries, so I opened up per-job model and effort selection. The feature took an hour, and then the self-review pointed out that effort had not been doing anything in the first place.
What I set out to do
It was the night I was moving the disclosure-summary service’s same-day summaries to the shared queue. The news project I was building new was about to use the same queue, and that side needed Sonnet for summaries. The queue was fixed at Haiku and effort low.
Can you make the model and effort selectable? When nothing is selected, the default should be what’s in use now.
How I asked
The feature came out within an hour. Model and effort fields on the job; the server default if absent. Model validation was by shape, not a whitelist. One of four aliases or a claude- prefix passes. effort is a fixed enum of five.
I had it split into two commits, merge one, and self-review the other.
Merge number 1 and self-review number 2.
Where it broke
The self-review left five inline comments, and one of them broke the feature’s premise. The agent ran nine scratch sessions and measured the same prompt four times per effort level.
| Condition | thinking tokens | Output tokens |
|---|---|---|
| low | 0 | 5–71 |
| max | 0 | 35–65 |
| max, control (no env var) | 179 | 186 |
Even with effort raised to max, thinking was 0. The decision five days earlier in episode 37 to unconditionally inject MAX_THINKING_TOKENS=0 was neutralizing effort. low answers instantly with 5 tokens and gets it wrong; max writes out its working. That’s visible working, not hidden reasoning. The sentence in the docs, “effort suppresses thinking tokens,” had the wrong subject.
The second was design. At first one column served as both the requested value and the actual value. The problem came up in re-queuing. You can’t distinguish the value the producer chose from last time’s default. If the server default changes from Haiku to Sonnet, jobs with nothing specified should go to Sonnet and jobs that specified Opus should stay on Opus. I split the column into requested_model and model and verified with re-queuing on local Postgres.
How I fixed it
The review pointed out one more thing. Once this feature opens up, the queue’s three caps lose their premise. The 1.0-dollar cost cap and the 5,000-output-token alert threshold are all values chosen from the Haiku and low distribution. Give Sonnet max and normal jobs exceed them. That day I left it as only a note, and two days later in episode 41 I went as far as removing all the caps.
The last twist came while verifying the deploy. When the agent read the production environment variables and checked them against the docs, five places differed. Concurrency 3 vs 6, timeout 180 seconds vs 600 seconds, stale threshold 24 hours vs 15 minutes, 5-hour window 0 vs 10 million, weekly 0 vs 100 million. The docs even carried a warning, “disabled at 0 until measured,” but the gate had long been on. Three were values I had raised myself three days earlier. “Increase it 10x,” “make it 10 minutes,” “the weekly limit to 100 million too.” At that moment the review’s “latent” finding became “real.” I added a production-actual-value column to the table.
The same night it also came out that this repo had no PR CI, and a test with a real Postgres attached came into being. Within an hour of attaching it, it caught one build broken by a signature change. After deploy, 2 of the disclosure-summary service’s jobs completed. It was the first moment that service’s jobs finished on this queue.
Lessons
- Measure whether an option you “opened up” actually works. effort max was thinking 0. A setting that neutralized the premise had gone in five days earlier.
- Split the requested value and the actual value into separate columns. Double one up and re-queuing lies the moment the default changes.
- Open up an option and the caps chosen from that option’s distribution lose their premise. Look at the list of caps before opening it.
- Keep a production-actual-value column in the docs’ environment variable table. Three values I had raised were not in the docs, and the docs were giving the opposite warning.
- Have the self-review measure. If the review had only read the code, effort would have stayed empty.