Rebuilding a Principle One Layer Down
An honest return-rate answer from the AI collided with the trading bot's rule number one, and instead of scrapping it I redefined it one layer down.
The trading bot runs on a brokerage Open API, and its skeleton is a hybrid: the rule engine generates signals and the LLM only reviews them as APPROVE/REJECT/REDUCE. The sentence I nailed down as rule number one at kickoff was that the LLM cannot generate orders. Then I asked about expected returns, got an honest number back, and declared that rule dead in my own words.
What I set out to do
One week into the trading bot. The skeleton is a hybrid: a rule engine generates signals and the LLM only reviews them as APPROVE/REJECT/REDUCE. At kickoff I nailed down rule number one, “the LLM cannot generate orders,” and had it proven all the way down to enforcement at the code-path level. Money is on the line.
Honestly, this wasn’t even the first time this week the design got flipped. Two days earlier the execution modes were on the chopping block. The original design had three modes, MOCK/DRYRUN/LIVE, but I pushed back: the dev environment is already paper trading (fake money), so why does it need another fake mode on top? Claude conceded immediately. MOCK wasn’t a “behavior,” it was the dev environment itself, so it ripped out a whole mode, and things simplified to two axes: environment (dev/prod) and a dry-run flag. After watching one rebuttal erase an entire axis, the design stopped being a final draft and became something to keep hammering on.
Then on Saturday, on a whim, I asked about expected returns. Claude Code’s answer was honest. Long term, 4-8% a year.
How I asked
Then this isn’t much different from a savings account.
Safety was eating the purpose. So I restated the vision.
What I want is to build a pipeline where the LLM absorbs information the way an investment professional does.
That meant promoting the reviewer to strategist. In my own words, I was declaring rule number one dead.
Where it broke
Rule number one and “the LLM writes the strategy” collide head-on. Just throw it away and you also release the accident the rule was preventing: a hallucination going straight to an order.
Wrong answers piled on too. While looking into accounts, Claude confidently answered “up to 2 paper accounts, confirmed by multiple sources.” I checked for real: only 1 could be opened. The conversation also went off the rails. Rule track, virtual ledger, LLM track were all tangled together, and I had to tell it to write so I could actually follow.
How I fixed it
Claude’s answer wasn’t to scrap it but to redefine it one layer down. What we were protecting wasn’t the sentence but the safety underneath it. The principle became “nobody can bypass the guardrails,” and I recorded it as an ADR. The guardrails in practice are a validation layer at the chokepoint where orders go out: hard limits like maximum weight per ticker, loss limits, and order quantity/price validation. Whether a strategy proposal came from the rules or from the LLM, if it doesn’t pass this gate there is no order. Only the principle’s position moved, yet the door to promoting the LLM to strategist opened while the path from hallucination straight to order stayed closed.
The structure became a two-track A/B, running both approaches side by side for comparison. In the same session, the rule approach places real orders in the paper account, and the LLM approach, having no account, lives on virtual fills in a DB ledger. Which one is better is for the scorecard to say. I pinned down four terms too: rule approach, Claude approach, paper account, live account. Only after unifying them did the conversation lock back into place.
Lessons
- When a principle collides with the vision, go one layer down before discarding it. Find the invariant instead of the sentence and both survive.
- The more a decision gets reversed, the more it needs an ADR. Three months from now I won’t remember why.
- The AI’s confidence in “confirmed by multiple sources” is worth less than one real measurement. Verify external constraints yourself.
- Unifying terminology isn’t busywork, it’s design work.
- The honest 4-8% triggered all of this. Had it been a pleasant number, I’d have found out much later.