#llm
18 posts tagged llm. · All tags
- One Empty Array Turned Hitting the Limit Into an Ordinary Failure For two hours before dawn, jobs failed across every project. The only message left was one line with the cause slot empty. And fixing it meant a number in a disclosure body could trigger a global pause
- The Prompt Did Not Do It, Splitting the Stages Did I tried to fix thin summaries with the prompt and made them worse. Separating number extraction from sentence writing raised recall 18 points and presence 2.6 points together
- I Turned On Auto-Publishing and Four Things Surfaced in Three Days None of them were visible while a human sent each issue by hand. A rerun pinned an old response to a new list, a slug got rebuilt and broke the links, a shell that only filled the schema reached the page, and one source took nine of twenty-five slots
- I Cut the Input to a Third, and What I Thought I Had Fixed Split in Two Two-stage processing fixed both numeric accuracy and omission. Measuring again on short inputs showed one was a length problem and the other was a structure problem
- I Measured Four Models on 200 Documents, and What I Did Not Measure Overturned the Result On numeric fidelity the challenger won. Then I found that Korean number spacing was broken in all 200 outputs
- I Stopped Judging Runaways Per Job I built a per-call cost cap, raised it, turned it into an alert, and finally removed it. After fixing the same safeguard four times in four days, all that's left is the aggregate gate and a timeout. "Didn't I say just alert me?"
- I Opened Up Effort and Effort Was Empty I added per-job model and effort selection to the shared queue. When I had it do a self-review, the agent measured with 9 scratch sessions and found that the thinking I had turned off five days earlier was neutralizing effort. Why I split the requested value and the actual value into two columns, and how after deploy the docs and production disagreed in five places
- I Built the Non-LLM Preprocessing First: 150 Items Down to 39 The collector for the newly built AI-news digest dumped 248 items from three days. Rule-based preprocessing that started from "I can't use the LLM without limit," rejecting embeddings, and the one regex character that was throwing away every Korean title as a duplicate
- I Turned Thinking Off 85.8% of the summary job's output tokens were thinking. One environment variable turned it off: output −85%, 4.5x faster, schema 10/10, and the summaries actually had more numbers. And why the one line that turns it off could have killed auth for three projects at once
- When the Constraint Changed From Money to Quota, I Needed a Queue I shut down self-hosted GPU serving and switched to the subscription CLI, and the shared layer I had nailed down in April as "don't build it," this time I built.
- I Fed It Structured Data Instead of Raw Text Only 15% of periodic-report text was being read; it ate 55% of LLM input and the result was cover-page restatements. Swapping raw text for six confirmed figures from the disclosure API cut input to 1/40 and put numbers in the summaries
- One Division Eliminated Half the Model Candidates Measured DGX Spark memory bandwidth at 231GB/s. Divide by active parameters and you get the decode ceiling; every dense model was eliminated by arithmetic alone
- The Local LLM That Read "6 Shares" as 0.06 Moving the trading bot's daily LLM roles to a self-hosted model, and pinning down the misreading bug with a byte-exact DB reproduction and A/B measurements
- I Got Stopped From Building One More Thing in Front of the Server I cut off the creation of a general-purpose endpoint. And while I was at it, I decided to shut down a whole project.
- Shrinking the Prompt Made It 55% Faster, and Lost What Actually Mattered The compression that produced −55% was throwing away the core of the domain. A story about a metric I wasn't measuring collapsing.
- Five Days of Fine-Tuning Lost to a 108-Character Prompt I spent five days on LoRA for Gemma 4 26B-A4B and it barely differed from no tuning at all. What happened when I attacked the same task with prompt compression instead.
- Two AIs Blamed Each Other: What Happens When Every Project Gets Its Own Agent Each project's agent looked only at its own logs and pleaded innocent, so I laid both logs side by side to reach a conclusion
- I Was Running a Model per Project, So I Merged Them into One LLM Server Standing up a vLLM server on a single DGX Spark, splitting it out instead of running a model per project, and wiring up monitoring on day one