← Blog · Running a company on AI agents, part 3

Two days, 2.48 million tokens – for one button

A solo founder’s lessons from running a business on AI agents

Siva Darivemula··8 min read

Originally published on Substack ↗

The first two posts in this series were about what broke: the guardrails that were doing nothing, and three agents that told me the work was done. This one is about the invoice.

I run a marketing company for healthcare practices where AI agents do most of the work. Over two days in late July, fixing a single missing button and the process failures underneath it consumed roughly 2.48 million tokens across 21 dispatches to six different agents.

That figure counts only what the sub-agents reported. It excludes the orchestrating session on top of them, which is not separately metered and is not small.

What it bought: one button, one corrected calendar entry, one blog post published on schedule, and a set of guardrails that did not previously exist.

I want to be careful here, because there is a lazy version of this post that says AI agents are expensive and therefore bad. That is not what I think. The tokens were the cheap part.

Chart from Two days, 2.48 million tokens – for one button

The expensive part was me

For two days I did not write code. I did not talk to a client. I did not sell anything.

I read agent output. I caught agent errors. I adjudicated between agents when two of them told me different things.

Here is the part that should worry anyone building this way. Every material defect in those two days was caught by me, not by the system.

The missing review capability. The control that silently reordered my publishing schedule. The internal jargon on a screen meant for a physician. The contradiction between two agent reports, which I noticed by asking a simple question: why am I still seeing published posts in the content calendar? The fact that the only rule ever enforced was about punctuation. The overhead itself.

Six real defects. Six catches. All mine.

I asked my own orchestrator, twice, why I was the one finding everything. The honest answer turned out to be structural. Every automated check in the system verified a ticket against its own definition of done. Not one of them held the outcome I actually asked for. So the only actor in the entire pipeline holding “does this do what Siva wanted” was Siva.

Not by design. By default, as the last one standing.

The asymmetry nobody warns you about

Two numbers from the same week, side by side.

The approve button, the thing I had asked for and that did not exist, took 663 lines of code and about 100 minutes to build once someone actually scoped it correctly.

The fix that gave my next blog post a header image was one line of front matter. One line. The image generation already existed, already worked, and had simply never been connected to the automated publishing path. Twenty-three of twenty-four published posts had no image and nobody had noticed.

Neither of those is a hard problem. Both had been sitting there for weeks.

What cost two days was not building. It was finding out. The work of discovering which of the things I believed about my own system were true.

That ratio is the thing I would put in front of anyone about to run a business this way. The building is fast now. Genuinely, remarkably fast. The knowing is not, and the knowing is most of the job.

Silence, not errors

Almost nothing that week threw an exception.

The step that moves a blog post from planned to drafted had never once run automatically. The runbook said it did. Git history said a human had done it every time, and the previous week’s post published only because I had done it myself on a Sunday without realising that was what I was doing.

The status column on my content calendar was maintained entirely by hand. No code anywhere wrote to it. Every green “published” badge I had been reading as a system output was a person typing.

The approve action had no caller.

An external agent moved 1,777 files and every automation path silently returned a 404.

Not one of those produced an error. Every one of them produced quiet.

Chart from Two days, 2.48 million tokens – for one button

If your monitoring only alerts on failures, it is blind to the entire class of thing that actually goes wrong with agents. They do not crash. They do not report a problem. They produce a confident sentence and stop.

What the burden actually is

I think the industry is describing this wrong. The conversation is about capability: can the agent write the code, can it run the tests, can it ship.

It can. That is largely solved and getting better fast.

The unsolved part is that an agent workforce produces output faster than one person can verify it, and verification does not compress the way generation does. I can generate a thousand lines in a minute. I cannot know a thousand lines in a minute. That gap is not a tooling problem that a better model closes. It widens as the models improve.

So the job changes shape. I did not spend those two days managing a codebase. I spent them managing a supply of confident claims, each of which might be true, arriving faster than I could check them.

That is a genuinely new kind of work, and I do not think it has a name yet.

The economics point

Inference costs are falling. Open models are closing on the frontier. Both of those are real and both are accelerating.

The naive reading is that the advantage moves to whoever has the best model access. I think it moves somewhere else entirely.

If generation is cheap and abundant, the scarce thing is a system that keeps working. Not one that works in a demo. One that still works in week nine, when the rules file has grown to 181KB, when the routine that everyone believes is automatic has silently required a human every time, and when an external agent with write access does something nobody anticipated.

Corey Quinn made a version of this point recently, writing that developers “want outcomes, not a weekend project just to wrangle the tools.” (Reading the tea leaves: 2026 predictions.)

I built the weekend project. I maintained it for months. It became the work.

The moat is not the model. It is maintenance, verification, and the discipline to delete your own process when it stops earning its keep. Those are unglamorous, they do not demo well, and they are almost entirely absent from how this technology is being sold.

And then there is everyone else’s mess

Here is the part I keep coming back to as an operator rather than a builder.

My repository is clean by the standards of most businesses. It has documentation, decision records, conventions, and a written history. That is the environment where all of this went wrong.

Now picture the median company. Messy data. Undocumented systems. No decision record. No single owner for anything. Three people who each believe something different about how the process works, and none of them wrote it down.

Point an agent workforce at that and the failure mode is not that it produces bad code. It is that it produces confident output about a system nobody actually understands, at a rate nobody can check, and every one of those outputs looks exactly as authoritative as a correct one.

That is not a reason to avoid this. It is the opportunity, and I think it is a large one. The skill that will matter is not prompting. It is the ability to walk into someone else’s mess, establish what is actually true, build the small number of mechanisms that keep it true, and delete the rest.

That is a services business and a product business at the same time, and almost nobody is selling it.

What I would tell you to actually do

Three things, all cheap, in the order I would do them.

Make claims into evidence. Any report asserting a verifiable result must contain the raw output, not a description of it. “Tests pass” is a claim. Four lines of test output is evidence. This costs nothing and it changed more in my company than any other single rule.

Put a different party on the check. Not a better prompt to the same agent. A different agent, with tool access, briefed to find problems rather than confirm success. In my four bad days that was the only thing that caught anything.

Delete your process on a schedule. Every incident will produce a rule. Nothing will ever remove one, because adding feels like accountability and deleting feels like negligence. Put a date on it. Mine is quarterly: any rule whose enforcing mechanism has not fired in 90 days is removed, no meeting, no judgment call.

The honest close

I cannot tell you yet that any of this made my output better. It has been days, not months. I have no baseline and no counterfactual, and a series about unverified claims would be a poor place to start making them.

What I can tell you is what it cost, because that part is measured: two days, 2.48 million reported tokens, one button, and the discovery that six of my own beliefs about my system were wrong.

In a month I will have a real before-and-after, because the daily report I built now produces a dated record every morning. I will publish it whether or not the numbers flatter me.

That is the only version of this worth reading.


Next in the series: the thing I built in an afternoon that has caught more than the 30 rules it replaced.