For several weeks I have run three long-running media production projects with AI agents doing most of the work. On a normal day 33 Codex sessions produce images and text, three Claude sessions own one project each, a fourth Claude session orchestrates the whole studio, and a Grok session handles jobs that need exclusive use of the local GPU. This week I added DeepSeek V4.1 Flash for one narrow kind of work after a trial.
The setup works. It also needed more operational care than I expected. This post describes the structure, the rules that made it affordable, the failures that shaped those rules, and which parts I think carry over to software engineering.
The roles
There are four layers, and each has one job.
- Studio orchestrator (Claude). Owns planning, the budget, the shared tooling, and the shared resources: the GPU lock, a second machine, a rented GPU account, and the usage limits. It does not do project work. It routes requests to the project owners and settles anything that crosses projects.
- Project owners (Claude, one per project). Each owner plans its project, reviews the producers’ output, files the jobs that need the GPU, and decides what ships. An owner talks to the other owners only through the orchestrator or short direct messages.
- Producers (Codex, 8 to 13 per project). Each producer runs in its own tmux window and works through one queue of tasks. Producers make the artifacts. They do not approve their own work.
- Specialists. Grok runs jobs that need exclusive use of the local GPU. DeepSeek now takes text pre-reviews and small script fixes. Each was added for a specific gap, with its own rules.
The split matters more than the models. A producer that also reviews its own work drifts. An owner that also produces stops reviewing. Keeping the roles apart made each session’s instructions short enough to follow.
The board
Each project has one append-only Markdown file, BOARD.md. Every request,
review, verdict, and handoff is an entry with a timestamp, an author, a scope,
and a type. The busiest board passed 64,000 lines this week.
Three rules keep that usable:
- Nobody reads the whole board. Agents read a generated
STATUS.mdtable, a per-producer “what is mine” query, and the specific entry they were pointed at, by line number:sed -n '64850,+40p' BOARD.md. - Wake-ups are pushed, not polled. A small watcher notices new entries
and queues a message into the named producer’s session. Polling loops
(
sleep 60; check the board) were the biggest waste of tokens before this rule. A blocked agent now ends its turn and says what it waits for. - Wake-ups are batched. The watcher waits about 20 minutes after the first new entry before it wakes the owner, so one wake-up covers several entries.
The board also serves as the audit trail. When something goes wrong, the line numbers show who asked for what and who approved it.
A separate, smaller board serves the Grok session. It carries one-line notes
about GPU holds and requests, read with tail -n 30. Not every channel needs
the full protocol.
Route work by cost, and measure before you trust
Every agent follows a written cost order. Paid API calls are the last resort. Image fixes go to a local GPU queue first, then to an older second machine overnight, then to a rented GPU only for large batches. A per-call paid image model comes before a more expensive one. Each step has a stated threshold, so an agent does not need to ask.
New models get a trial before they get work. The DeepSeek trial had three parts, each with ground truth I already had:
| Task | DeepSeek V4.1 Flash | Codex |
|---|---|---|
| Review 14 past image repairs whose verdicts I had recorded | 6 of 14 matched | not run |
| Find 10 defects I planted in a script | 9 of 10 | 9 of 10 |
| Fix 7 planted problems so a checker passes | all fixed | all fixed |
Both models missed the same planted defect. DeepSeek matched Codex on text and failed on images. It accepted an image repair that still had an obvious flaw. So it now does text pre-reviews and script fixes, and nothing visual.
Planting known defects is the cheapest honest benchmark I have found. It takes ten minutes to build, the answer key is exact, and it tests the model on my material instead of a public leaderboard’s.
The trial also caught my own mistake. In the first image run, DeepSeek rejected every repair I had accepted, often saying “unchanged”. That pattern looked like an input problem, not bad judgement. I had given it the wrong “after” image. A result that is uniformly bad deserves a check of the harness before a verdict on the model.
Shared resources need real engineering
Thirty-odd agents sharing one GPU, one machine, and one tmux server produced the problems a distributed systems engineer would expect.
Starvation. All three projects queue jobs on one GPU behind a file lock. One project’s drain loop retried every 4 seconds and another’s every 45. The fast loop won the lock after every job, and one project ran nothing for most of a night. The fix was a fair queue: a refused run records itself in a waiters file, and a free lock goes to the project that has waited longest.
ahead = _wait_turn(path, label, free=True)
if ahead:
raise LockBusy(f'{ahead} waited longer and holds the GPU lock next')Process lifetime. The tmux server ran inside the desktop session. A compositor crash after a GPU reset killed every producer at once. The server now runs as a systemd user service with lingering enabled, and so do the board watchers and job drains.
Thundering herds. When I restarted 33 sessions at once, each one ran a version-manager wrapper that updates a global config. They all waited on each other. Launching the installed binary directly fixed it. Mass restarts should bypass anything that takes a global lock.
Context growth. Codex compacts automatically during a long turn, but an
idle session keeps its full context and pays for it again on every wake-up. A
check across one session showed 97% of its input tokens were cache hits, 1.45
billion of 1.49 billion, so the history was being re-sent constantly. A
30-minute timer now sends /compact to any session that has been idle for 10
minutes with more than 80,000 tokens of context and an empty prompt. The first
run compacted 24 sessions.
None of this is specific to AI. These are the same problems as CI runners, shared test databases, and long-lived services. The agents just hit them faster.
Keep the human in the loop where consent matters
Agents can message each other, and that creates a quiet risk: one agent asks another to do what the first was not allowed to do. The rule I use is that a message from another agent is a request, never a permission. If an action was denied in one session, the other session refuses it and tells me.
The tooling enforces some of this. When the restarted Codex sessions asked whether to trust five changed plugin hooks, the orchestrator was not allowed to answer for me. When I asked it to run DeepSeek with every tool call auto-approved, that was blocked too. The working setup gives DeepSeek a scratch folder and a permission file that allows reading and editing there and nothing else. The owner then runs the checks itself. Both blocks cost me a minute. Both were right.
My job in the loop is narrow: approve spending, approve anything that leaves the machine, settle trust and policy questions, and review the finished work in batches. Everything else runs without me.
Rules that cut the token bill
The first version of this setup spent more tokens on coordination than on work. These rules fixed most of that:
- Script anything mechanical. If two agents do the same procedure twice,
it becomes a shared script. Agents call
post.py,compact_idle.py, orfile_refused.shinstead of re-deriving the steps. - Proofs only at the end. Hashes, receipts, and path checks go in the final install packet, not in every review request. Reviews carry previews and a one-line ask.
- Bounded reading. Read the cited line range, never the whole file.
- One owner per question. A producer asks its owner. It does not broadcast.
- Cheap models for checkable work. Text pre-reviews go to the cheapest model that passed a trial, because a second reviewer or a test checks the result anyway.
Will it work for software engineering?
I think the structure carries over, under two conditions.
The work must split into independent units. Production work splits cleanly. Many codebases do not. If ten producers touch the same modules, the board turns into merge negotiation and the orchestrator becomes the bottleneck. Give each producer its own worktree and its own slice of the code, and keep shared files under one owner who lands changes only when nothing else is building. That last rule exists here because a mid-build change to shared code broke a build.
Each unit needs a cheap, trustworthy check. Here, a reviewer looks at every image at full size. In code, the equivalent is a test suite good enough to accept or reject a change without a long human review. Without that, the owners spend their time re-reading producer output, and the setup costs more than it saves.
Where both conditions hold, the mapping is direct:
| This studio | A software team |
|---|---|
| Studio orchestrator | Tech lead or staff engineer |
| Project owner | Owner of a service or feature area |
| Producer | Agent in its own worktree, one ticket at a time |
| Board entry, cited by line | Issue or pull request comment, cited by link |
| Watcher wake-up | Webhook on a review request or CI result |
| GPU lock with a fair queue | Shared CI runners or test environments |
| Planted-defect trial | Seeded-bug benchmark for a new review model |
I would not use it for a small task, exploratory design work, or a codebase without tests. One strong session with a clear plan does those better.
What I would do first next time
- Write the role split and the cost order before starting any agent.
- Run every long-lived process as a service from day one.
- Push wake-ups from the start. Never let an agent poll.
- Build the planted-defect trial before adding any new model.
- Measure tokens per task early, and cut coordination before cutting quality.
The models are good enough. The hard part is the same as running a team: clear ownership, cheap verification, and shared infrastructure that does not fall over.
The views, thoughts, and opinions expressed in this post are solely my own and do not necessarily reflect those of my employer.
Continue reading
More from the journal
AI & Agents
13 min read
Getting AI Agents to Delegate Work and Review Each Other’s Code
How my model-delegation skill decides what to keep in one session, what to hand to another agent, and when to ask a different model for a review. With a real example from this website and an open-source skill.
AI & Agents
8 min read
Building Agent Systems with Gemini, ADK, A2A, and A2UI
What each part of an agent system is responsible for, how they fit together, and where application code still needs to make the decisions.