Lessons learned from running Claude Code swarms at scale

Some time ago I built a simple app to run swarms of coding agents — I call it fleet (https://news.ycombinator.com/item?id=48256389). It's based on centralized beads with a Python orchestrator and can run any coder (Claude, agy, Codex). Recently I added a UI to manage the whole agent lifecycle: adding new tasks, monitoring running ones, and a chat interface built on MCP with a centralized SQLite DB. From the UI I can spawn agents to run in any directory, define dependencies on other tasks, and specify which coder/model should do the job. Today I can run 10–15 agents concurrently. At that scale you burn through limits very fast, so I spent some time investigating where those limits go and how to maximize efficiency. Here are the lessons learned after a few weeks of running the fleet:

- CLAUDE.md is a terrible abstraction. These files load unconditionally, they often contain descriptions irrelevant to the task at hand, and they stack from your working directory upward. The result is wasted tokens and confusion from injecting irrelevant instructions into the session.

- Skills are bad, but not as bad as CLAUDE.md. They use a progressive disclosure approach: only the skill description goes into the session, and Claude loads the full skill text with a tool when it's needed. That's one level better, but it still doesn't let you scale — you can't create 10K skills, as that would eat your entire usable context. Claude recently introduced a skills budget that silently drops less frequently used skills from the session entirely. You can still invoke them in an interactive session, but the model can't invoke them in a background session.

- Some plugins may be installed more than once. During cleanup I found that a few of mine were installed in multiple locations, consuming double the tokens on duplicated instructions.

- Attaching plugins to every session is a bad idea at scale. You want to be precise about which plugins are actually useful and attach them per task.

- Use a hierarchical knowledge base instead of CLAUDE.md / skills / plugins. It lets you benefit from real progressive disclosure: keep your instructions and tool descriptions in it and let Claude navigate through it quickly and cheaply.

- System tools consume ~15K tokens (7% of the session). You can't manage this — they're just attached, and disabling tools doesn't remove them from the context.

- AskUserQuestion isn't available in background sessions. You need to implement your own tool — MCP- or CLI-based — to give `claude -p` the ability to talk to you.

- You become selective about which model handles each task. Decompose work into harder and simpler subtasks so you can route the simpler ones to weaker, cheaper models and save tokens.

- Your context-switching skill improves over time.

Fleet repo: https://github.com/sermakarevich/fleet