Back to blog

2026.02.21

Building an Operations Dashboard for a Team of AI Agents

Four AI agents could collaborate on code, but operating the team was still a manual job. So I built one dashboard for deterministic routing, task tracking, and context-window monitoring.

aiagentengineeringdashboarddevops

The previous post covered how I assembled a team of AI agents. Once the team could actually ship work, a new problem replaced the old one: operations hell.

Four agents were working at the same time in four Discord channels. Tasks moved through more than a dozen states. When CI finished, someone had to alert QA. When QA rejected a change, someone had to send it back to development. I had originally handled all of that by letting the agents message one another. Every message woke another session, every wake-up triggered an LLM inference, and every inference created another chance to be slow, expensive, or confidently wrong.

So I spent a day building an operations dashboard.

The core problems

1. Inter-agent communication was too expensive

Suppose Agent A needs to tell Agent B that a build passed. The path looked like this: Agent A posts in Discord, Agent B's session wakes up, Agent B runs a complete LLM inference over its context, and Agent B produces a prose response. Only then has the system accomplished the equivalent of "acknowledged."

A sentence as small as "your CI passed" could burn tens of thousands of tokens. Worse, agents sometimes embellished the reply: a confirmation became a summary, a suggestion, or a new plan nobody had requested. A deterministic handoff gained cost, latency, and uncertainty.

2. There was no global view

Task state lived in two places that were individually useful and collectively miserable: JSON files held machine-readable status, while Discord held the conversation around each change. To answer a basic operational question such as "which agent is doing what right now?" I had to open the channels one by one, reconstruct the latest handoffs, and mentally join them with the task records.

3. Context windows grew invisibly

Sessions managed by OpenClaw keep accumulating context. A 200k context window sounds enormous until a long-running coding channel quietly reaches 80% capacity. By that point the agent has to work through a huge history on every turn, and response quality starts to decline. The failure mode is gradual: nothing crashes, no alert fires, and the only symptom is a vague feeling that the agent has recently become less precise.

The solution

Non-LLM task routing

The largest improvement was also the least intelligent component in the stack. I moved every cross-agent notification into a purely mechanical Discord bot. It performs zero AI inference. It is a state machine connected to a REST API.

task state change → Router detects → notify via Discord REST API

Its job is deliberately narrow:

  • Automatic dispatch: when a task enters new, ping the PM channel; when it reaches approved, ping the assigned development channel.
  • CI polling: check GitHub Actions every 60 seconds. Route a successful run to QA, and send a failed run back to development.
  • Merge detection: after QA approves a pull request and that PR is merged, close the task automatically and post the final notification.

These transitions have exact inputs and exact destinations. They need no interpretation or personality. The router sees a state change, looks up the channel, and calls Discord's REST API.

The whole router is a Node.js module of roughly a hundred lines. It uses a Discord bot token to call the REST API and replaces a large volume of LLM-to-LLM communication. The agents reason about code, requirements, and reviews; the router handles the clerical work between them.

The Kanban dashboard

The dashboard uses React and Vite on the frontend, Express and SQLite on the backend, and Docker Compose for one-command deployment.

A five-column Kanban board compresses more than a dozen workflow states into groups a human can scan quickly:

Column Included states
Backlog new, pending, approved
In Progress dispatched, in_progress
Review pr_submitted, ci_passed, ci_failed
QA qa_pass, qa_fail
Done done

The detailed states still drive routing, while the five groups let me see where work has accumulated without reading every record.

Tasks are sorted by the time of their last update, newest first. Opening one shows the task details, linked proposal, and QA report in a split layout with Markdown rendering and syntax highlighting. The state and the artifacts that justify it stay visible together.

Context-window monitoring

This became the feature I use most. Every agent card has a progress bar that turns context consumption into an immediate visual signal:

  • Green below 60%: healthy
  • Orange from 60% to 80%: filling up
  • Red above 80%: reset time

The data path is intentionally simple. A cron job on the host runs every 2 minutes and parses the output of openclaw sessions list into JSON. The Docker container mounts and reads that file. The frontend polls the backend every 10 seconds.

As soon as an agent turns red, I click "Reset Sessions." The dashboard sends /new to every channel automatically. A resource that used to fail silently now has a threshold, a color, and a one-click recovery path.

Strict mode for QA

The QA agent operates under hard rules:

  • Any P1 or P2 issue means the review must fail.
  • Missing unit tests means the review must fail.
  • Every review must produce a QA report as a Markdown file.

The important part is what happens after a rejection. When development fixes the issues and submits the pull request again, the router waits for CI to pass, then sends QA a RE-VERIFICATION REQUIRED message. That message includes the path to the previous QA report, so the agent knows exactly which findings need to be checked again.

Technology choices

Layer Technology
Frontend React + TypeScript + Vite + Tailwind
Backend Express + SQLite
Routing Discord REST API (fully mechanical)
Containers Docker Compose
Theme GitHub Dark (#06090f / #0d1117 / #238636)

SQLite fits because the dataset is small: a few hundred tasks plus an activity log. That does not justify Postgres. Docker gives me restart: unless-stopped, so the stack recovers automatically when the machine reboots.

A few lessons

If a state machine can do it, don't use an LLM. Task routing is if status == X then notify channel Y. Giving that logic to an LLM is like asking GPT to do addition: it can, but there is no reason to pay for intelligence and creative behavior here.

Monitoring comes before optimization. Context used to creep toward 80% while I only felt that "the replies seem worse lately." The progress bar turned that vague symptom into a visible threshold, and the remedy became trivial.

The dashboard is for the human. Agents interact through APIs. Its only purpose is to show me the global state at a glance, so information density matters more than decorative interaction.

Next steps

  • Cost tracking: calculate daily token consumption and dollar cost for each agent.
  • Automatic resets: trigger /new automatically when context reaches 80%.
  • Replace polling with webhooks: use GitHub webhooks instead of CI polling for faster feedback.

The entire system runs on one machine: 4 agents, 1 router, and 1 dashboard, with zero cloud dependencies beyond the AI APIs. Sometimes the simplest architecture really is the best one.