AriakiI was discussing a claim with friends recently: as prompting grows into production-grade agent systems, the central problems converge on classic software engineering—specification, safety, and correctness. In practice, we're defining a DSL for agent behavior.
That conclusion came from operating a real system. For the past few months, I've run AI agents as PM, frontend engineer, backend engineer, and QA. They coordinate through message channels, follow a defined workflow, and have explicit permission boundaries. After enough failures, the core problems looked almost identical to conventional software engineering.
Five conclusions that grew out of practice
1. Explicit state machines beat implicit coordination
A task passing through several agents will drift unless its state has a precise definition. Natural-language context feels like shared state until two agents interpret it differently.
I eventually used an explicit state machine:
PROPOSED → DISPATCHED → DONE → QA_PASS / QA_FAIL
Every transition has a trigger and an owner. A PM approves a proposal and marks it DISPATCHED. The developer finishes, marks it DONE, and triggers QA. QA then produces QA_PASS or QA_FAIL.
This is a traditional workflow engine with an LLM agent as executor. Because LLMs interpret ambiguity "creatively," the state definitions need to be stricter, not looser.
2. Permission boundaries are mandatory
Give an agent broad capabilities without hard boundaries and it will find creative ways around the intended process. "Stay in your project" is only a request unless the system limits what the agent can reach.
In practice, four things had to be explicit:
- Tool allowlists: which tools each agent may use
- Directory mappings: which code directory belongs to each work channel, with no cross-directory access
- Branch rules: no direct pushes to the main branch; changes must go through a PR
- Communication protocols: inter-agent messages must use designated channels, with no private injection path
This is capability-based security: an actor receives only the capabilities required for its role. Operating systems and containers do this in conventional infrastructure. Agent systems need the same controls declared in configuration.
3. Invariants grow out of incidents
The most important safety rules often begin as the most painful incidents.
I lost data once. Restarting a service triggered an automatic sync, which cleaned up "orphaned" records and permanently deleted thousands of conversation-history entries. The data was still in the database when I found the problem. I rushed to restore functionality instead of taking an immediate backup. The loss became irreversible.
That failure produced several hard rules:
- Back up first, then act — every operation involving a database begins with a backup
- Data > functionality — lost data is irreversible; broken functionality can be repaired
- Ask when uncertain — one extra confirmation is cheaper than an assumption
These invariants live in the agents' configuration files and load on every startup. They are hard constraints, written into the system instead of left to memory or judgment.
4. A prompt is only one component
Agent development is often presented as prompt engineering with tools attached. Once the system runs continuously, the prompt becomes one component among many.
The actual "program" is the composition of:
- Role definitions: persona and behavioral rules
- Workflow rules: state machines and approval paths
- Permission models: tools, directories, and branches
- State management: task tracking and memory systems
- Communication protocols: channel routing and message formats
- Scheduled jobs: inspections, logs, and automation
Together, these form a DSL, a domain-specific language. It describes who may act, which transitions are legal, where messages go, and what must be remembered. The agent framework is the runtime that executes it.
5. The closer an agent gets to production, the more engineering it requires
A demo agent can chat, call a tool, and write code. Reliable operation requires:
- Precise specifications: an ambiguous instruction is a production bug
- Failure-mode design: agents will make mistakes, so the system needs rollback and audit paths
- Observability: every action needs a log, and critical decisions need human approval
- Idempotency: running the same task twice should not create additional side effects
These are old software-engineering concerns. They matter even more here because the executor is a nondeterministic LLM. A model whose output can vary between runs needs stronger engineering constraints than deterministic code.
Conclusion
The future of agent systems depends far more on better engineering than on smarter models.
As an agent moves from demo to production, less time goes into prompts. More goes into state management, access control, failure recovery, and audit logs. Agent systems are becoming real software systems.
The best agent engineers will very likely be the best software engineers.