Treating AI Agents Like Infrastructure: Provisioning Hermes Agent with Ansible

I’ve spent 18 years making cloud infrastructure reproducible. Terraform modules for databases. Ansible roles for VM configuration. Packer templates for golden images. When I started running an AI agent on a Linux VM, I didn’t think about it — I reached for Ansible.

Three days and 15 commits later, I had a fully idempotent playbook that provisions Hermes Agent from bare Ubuntu to production-ready. And the patterns I used? They’re the same ones I use for databases and caching clusters. That’s not a coincidence — it’s the point.

The Problem Nobody’s Solving Yet

Most AI agent setups look like this: install the tool, configure it by hand, hope you remember what you did when you need to do it again. It’s 2015-era infrastructure management applied to a 2026 tool.

AI agents accumulate state:

• Memory files that grow and evolve across sessions
• Configuration with model endpoints, API keys, MCP server definitions, and security policies
• Profiles — isolated personas with their own config, memory, and gateway connections
• Skills cloned from Git repos
• Hooks and scheduled tasks that trigger behavior at lifecycle events
• Secrets for a dozen different LLM providers and integrations

Lose any of this and you’re hand-restoring for hours. Worse, you can’t reproduce the setup on a new VM without remembering every config set command you ran three weeks ago.

This is the same problem platform engineering solved for databases a decade ago: infrastructure that accumulates state needs to be provisioned declaratively, or it rots.

The Architecture: 11 Tasks, One Role

The playbook is a single Ansible role with 11 task files, each gated by tags and feature flags:

harden → install → configure → memory → profiles →
mcp → skills → hooks → services → gateway → cron_jobs

Here’s what each phase handles:

Every phase can run independently via --tags. After the initial provision, a config change is --tags configure and takes 12 seconds.

The Patterns That Transfer from Cloud Infra

This isn’t a “cool Ansible project” post. The interesting part is that every pattern I used comes directly from provisioning databases and caching clusters. If you’ve built IaC at scale, you already know how to do this.

1. Feature Flags for Subsystems

Every Hermes subsystem gets a toggle in defaults/main.yml:

hermes_enable_mcp: true              # MCP servers + npm/pip installs
hermes_enable_skills: true           # Clone skill repos
hermes_enable_hooks: true            # Event hooks
hermes_enable_scheduled_tasks: true  # Cron-based prompt execution
hermes_enable_gateway: true          # Messaging daemon + API server

This is the same pattern I use for Aurora clusters — optional features like Performance Insights or Enhanced Monitoring are boolean flags, not separate modules. You don’t want to fork the role to disable one subsystem.

2. Seed Once, Then Hands Off

Memory files use force: false:

- name: Seed MEMORY.md
  ansible.builtin.template:
    src: memory.md.j2
    dest: "{{ hermes_home }}/memories/MEMORY.md"
    mode: "0644"
    force: false   # Hermes owns this after first run

Ansible writes the initial content. After that, the agent rewrites its own memory every session. Re-running the playbook doesn’t clobber evolved state.

This is identical to how I handle RDS parameter groups — Terraform creates them, but the DBA tunes individual parameters at runtime. The IaC tool seeds the starting state; the runtime system owns the drift.

3. The File Ownership Model

This is the pattern I’m most opinionated about. Every file in the system has exactly one owner:

Ansible manages (rewrites every run):
  ~/.hermes/config.yaml
  ~/.hermes/.env
  ~/.hermes/SOUL.md

Ansible seeds once (force: false):
  ~/.hermes/memories/MEMORY.md    ← agent writes during sessions
  ~/.hermes/memories/USER.md      ← agent writes during sessions

Agent owns (Ansible never touches):
  ~/.hermes/state.db              ← session DB (SQLite)
  ~/.hermes/logs/                 ← runtime logs

No ambiguity. No “who changed this and why” confusion. If Ansible manages it, the template is the source of truth. If the agent owns it, Ansible doesn’t touch it. This is the same ownership contract I enforce between Terraform and application teams — the module owns the resource lifecycle, the team owns the application config within it.

4. Secrets Management via ansible-vault

All secrets live in group_vars/all/vault.yml, encrypted with ansible-vault:

vault_glm_api_key: "..."           # Primary LLM provider
vault_openrouter_api_key: "sk-or-..."
vault_discord_bot_token: "..."
vault_honcho_api_key: "..."        # Persistent memory backend
vault_hermes_api_key: "..."        # API server bearer token

The .env template conditionally renders each key:

{% if vault_glm_api_key is defined and vault_glm_api_key %}
GLM_API_KEY={{ vault_glm_api_key }}
{% endif %}

No plaintext secrets committed to Git — the rendered .env exists only on the host, protected by filesystem permissions. No .env files in the repo. No “I’ll just hardcode it for now.” The same discipline you’d apply to database credentials or cloud provider keys.

5. Idempotent Profile Creation

Profiles are created via the Hermes CLI, wrapped in idempotent Ansible tasks:

hermes_managed_profiles:
  - name: vector
    model: "glm-5.1"
    provider: "zai"
    install_gateway: true
    soul_template: "soul-vector.md.j2"
    memory_template: "memory-vector.md.j2"

The playbook runs hermes profile create (skips if the name exists) and re-applies config set on every run. Each profile gets its own systemd gateway service, its own .env, its own SOUL.md. Three isolated agent personas from one data structure.

This is the Terraform module pattern — declare what you want, let the tool figure out whether it needs to create or update.

The Reprovisioning Story

The real test of any IaC setup: can you nuke the VM and rebuild?

# 1. Terraform: new VM
# 2. Update vault_vm_ip, re-encrypt
# 3. Run the playbook
ansible-playbook -i inventory.yml playbook.yml --ask-vault-pass
# 4. Done. Long-term memory comes back via Honcho, config from Ansible.

Worth being precise about the state boundary, because this is where most “IaC for agents” write-ups get sloppy. Long-term memory lives in Honcho, an external persistent-memory backend that runs alongside Hermes’ built-in local memory — it does not replace it. So the on-box picture is: state.db and logs/ are disposable, config and secrets come from Ansible templates, and the two files that genuinely evolve on-box — MEMORY.md and USER.md — are covered by a scheduled off-box state backup. “Disposable compute” only holds because every byte of state has a named home that isn’t the VM’s disk.

There’s a second half to this contract: drift. The agent rewrites its own memory every session, and some of that is beneficial drift — it learns context rules you never wrote. Backups capture it, but they don’t put it under version control. My rule: when the agent lands on something that should be baseline behavior for every future provision, I promote it by hand into the SOUL.md template in Git. Config lives in Git; memory lives in backups; promotion between the two is a deliberate human act.

This is the same pattern as RDS: the data lives in managed storage, the compute is disposable. Reprovisioning is a terraform apply + ansible-playbook away from a clean slate.

Fallback Routing: Because Primary Providers Go Down

The primary model is GLM-5.1 via Zhipu’s first-class zai provider. But LLM providers have usage windows, rate limits, and occasional outages. The playbook configures automatic failover:

hermes_fallback_enabled: true
hermes_fallback_provider: "minimax-cn"
hermes_fallback_model: "MiniMax-M2.7"

When zai returns a 429, 500, or 503, Hermes switches to MiniMax mid-turn, preserving conversation history. The failover is turn-scoped — the next user message retries the primary, so a provider’s bad hour doesn’t pin you to the fallback. One-shot per turn; if the fallback also fails, normal error handling takes over.

This is the read-replica failover pattern, with one caveat a database engineer should appreciate: the “replica” is not identical. A fallback model can have a different context window and different instruction-following behavior than the primary, so failover is a degradation path, not a transparent one — pick a fallback whose context window covers your typical session length. Not sophisticated, but it covers the 90% case: your agent keeps working when a single provider has a bad hour.

When to Do This

This approach pays off when the conditions are right:

Your agent runs on a VM and accumulates state. Memory files, skills, scheduled tasks, MCP server configs — if losing any of this means hours of manual restoration, you need IaC.

You’re running the same agent for months, not experimenting weekly. The upfront investment in playbook structure pays off when you’re iterating on configuration, not switching agents. Fifteen commits over three days, then every change after that is a one-liner in vars.yml and a 12-second --tags configure run.

You need to reproduce the setup on a new machine. Whether it’s a VM migration, a disaster recovery scenario, or just spinning up a second instance — if the answer to “how do I rebuild this?” is “I don’t remember,” you need declarative provisioning.

You already think in IaC. If you’re comfortable with Ansible roles, Terraform modules, or similar tools, applying them to AI agents is a natural extension. The patterns are identical — the workload is different.

And the inverse: if you’re still evaluating which agent to run, or your setup changes weekly, don’t do this yet. Declarative provisioning pays off on a stable workload. Automating a moving target just gives you a playbook that’s always out of date. Run the agent by hand for a month; when the config stops churning, that’s your signal to write the role.

Conclusion

The platform engineering instinct is simple: if it runs on infrastructure, treat it like infrastructure.

AI agents are infrastructure now. They run on VMs. They accumulate state. They need secrets management, health monitoring, reproducible provisioning, and a clean reprovisioning story. The patterns for all of this already exist — we’ve been using them for databases and caching clusters for years.

None of the patterns here were invented for this project — feature flags, seed-once semantics, ownership contracts, vault-managed secrets, idempotent creation are the same discipline I apply to Aurora clusters and Terraform modules. The workload is new; the discipline isn’t. And as agents get more autonomous, the cost of skipping it goes up: a stateless-microservice mindset applied to an agent doesn’t give you a crash, it gives you catastrophic amnesia. Declarative state management is how that scales.

If you’re running AI agents in production (or getting close), stop treating them like desktop apps. They’re infrastructure. Provision them like it.

The playbook described here provisions Hermes Agent, an open-source AI agent from Nous Research. The patterns apply to any agent that runs on a VM and accumulates state.