Guides

Monitoring

Scrape the office with Prometheus, and the alerts worth setting.

The office answers GET /healthz for liveness and, once you turn it on, GET /metrics in the Prometheus text format.

Turn on /metrics#

$ sudo -u office officectl secrets set metrics.token     # paste a long random string

Until a token is stored, /metrics answers 404. Scrapers send the token as a bearer:

prometheus.yml

scrape_configs:
  - job_name: agent-office
    scheme: https
    authorization: { credentials: <the token> }
    static_configs: [ { targets: [ office.example.com ] } ]

What it reports#

Metric	Type	Meaning
`agent_office_info{version,office}`	gauge	Always 1; the labels say which platform version runs.
`agent_office_agents{status}`	gauge	Agents by status (hired, suspended, …).
`agent_office_runs_total{status}`	counter	Runs by status since the office began.
`agent_office_runs_failed_24h`	gauge	Runs that failed in the last 24 hours.
`agent_office_spend_usd_today{agent}`	gauge	Spend today per agent (office time zone).
`agent_office_approvals_pending`	gauge	Approvals waiting for a person.
`agent_office_ledger_seq`	counter	The newest ledger entry.
`agent_office_ledger_export_lag`	gauge	Entries not yet delivered to your collector (with `ledger.export`).
`agent_office_backup_last_success_timestamp_seconds`	gauge	When the last scheduled or manual backup finished.
`agent_office_notifications_failed_24h`	gauge	Notifications that could not be delivered in the last 24 hours.

Metrics carry counts and states only: no mail subjects, chat or file names.

Alerts worth setting#

/healthz failing for 5 minutes.
agent_office_runs_failed_24h > 3: agents are failing; check Runs.
agent_office_ledger_export_lag > 1000 for 30 minutes: the audit trail isn't leaving the host.
time() - agent_office_backup_last_success_timestamp_seconds > 36 * 3600: no backup in a day and a half.
agent_office_approvals_pending > 0 for 24 hours: someone isn't seeing Needs you (set up notifications).