Automate Database Health Checks: Build an AI Agent That Monitors Query Performance
Automate Database Health Checks: Build an AI Agent That Monitors Query Performance
If you manage more than a handful of production databases, you already know the drill. Every Monday morning—or worse, every incident at 2 a.m.—someone has to log in, run the same diagnostic queries, squint at the same dashboards, and try to figure out whether that CPU spike is a real problem or just the end-of-month batch job doing its thing.
It's tedious. It's error-prone. And it's one of the highest-leverage workflows you can hand to an AI agent.
This post walks through exactly how to build a database health check agent on OpenClaw—one that monitors query performance, flags anomalies, generates plain-English reports, and only bothers a human when it actually matters. No hand-waving. Specific steps, specific tools, specific code.
Let's get into it.
The Manual Workflow Today: What DBAs Actually Do
Before automating anything, you need to understand what you're replacing. Here's what a typical weekly database health check looks like for a team managing, say, 20–50 PostgreSQL or SQL Server instances:
Step 1: Connect and run diagnostics (~15–30 min per database)
Fire up pgAdmin or SSMS. Run your greatest hits playlist of diagnostic queries:
-- PostgreSQL: Top slow queries
SELECT query, calls, mean_exec_time, total_exec_time
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 20;
-- SQL Server: Current activity
EXEC sp_who2;
-- PostgreSQL: Cache hit ratio
SELECT
sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) AS cache_hit_ratio
FROM pg_statio_user_tables;
Step 2: Review resource metrics (~10–15 min)
Check CPU, memory, disk I/O, connection counts, replication lag, lock/wait statistics. Usually involves switching between your monitoring tool (Datadog, Grafana, CloudWatch) and the database itself.
Step 3: Analyze query performance (~20–40 min)
This is where the real skill comes in. Identify plan regressions, missing indexes, outdated statistics, queries that suddenly went from 50ms to 5 seconds. Compare against last week's baseline.
Step 4: Check logs and errors (~10–15 min)
Scan error logs, deadlock graphs, audit logs for anything unusual.
Step 5: Validate backups and integrity (~5–10 min)
Confirm backup jobs ran. On SQL Server, maybe run DBCC CHECKDB. On PostgreSQL, verify pg_dump or WAL archiving status.
Step 6: Document and act (~15–30 min)
Write up findings. Create Jira tickets. Ping the app team about that query they shipped last Tuesday that's now eating 40% of CPU. Schedule a maintenance window for the index rebuild.
Total time per database: 1–2.5 hours.
Multiply by 50 databases. That's 50–125 hours per week. For context, Quest Software's DBA surveys consistently show the average DBA spends 10–20 hours per week just on monitoring and health checks, and they're managing 50–100+ instances. The math doesn't work. Things get missed.
What Makes This Painful: The Real Costs
The time cost is obvious. But the downstream costs are what actually hurt:
Alert fatigue kills response quality. Static threshold alerts ("CPU > 80%!") fire constantly. After the 50th false positive this month, your team starts ignoring them. Then a real incident slips through. Gartner estimates 70% of database outages stem from configuration drift, failed changes, or undetected performance degradation—exactly the things that get missed when people are numb to alerts.
Context blindness makes triage slow. Your monitoring dashboard says "high CPU on prod-db-07." Okay. Is that the payroll batch job that runs every month? A query regression from yesterday's deploy? A connection leak from a misbehaving microservice? The tool doesn't know. A human has to investigate, and that investigation takes 30–60 minutes minimum.
Off-hours coverage is a gap. Unless you're paying for 24/7 on-call DBA coverage (expensive), issues that surface at 3 a.m. sit until morning. A single hour of downtime on a revenue-critical database costs $100K–$1M+ according to Ponemon Institute data. An hour of undetected degradation can be worse because it compounds.
The skill gap is real. Interpreting complex wait stats, reading execution plans, understanding buffer pool dynamics—this takes years of experience. Junior team members can run the scripts but often can't interpret the results correctly. Senior DBAs become bottlenecks.
Knowledge lives in people's heads. The tribal knowledge about why prod-db-12 always spikes on Thursdays, or which queries are expected to be slow during month-end close—that's rarely documented. When someone leaves, the context leaves with them.
What AI Can Handle Now (And What It Can't)
Let's be honest about what's realistic today. AI isn't replacing your DBA team. But it's excellent at the repetitive detection, correlation, and reporting layers that consume most of their time.
AI handles well:
- Anomaly detection with learned baselines. Instead of "CPU > 80%," the agent learns that CPU on prod-db-07 typically runs at 75% between 2–4 a.m. on the last business day of the month. It only alerts when behavior deviates from that database's normal pattern.
- Query performance regression detection. Comparing this week's
pg_stat_statementsoutput against historical baselines and flagging queries whose mean execution time increased more than a configurable threshold. - Log correlation. Grouping related errors, slow queries, and resource spikes into a single root-cause narrative instead of 47 separate alerts.
- Index and optimization recommendations. Analyzing query patterns and suggesting missing indexes, statistics updates, or query rewrites.
- Health scoring. Synthesizing dozens of metrics into a single "database health score" with plain-English explanations.
- Capacity forecasting. Projecting disk growth, connection pool exhaustion, and other resource trends.
- Report generation. Turning raw diagnostics into stakeholder-friendly summaries.
AI still struggles with:
- Business impact judgment. Is a 200ms query regression acceptable? Depends on whether it's powering the checkout page or an internal admin tool. The agent doesn't know your business priorities without explicit context.
- Risk evaluation of changes. Adding an index might fix one query and break another. Approving schema changes requires understanding application behavior.
- Architectural decisions. When to shard, migrate, or change data models.
- Novel failure modes. Completely new failure patterns not present in historical data.
- Stakeholder communication. Explaining to the VP of Engineering why you need a maintenance window during peak hours.
The sweet spot: AI handles detection, correlation, and recommendation. Humans handle decision-making and accountability.
Step-by-Step: Building the Agent on OpenClaw
Here's the concrete implementation. We're building an agent that connects to your databases on a schedule, runs health checks, analyzes results, and delivers actionable reports—escalating to humans only when needed.
Step 1: Define Your Data Sources and Check Library
Start by cataloging what you want the agent to monitor. Create a structured check library:
HEALTH_CHECKS = {
"postgresql": {
"slow_queries": {
"query": """
SELECT query, calls, mean_exec_time, total_exec_time,
stddev_exec_time, rows
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 25;
""",
"description": "Top 25 slowest queries by mean execution time",
"threshold_type": "regression", # compare against baseline
},
"cache_hit_ratio": {
"query": """
SELECT
sum(heap_blks_hit) / NULLIF(sum(heap_blks_hit) + sum(heap_blks_read), 0)
AS cache_hit_ratio
FROM pg_statio_user_tables;
""",
"description": "Buffer cache hit ratio (should be > 0.99)",
"threshold_type": "minimum",
"threshold_value": 0.99,
},
"connection_count": {
"query": """
SELECT count(*) as active_connections,
current_setting('max_connections')::int as max_connections
FROM pg_stat_activity
WHERE state = 'active';
""",
"description": "Active connections vs max",
"threshold_type": "ratio",
"threshold_value": 0.8, # alert at 80% of max
},
"replication_lag": {
"query": """
SELECT EXTRACT(EPOCH FROM replay_lag) as lag_seconds
FROM pg_stat_replication;
""",
"description": "Replication lag in seconds",
"threshold_type": "maximum",
"threshold_value": 30,
},
"table_bloat": {
"query": """
SELECT schemaname, tablename,
pg_size_pretty(pg_total_relation_size(schemaname || '.' || tablename)) as total_size,
n_dead_tup,
n_live_tup,
CASE WHEN n_live_tup > 0
THEN round(n_dead_tup::numeric / n_live_tup, 4)
ELSE 0 END as dead_ratio
FROM pg_stat_user_tables
WHERE n_dead_tup > 10000
ORDER BY n_dead_tup DESC
LIMIT 10;
""",
"description": "Tables with significant bloat needing vacuum",
"threshold_type": "maximum",
"threshold_value": 0.2, # dead/live ratio
},
"long_running_queries": {
"query": """
SELECT pid, now() - pg_stat_activity.query_start AS duration,
query, state
FROM pg_stat_activity
WHERE (now() - pg_stat_activity.query_start) > interval '5 minutes'
AND state != 'idle'
ORDER BY duration DESC;
""",
"description": "Queries running longer than 5 minutes",
"threshold_type": "count",
"threshold_value": 0,
},
"index_usage": {
"query": """
SELECT schemaname, tablename, indexrelname,
idx_scan, idx_tup_read, idx_tup_fetch
FROM pg_stat_user_indexes
WHERE idx_scan = 0
AND schemaname NOT IN ('pg_catalog', 'pg_toast')
ORDER BY pg_relation_size(indexrelid) DESC
LIMIT 10;
""",
"description": "Unused indexes consuming storage",
"threshold_type": "informational",
},
}
}
This isn't just a list of queries—it's structured metadata that the OpenClaw agent uses to interpret results intelligently.
Step 2: Build the Data Collection Layer
The agent needs to safely connect to each database, execute checks, and store results. Here's the collection function:
import psycopg2
import json
from datetime import datetime
def collect_health_data(db_config: dict, checks: dict) -> dict:
"""
Connect to a database instance and run all health checks.
Returns structured results with metadata.
"""
results = {
"database": db_config["name"],
"host": db_config["host"],
"timestamp": datetime.utcnow().isoformat(),
"checks": {},
"errors": []
}
try:
conn = psycopg2.connect(
host=db_config["host"],
port=db_config.get("port", 5432),
dbname=db_config["dbname"],
user=db_config["user"],
password=db_config["password"],
connect_timeout=10,
options="-c statement_timeout=30000" # 30s timeout per query
)
conn.set_session(readonly=True) # CRITICAL: read-only
with conn.cursor() as cur:
for check_name, check_config in checks.items():
try:
cur.execute(check_config["query"])
columns = [desc[0] for desc in cur.description]
rows = cur.fetchall()
results["checks"][check_name] = {
"status": "collected",
"columns": columns,
"data": [dict(zip(columns, row)) for row in rows],
"row_count": len(rows),
"description": check_config["description"],
"threshold_type": check_config["threshold_type"],
"threshold_value": check_config.get("threshold_value"),
}
except Exception as e:
results["errors"].append({
"check": check_name,
"error": str(e)
})
conn.close()
except Exception as e:
results["errors"].append({
"check": "connection",
"error": str(e)
})
return results
Two critical details here: the connection is read-only (the agent should never write to production), and every query has a timeout (you don't want the health check itself becoming a performance problem).
Step 3: Wire It Into OpenClaw as an Agent
This is where it comes together. In OpenClaw, you define the agent with its tools, context, and instructions. The agent orchestrates the data collection, interprets results against baselines, and generates the report.
Configure the agent in OpenClaw with access to these tools:
- Database collector tool — Wraps the
collect_health_datafunction above, parameterized by database config. - Baseline store — A simple key-value store (Redis, DynamoDB, even a JSON file) holding previous check results for comparison.
- Notification tool — Sends reports via Slack, email, or PagerDuty based on severity.
The agent's system prompt should encode your team's domain knowledge—the context that usually lives in senior DBAs' heads:
You are a database health monitoring agent. You analyze PostgreSQL
diagnostic data and produce actionable health reports.
CONTEXT FOR THIS ENVIRONMENT:
- prod-db-01 through prod-db-05 are customer-facing OLTP databases.
Query latency regressions on these are high-priority.
- analytics-db-01 and analytics-db-02 run heavy batch jobs between
02:00-05:00 UTC. High CPU/IO during this window is expected.
- The payments service (prod-db-03) has a zero-tolerance SLA.
Any degradation should escalate immediately.
- End-of-month (last 2 business days) always shows elevated load
on prod-db-01 and prod-db-02. This is normal.
ANALYSIS INSTRUCTIONS:
1. Compare current results against the stored baseline from the
previous period.
2. Flag any query whose mean_exec_time increased by more than 50%
with more than 100 calls.
3. For anomalies, attempt to correlate across checks (e.g., slow
queries + high connection count might indicate connection pool
exhaustion).
4. Classify each finding as: CRITICAL, WARNING, INFO.
5. For CRITICAL and WARNING, include a specific recommended action.
6. Generate a health score from 0-100 for each database.
OUTPUT FORMAT:
Produce a structured JSON report, then a plain-English summary
suitable for a Slack message.
This is the key advantage of building on OpenClaw rather than duct-taping scripts together: the agent combines structured data analysis with natural language reasoning. It doesn't just check if a number exceeds a threshold—it understands that high CPU on analytics-db-02 at 3 a.m. is expected, while the same pattern on prod-db-03 at 3 p.m. is an emergency.
Step 4: Schedule and Run
Set the agent to run on a schedule that matches your needs:
- Every 15 minutes: Lightweight checks (connection counts, replication lag, long-running queries)
- Every hour: Query performance analysis against baselines
- Daily at 6 a.m.: Full health report with trend analysis
- Weekly: Comprehensive review with capacity forecasting and optimization recommendations
For the scheduling layer, use whatever you already have: cron, Airflow, AWS EventBridge, a simple GitHub Action. The agent invocation from your scheduler is straightforward—call the OpenClaw API with the appropriate check configuration, and the agent handles the rest.
Step 5: Baseline Learning and Anomaly Detection
The agent gets smarter over time. After each run, store the results:
def update_baseline(db_name: str, check_name: str, current_data: dict,
baseline_store: dict, alpha: float = 0.3):
"""
Exponential moving average for baselines.
Alpha controls how quickly the baseline adapts.
"""
key = f"{db_name}:{check_name}"
if key not in baseline_store:
baseline_store[key] = current_data
return
previous = baseline_store[key]
# For numeric metrics, compute EMA
if "value" in current_data and "value" in previous:
baseline_store[key]["value"] = (
alpha * current_data["value"] +
(1 - alpha) * previous["value"]
)
baseline_store[key]["last_raw"] = current_data["value"]
baseline_store[key]["updated_at"] = datetime.utcnow().isoformat()
After a few weeks of data, the agent has per-database, per-check baselines that account for daily and weekly patterns. Anomaly detection becomes comparison against learned normal rather than static thresholds—which is exactly what eliminates alert fatigue.
Step 6: Report and Escalate
The agent's output has two modes:
Routine report (daily/weekly): A summary delivered to Slack or email. Looks something like:
📊 Daily Database Health Report — June 12, 2026
Overall Health: 94/100
✅ prod-db-01: 97/100 — All checks nominal
✅ prod-db-02: 95/100 — All checks nominal
⚠️ prod-db-03: 82/100 — 2 warnings
- Query regression: `SELECT * FROM transactions WHERE...`
mean_exec_time increased 73% (12ms → 21ms) over 48 hours.
2,340 calls/hour. Recommended: Review execution plan,
consider index on transactions(merchant_id, created_at).
- Connection utilization at 76% of max (152/200).
Trending upward. At current rate, will hit 80% threshold
in ~6 days.
✅ prod-db-04: 98/100 — All checks nominal
✅ prod-db-05: 96/100 — All checks nominal
ℹ️ analytics-db-01: 91/100 — Expected elevated I/O from batch jobs
✅ analytics-db-02: 93/100 — All checks nominal
📈 Trends: prod-db-03 disk usage growing 2.1GB/week.
Current: 340GB/500GB. Projected to need attention in ~11 weeks.
🗑️ Optimization opportunities: 4 unused indexes found across
estate consuming 12GB total. See detailed report.
Escalation (immediate): For CRITICAL findings, the agent sends a PagerDuty alert or direct Slack message with full context:
🚨 CRITICAL: prod-db-03 (payments)
Replication lag: 45 seconds (threshold: 30s)
Correlated finding: 3 long-running queries blocking replication
Longest running: DELETE FROM payment_logs... (running 12 minutes)
Recommended immediate action:
1. Check if PID 28471 can be safely terminated
2. Verify replication catches up after resolution
3. Investigate why payment_logs cleanup is not batched
This database has a zero-tolerance SLA. Escalating to on-call DBA.
That context—the correlation between replication lag and long-running queries, the specific PID, the knowledge about the SLA—is what turns a noisy alert into an actionable notification.
What Still Needs a Human
Even with a well-tuned agent, certain decisions stay with your team:
- Approving index changes. The agent can recommend
CREATE INDEX idx_transactions_merchant_created ON transactions(merchant_id, created_at), but a human should review the write overhead implications and approve it. - Business priority calls. The agent flags a regression on an internal tool. Is it worth fixing this sprint? That's a product decision.
- Architectural changes. The agent can tell you prod-db-03 is approaching capacity limits. Whether you scale vertically, add read replicas, or redesign the schema is a human call.
- Compliance interpretation. The agent can detect anomalous access patterns, but determining whether they constitute a HIPAA violation requires human judgment.
- Novel incidents. A failure mode the agent hasn't seen before will get flagged as anomalous, but root-cause analysis of truly novel issues still needs experienced engineers.
The goal isn't to remove humans from the loop. It's to make sure humans only engage on problems that actually require human judgment, rather than spending 20 hours a week running the same SQL scripts and eyeballing the same dashboards.
Expected Time and Cost Savings
Based on the patterns we see across teams using AI agents for database monitoring:
Time savings:
- Weekly health check effort: From 50–125 hours/week (manual across a 50-database estate) to ~5–10 hours/week of human review time. That's a 75–90% reduction in routine effort.
- Incident triage time: From 30–60 minutes to identify root cause down to near-instant correlation. Dynatrace customers report 60–80% reduction in MTTR with AI-assisted root cause analysis, and you can achieve similar results with a well-configured OpenClaw agent.
- Report generation: From 2–3 hours of manual writeup to zero. The agent writes the report.
Cost savings:
- Reduced downtime: If AI-driven monitoring catches even one issue per quarter that would have caused an hour of outage, that's $100K–$1M+ in avoided losses for revenue-critical systems.
- DBA leverage: Your senior DBAs spend time on architecture, optimization, and mentoring instead of running
sp_who2for the thousandth time. A team of 3 DBAs managing 50 databases can often scale to 100+ without additional headcount. - Fewer missed issues: The agent checks every database, every check, every time. Humans skip steps when they're busy. Consistency has compounding value.
Realistic timeline to value:
- Week 1: Basic checks running, raw data collection working.
- Week 2–3: Agent producing interpretive reports, initial baseline established.
- Month 2: Baselines stabilized, anomaly detection accurate, alert fatigue significantly reduced.
- Month 3+: Agent generating optimization recommendations from trend data. Continuous improvement from there.
Get Started
If you're running enough databases that health checks have become a tax on your team's time, this is one of the highest-ROI automations you can build. The workflow is well-defined, the data is structured, and the interpretation rules are learnable—all characteristics that make it ideal for an AI agent.
You don't need to automate everything at once. Start with the three checks that consume the most time or cause the most incidents. Get those running on OpenClaw, prove the value, and expand from there.
You can find OpenClaw and pre-built database monitoring agent templates on Claw Mart. If you want help designing an agent for your specific database estate, check out Clawsourcing—you can work directly with builders who've done this before and get a production-ready agent without starting from scratch.