Diagnostic System
Operational reference for backend/scripts/diagnose.py
What is the diagnostic?
The diagnostic is a standalone CLI tool that validates every layer of the Ghost Biographer infrastructure stack — environment, database, blob storage, AI services, security controls, and end-to-end integration — at five progressively deeper levels.
It runs outside the server process. If the API will not start, the diagnostic still works and can tell you exactly why.
Quick start
# Activate the venv first
.venv\Scripts\activate # Windows
source .venv/bin/activate # macOS / Linux
# Run from the backend/ directory
cd backend
# Most common — Level 2 connectivity check
python scripts/diagnose.py
# Specific level
python scripts/diagnose.py --level 3
# Only certain check categories
python scripts/diagnose.py --level 4 --checks isolation,tokens
# Machine-readable JSON output (for CI / monitoring)
python scripts/diagnose.py --level 3 --json
# Override per-check timeout (seconds)
python scripts/diagnose.py --level 2 --timeout 30
The script reads
../.env (relative to backend/) using the same
Settings object the application uses. No separate configuration is needed.
The five levels
1
Environment
ENV vars, imports, config object — all local, no network
2
Connectivity
TCP + read-only probe of every external service
3
Functional
Schema, auth, TOTP, blob round-trip, prompt builders — no AI calls
4
Security
Auth hardening, token integrity, cross-user isolation, injection resistance, information disclosure
5
Comprehensive
Everything in 1–4 plus real AI calls, Whisper STT, and full end-to-end stack exercise
Reading the output
Ghost Biographer — Level 3 Diagnostic
Endpoint: http://localhost:8000 DB: ghostbio-prod.postgres.database.azure.com
══════════════════════════════════════════════════════════════
Environment
✓ ENV vars present 23 / 23 required 0.01s
✓ Settings loaded 0.01s
Database
✓ TCP connection connected 0.31s
✓ Alembic revision at head (d8edd26) 0.09s
Auth System
✓ JWT round-trip access token OK 0.01s
⚠ bcrypt cost factor 12 — prefer ≥ 14 0.08s
Stripe
✗ API connectivity connection timeout 5.01s
══════════════════════════════════════════════════════════════
17 passed 1 warning 1 FAILED
Total: 7.4 s
FAILED:
• Stripe / API connectivity — connection timeout
Check STRIPE_SECRET_KEY in .env and verify outbound HTTPS to api.stripe.com
Each row shows: status icon · category · check name · detail message · duration.
Failures are also summarised at the bottom with a remediation hint. Warnings are informational and do not affect the exit code.
Exit codes
0 — all checks passed
1 — one or more checks failed
2 — config error prevented run
Exit code 1 is suitable for use in CI pipelines — a failed diagnostic blocks the pipeline.
Level 1 — Environment
Everything here is synchronous and local. If Level 1 fails, no other level will work. Run this first on a new machine or after changing .env.
| Check | Pass condition |
|---|---|
| Required ENV vars present | All 23 required variables set and non-empty |
| Settings object loads | Settings() instantiates without exception |
| Database URL scheme | Starts with postgresql+asyncpg:// |
| Python imports | fastapi, sqlalchemy, openai, anthropic, etc. all import cleanly |
| alembic.ini present | File exists and is parseable |
Level 2 — Connectivity
Read-only network probes. Use this when setting up a new machine or troubleshooting firewall rules. All checks that require optional configuration (Grok, Claude models, Redis, Kokoro, ACA) issue a warn rather than fail when the relevant env vars are not set, so local dev is never broken by unconfigured optional services.
Database & Storage
| Check | Pass condition |
|---|---|
| Database TCP connect | Port 5432 reachable at the configured host |
| Database SELECT 1 | Query completes and returns expected value |
| Blob Storage auth | Client authenticates and lists containers |
AI Models
| Check | Pass condition |
|---|---|
| Azure AI Foundry reachable | HTTP probe to /openai/models returns non-5xx with valid API key |
| GPT-5.4 smoke test | Minimal inference call returns valid text; token usage reported |
| Claude Haiku endpoint reachable | HTTP probe returns non-5xx; warn if not configured |
| Claude Sonnet endpoint reachable | HTTP probe returns non-5xx; warn if not configured |
| Claude Opus endpoint reachable | HTTP probe returns non-5xx; warn if not configured |
| Grok endpoint reachable | HTTP probe to /models with api-key header; warn if not configured |
| Whisper smoke test | 0.2 s silence clip transcribes without error; deployment responds |
Infrastructure
| Check | Pass condition |
|---|---|
| Redis TCP connect | Port 6379 reachable; warn if REDIS_URL unset |
| Kokoro TTS service reachable | GET /health returns non-5xx; warn if unreachable |
| SMTP TCP connect | Configured SMTP port reachable; warn if unreachable |
| History Service reachable | Health endpoint returns OK; warn if HISTORY_SERVICE_URL unset |
| Stripe API ping | API ping succeeds; key accepted; warn if in live mode |
| ACA runtime context | ACA-injected env vars (CONTAINER_APP_NAME, revision, replica) present; warn if not running in Azure Container Apps |
| ACA HTTPS health probe | HTTPS GET to live container app FQDN returns 200; warn if AZURE_CONTAINER_APP_FQDN unset |
| Azure Container Registry reachable | Probe to {ACR}/v2/ returns 200 or 401 (registry live); warn if AZURE_ACR_SERVER unset |
The most common Level 2 failure is Database TCP timeout. The Azure PostgreSQL
server only accepts connections from IP addresses on its firewall allowlist. Add your current
public IP in Azure Portal → PostgreSQL server → Networking.
Level 3 — Functional
Writes small test fixtures to verify the application logic works. All fixtures are deleted when the run completes. No AI API calls are made. Run this before a demo or after a deployment.
| Category | Check | Pass condition |
|---|---|---|
| Database | users table schema | Columns match SQLAlchemy model |
| Database | photos table schema | Columns match SQLAlchemy model |
| Database | life_arc_nodes schema | Columns match SQLAlchemy model |
| Database | Alembic revision at head | DB revision matches alembic heads |
| Blob Storage | Canary write/read/delete | 1 KB file round-trips correctly |
| Auth | JWT access token round-trip | Encode → decode → correct sub and type |
| Auth | JWT refresh token round-trip | Encode → decode → correct sub and type |
| Auth | TOTP generate + verify | Secret → code → verify passes |
| Auth | Password hash + verify | Correct password passes; wrong fails |
| Prompts | Phase 1 prompt builder | Builds from synthetic data; structure valid |
| Prompts | Phase 3 message builder | Builds interview messages; structure valid |
Level 4 — Security
Active probing of authentication, access control, cross-user data isolation, input
sanitisation, information disclosure, secrets hygiene, authorization controls, audit
trail integrity, and the maintenance kill switch. Creates two ephemeral test users
(Alice and Bob) and destroys them in a finally block regardless of outcome.
A failure in section 4.3 Cross-User Isolation is a critical security
vulnerability. If Bob can read Alice's data, that section halts immediately and the
failure is printed prominently. This must be resolved before the application handles
real user data.
4.1 — Secrets & Configuration Hardening
| Check | Pass condition |
|---|---|
| JWT secret entropy | ≥ 32 chars warn if < 64 |
| JWT algorithm | Not none; matches decode_token() enforcement |
| bcrypt cost factor | ≥ 12 warn if < 14 |
| Debug mode off | settings.debug is False |
| CORS origins not wildcard | allow_origins ≠ ["*"] |
| Partial token expiry | ≤ 10 minutes |
| Database URL uses SSL | sslmode=require or equivalent present |
| Blob storage uses HTTPS | Storage account URL scheme is https:// |
4.2 — Authentication & Token Integrity
| Check | Pass condition |
|---|---|
| Wrong password rejected | 401 |
| User enumeration resistant | Same error text; timing delta < 5 ms for wrong password vs unknown email |
| Partial token as access token | Rejected by protected endpoints |
| Refresh token as access token | Rejected by protected endpoints |
| Access token as refresh token | Rejected by /auth/token/refresh |
| Expired token rejected | 401 |
| Tampered token rejected | 401 |
| Token without Bearer prefix | Rejected (403) |
| Wrong TOTP code rejected | 401 |
| TOTP code must be 6 digits | 5-digit and 7-digit → 422 |
| Inactive user rejected | 401 at login |
4.3 — Cross-User Data Isolation
Alice creates a photo, a life-arc node, a Phase 1 session, and an interview session. Every Bob request below must return 403 or 404 — never 200.
| Check | Pass condition |
|---|---|
| Bob → Alice photo (GET) | 403 or 404 200 = critical |
| Bob → Alice photo (DELETE) | 403 or 404 200 = critical |
| Bob → Alice photo (annotate) | 403 or 404 200 = critical |
| Bob → life-arc node list | Returns only Bob's nodes (zero) — not Alice's |
| Bob → Alice interview (GET) | 403 or 404 200 = critical |
| Bob → Alice interview (reply) | 403 or 404 200 = critical |
| Service layer user_id scope (AST) | Every select() in app/services/ and app/api/ has a .where(user_id) guard; violations → fail |
4.4 — Input Sanitisation & Injection Resistance
| Check | Pass condition |
|---|---|
| SQL injection in email | 422 or 401; never 200 or 500 |
| SQL injection in content | Stored verbatim (parameterised); not executed |
| XSS payload in content | Escaped in response; not echoed as raw HTML |
| Oversized payload (2 MB) | 413 or 422; server does not crash |
| Null byte in password | 422; not 500 |
| Invalid UUID in path param | 422; not 500 |
| Path traversal in filename | Stored as literal text; not traversed |
4.5 — Information Disclosure
| Check | Pass condition |
|---|---|
| No password hash in responses | hashed_password and totp_secret absent from all user-returning endpoints |
| No stack trace in 500 responses | Error response is {"detail": "..."} only (non-debug mode) |
| No user enumeration via error text | Login error text identical for wrong password and unknown email |
| Auth log written | Entries present in auth_events table for test operations |
4.6 — Secrets Hygiene
| Check | Pass condition |
|---|---|
| SECRET_KEY not a known-weak default | Not in list of 20 common placeholder values; ≥ 32 chars; warn if < 48 chars |
| Stripe key format valid | Starts with sk_; warn if sk_live_ (expected test mode during dev) |
4.7 — Token Replay & Rate Limiting
| Check | Pass condition |
|---|---|
| Refresh token replay rejected | Second use of the same refresh token returns non-200; surfaces stateless JWT replay vulnerability if present |
| Rate limiting on auth endpoints | 35 rapid login attempts trigger 429; warn (not fail) if no rate limiting — acceptable before beta but must be resolved for production |
4.8 — Authorization Controls
| Check | Pass condition |
|---|---|
| Admin endpoints locked to admin role | Anonymous request + regular user request to /feedback/admin both return 403; 200 is a critical failure |
| Data endpoints require authentication | 6 data endpoints probed without token all return 401 or 403; 200 or 500 is a failure |
4.9 — Extended Data Isolation
Alice and Bob extended to Life-Arc, persona, and face-cluster endpoints.
| Check | Pass condition |
|---|---|
| Bob → Alice life-arc nodes | Returns only Bob's nodes (zero) Alice's nodes = critical |
| Bob → Alice persona suggestions | Returns only Bob's personas (zero) Alice's personas = critical |
| Bob → Alice face clusters | Returns only Bob's clusters (zero) Alice's clusters = critical |
4.10 — Audit Trail Integrity
| Check | Pass condition |
|---|---|
| auth_events table indexed for incident response | Table has created_at and event_type indexes; recent events exist; warn if indexes missing (not fail — audit still functions) |
4.11 — Kill Switch Verification
| Check | Pass condition |
|---|---|
| Maintenance mode kill switch functional | SQL toggle to true → regular endpoint returns 503 → health still 200 → SQL toggle to false → regular endpoint returns non-503; restores in finally block regardless of outcome |
Section 4.11 is a quick passive check (one probe cycle). For a full 30-second live drill
with detailed probe timeline and phase report, use the
Kobayashi Maru drill.
Level 5 — Comprehensive
Everything in Levels 1–4, plus real AI calls. Before executing AI checks, the tool prints an estimated token cost and pauses 5 seconds — press Ctrl+C to abort.
| Check | Pass condition |
|---|---|
| Azure AI (GPT-5.4) — Phase 1 prompt | Real completion returns valid text; token usage reported |
| Azure AI (GPT-5.4) — node extraction | Response parseable JSON matching Life-Arc node schema |
| Anthropic (Claude Sonnet) — Phase 1 | Fallback path returns valid text |
| Grok smoke test | Minimal inference call returns valid text; warn if not configured |
| Whisper STT | Transcribes 0.2 s silence clip; deployment responds |
| DB write path — test user lifecycle | Create user → write node → read node → delete both |
| Blob — synthetic photo end-to-end | Upload PNG → verify in photo list API → delete |
| Stripe webhook secret format | Secret format valid (no live call) |
Level 6 — E2E Smoke Test
A single sequential check that exercises the complete happy-path user flow against a
live server. Safe to run against staging — creates a temporary smoke account and hard-deletes
it on completion (the cleanup step runs in a finally block, so no accounts are
ever leaked even if earlier steps fail).
Level 6 makes real AI calls (Phase 1 greet, outline generation, chapter draft) and incurs
token cost. Estimated 5 000–15 000 tokens per run depending on outline length.
Run on demand or post-deploy, not on every commit.
| Step | Pass condition | Failure behaviour |
|---|---|---|
| 01 server reachable | GET /health → {"status":"ok"} | Abort — all remaining steps skipped |
| 02 cleanup stale account | Any previous smoke account deleted from DB | Non-fatal — continues |
| 03 register account | POST /auth/register → 201 with user id | Abort — cannot proceed without account |
| 04 TOTP setup + confirm | Login → setup → confirm → full access token obtained | Abort — no token, all subsequent HTTP steps fail |
| 05 Phase 1 session + AI greet | GET /register/session + POST /register/greet return AI greeting text | Non-fatal — continues; outline still generated |
| 06 Phase 1 DB seed + complete | 4 Life-Arc nodes inserted; biography_phases.status set to complete | Non-fatal — outline may produce 0 chapters |
| 07 photo upload | POST /photos/ → 201 with photo id | Non-fatal — face detection and Memory Lens steps skip gracefully |
| 08 face detection + annotation | Photo leaves pending status within 45 s | warn if still pending; does not fail the run |
| 09 Memory Lens photo list | GET /photos/ returns list including the uploaded photo | Non-fatal |
| 10 biographer chat session | GET /biographer/chat/session → session id in response | Non-fatal |
| 11 outline generation | POST /phase4/outline/generate → 200 with ≥1 chapter | Fatal for run — raises CheckFailed |
| 12 chapter 1 draft | POST /phase5/draft/chapters/1/draft → 200, draft text ≥50 chars | Fatal for run — raises CheckFailed |
| 13 cleanup | Smoke account hard-deleted from DB | Always runs (finally block); non-fatal if account already gone |
Endpoints
Three ways to invoke Level 6:
CLI (local or staging, requires all env vars):
# Set SMOKE_TEST_URL if not probing localhost
SMOKE_TEST_URL=https://your-staging-url.azurecontainerapps.io \
python backend/scripts/diagnose.py --level 6
Admin diagnostics endpoint (JWT required):
POST /api/v1/admin/diagnostics/run
Authorization: Bearer <admin-jwt>
Content-Type: application/json
{ "level": 6 }
Smoke endpoint (X-Admin-Key — designed for CI, no TOTP needed):
POST /api/v1/admin/diagnostics/smoke
X-Admin-Key: <ADMIN_CLI_KEY from .env>
The smoke endpoint self-probes via SMOKE_TEST_URL env var set on the
Container App (default http://127.0.0.1:8000), so CI only needs to reach the
server once — the server runs its own end-to-end walk-through.
CI integration
The deploy-backend.yml workflow calls the smoke endpoint after the health
check passes. The step exits non-zero if any smoke step fails or errors, blocking the
deployment from being marked successful.
Kobayashi Maru — Live Kill-Switch Drill
A real, live test of the maintenance mode kill switch. Unlike the passive 4.11 check (one probe cycle), the Kobayashi Maru runs for 30 seconds and collects a detailed probe timeline to prove the switch engaged cleanly across multiple samples.
This is a live operation — the application will serve HTTP 503 to all
users for approximately 35 seconds. Do not run during business hours without advance notice.
Authentication
The drill requires two factors:
- A valid admin JWT (sign in at the home page with an admin account)
- Six NATO phonetic words from
KOBAYASHI_MARU_CODEin.env— spoken aloud to a second admin to confirm intent before proceeding
The code is stored only in .env — never logged, never returned in responses,
never committed to git. Wrong codes are logged by email address only (never by what was typed).
Change the default words before going to production.
Drill phases
| Phase | What happens | Abort condition |
|---|---|---|
| 1 — Pre-flight | Verify health baseline is 200; confirm not already in maintenance | Health ≠ 200 or already in maintenance |
| 2 — Engage | SQL upsert sets maintenance_mode = true; flush_cache() takes effect immediately | DB write fails |
| 3 — Capture (30 s) | Every 2 s: probe regular endpoint (expect 503), /health (expect 200), /admin/maintenance (expect 200) — 15 rounds × 3 probes = 45 data points | — |
| 4 — Restore | SQL upsert sets maintenance_mode = false; flush_cache(); always runs even if capture errored | Logs CRITICAL if restore fails; manual fix required |
| 5 — Verify | Re-probe health and regular endpoint to confirm 503 is gone | — |
Verdicts
| Verdict | Meaning |
|---|---|
| PASS | All regular probes returned 503, all health/admin passthroughs returned 200, restore confirmed clean |
| PARTIAL | Kill switch worked on most probes but anomalies detected — check probe timeline for which samples misbehaved |
| FAIL | Kill switch never produced 503, restore did not complete cleanly, or middleware not loaded |
| CRITICAL FAILURE | Restore failed — system is still in maintenance mode. Call POST /api/v1/admin/maintenance {"enabled": false} immediately with an admin JWT |
Endpoint
POST /api/v1/admin/kobayashi-maru
Authorization: Bearer <admin-jwt>
Content-Type: application/json
{ "code": "WORD1 WORD2 WORD3 WORD4 WORD5 WORD6" }
The drill is also accessible from the GhostFleet Sentinel page at /sentinel — enter the code in the Kobayashi Maru panel and click ENGAGE.
FAQ
The server starts fine but every API call returns 500. What level should I run?
Run
python scripts/diagnose.py --level 2. The most common cause is the database being unreachable — a 500 on every DB-backed endpoint is exactly what that looks like. The Level 2 connectivity check will confirm or rule it out within 15 seconds.Level 4 created test users and then crashed. Are they still in the database?
Possibly — test users are deleted in a
finally block, but if the database itself is unreachable during cleanup they will persist. The output will have printed their email addresses and IDs. Search the users table for emails matching the pattern diag-alice-*@ghostbiographer.test and diag-bob-*@ghostbiographer.test and delete them manually.The AST inspection check in 4.3 is failing but I can't find a missing user_id filter.
The AST check looks for
.where(Model.user_id == ...) or .filter(user_id=...) patterns. If you use a helper function that applies the filter internally, the AST checker won't see it. Add a # diag: user_id-scoped comment on the line to suppress the check and document why it is safe. If you're unsure, confirm with a manual runtime test: log in as two different users and verify neither can access the other's data.Level 5 is taking longer than 10 minutes. Is something wrong?
Each AI check has a 60-second timeout. If a check exceeds it, it fails with a timeout error and the run continues. A full run taking more than 10 minutes usually means the Azure AI Foundry endpoint is under load or a Whisper deployment is cold-starting. The per-check timing in the output will identify the slow step.
Can I run the diagnostic in CI?
Yes. Use
--json for machine-readable output and check the exit code: 0 = all pass, 1 = failures. We recommend Level 3 in CI (fast, no AI cost) and Level 4 on a nightly schedule. Level 5 should be run manually before major releases. Level 6 (E2E smoke) runs automatically in CI after every deployment via POST /admin/diagnostics/smoke with the X-Admin-Key header — no TOTP required.I'm getting a "Database TCP timeout" on a new machine.
The Azure PostgreSQL server only accepts connections from IP addresses on its allowlist. Go to Azure Portal → PostgreSQL server → Networking and add your current public IP. Find your IP at
whatismyipaddress.com. If you're on a corporate network or VPN, port 5432 outbound may also be blocked — test with Test-NetConnection -ComputerName <host> -Port 5432 (PowerShell).The "debug mode" check is failing but I need debug mode locally. Is this a real failure?
For local development,
DEBUG=true is expected and the check is correctly flagging it. In production, DEBUG=true exposes /docs, /redoc, and full Python tracebacks in error responses — all of which are genuine security risks. The diagnostic treats this as a failure because the same script is used in both environments. On local machines you can ignore or suppress this check with --skip debug-mode once that flag is implemented.How do I run only the security checks without re-running Levels 1–3?
Use
--checks to filter by category:
python scripts/diagnose.py --level 4 --checks secrets,tokens,isolation,injection,disclosure,authorization,audit,ops
Category reference: secrets (config hardening + hygiene), tokens (auth integrity + replay), isolation (cross-user + AST), injection (SQLi, XSS, overflow), disclosure (info leak), authorization (admin + endpoint auth), audit (auth_events index), ops (kill switch). Note that some checks depend on a working DB connection — run Level 2 first if connectivity is in doubt.
The Kobayashi Maru drill returned PARTIAL — some probes didn't get 503. What happened?
The 30-second TTL cache in
MaintenanceMiddleware is flushed immediately when the flag is toggled, so the first probe should always see 503. A PARTIAL usually means one of three things: (1) a probe request arrived during the sub-millisecond window between the SQL upsert and flush_cache(), (2) a transient network error caused the probe to fail with a connection error rather than 503, or (3) a second instance of the server is running with its own in-memory cache that hasn't been flushed. Check the probe timeline in the full JSON for the exact timestamps and error messages.The Kobayashi Maru returned CRITICAL FAILURE — the system is still in maintenance. What do I do?
The restore SQL failed, which is rare (it only fails if the DB is completely unreachable during the restore phase). Run this immediately with an admin JWT:
curl -X POST /api/v1/admin/maintenance \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"enabled": false, "reason": "emergency restore after drill failure"}'
Alternatively, directly update the DB: UPDATE system_config SET value='false' WHERE key='maintenance_mode'; then restart the server to clear the in-memory cache.
Common failures & fixes
| Check | Typical cause | Fix |
|---|---|---|
| Database TCP timeout | IP not on Azure firewall allowlist | Azure Portal → PostgreSQL → Networking → add IP |
| Settings object fails to load | Missing or blank required ENV var | Compare .env against .env.example |
| Alembic revision behind head | New migration not applied | alembic upgrade f0a1b2c3d4e5 for system_config; alembic upgrade heads for all pending (chain has multiple heads — use explicit revision IDs when needed) |
| Blob canary write fails | Storage account key rotated | Update STORAGE_ACCOUNT_KEY in .env |
| Azure AI connectivity fails | API version mismatch or key expired | Verify AZURE_OPENAI_API_VERSION=2025-04-01-preview and key in Azure Portal |
| Stack trace in 500 responses | DEBUG=true in .env | Set DEBUG=false for production deployments |
| Cross-user isolation failure | Missing user_id filter in service query | Audit the flagged service file; add .where(Model.user_id == user_id) |
| bcrypt cost factor warning | Cost set to 12 (minimum) | Raise to 14 in security.py hash_password(); re-hash on next login |
| SECRET_KEY is a known-weak default | Placeholder value left in .env | Generate a new key: python -c "import secrets; print(secrets.token_hex(32))" and update SECRET_KEY in .env |
| Refresh token replay check fails | Stateless JWT refresh tokens — no server-side revocation store | Implement a token rotation table or Redis blacklist; second use of a refresh token must be rejected until revocation is in place |
| Kill switch check fails (no 503) | system_config table missing or MaintenanceMiddleware not loaded | Run alembic upgrade f0a1b2c3d4e5; verify app.add_middleware(MaintenanceMiddleware) is in main.py |
| ACA runtime context warning | Not running inside Azure Container Apps | Expected in local dev — warn not fail; only becomes meaningful in production |
| Kobayashi Maru: 503 on POST | KOBAYASHI_MARU_CODE not set in .env | Add KOBAYASHI_MARU_CODE=WORD1 WORD2 WORD3 WORD4 WORD5 WORD6 with six NATO phonetic words; restart server |
Adding a check
Each check is an async function decorated with @check in one of the five
module files under app/diagnostics/checks/. The decorator registers it at the
right level automatically.
# app/diagnostics/checks/functional.py
from app.diagnostics.runner import check, CheckFailed, CheckWarning
@check(level=3, category="db", name="users table schema")
async def users_table_schema() -> str:
"""Verify the users table columns match the SQLAlchemy model."""
# hard failure — affects exit code
if missing_columns:
raise CheckFailed(f"missing columns: {missing_columns}")
# soft warning — shown but exit code unaffected
if deprecated_columns:
raise CheckWarning(f"deprecated columns still present: {deprecated_columns}")
# pass — return a short human-readable summary string
return f"all {len(expected)} columns present"
Return / raise conventions:
return "some message"— check passed; message shown in outputraise CheckFailed("reason")— hard failure; increments exit coderaise CheckWarning("reason")— soft warning; displayed but exit code unaffected
The name parameter overrides the function name in output (optional but
recommended for readability). The category value appears in the
--checks filter.
Keep checks focused on one thing. A check that does five things is five checks that happened to share a function.