> For the complete documentation index, see [llms.txt](https://docs.digit.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digit.org/complaints-management/deploy/operations-handbook/cheat-sheet.md).

# Cheat Sheet

Print this page and pin it in the alert channel for quick reference

### URLs

<table data-header-hidden><thead><tr><th width="218.3359375"></th><th></th></tr></thead><tbody><tr><td>Health dashboard</td><td><code>https://&#x3C;your-domain>/status/</code> — no login</td></tr><tr><td>Grafana</td><td><code>https://&#x3C;your-domain>/grafana/</code> — <strong>login required</strong>, your own account (admin creates it, Editor role)</td></tr><tr><td>Logs</td><td><code>https://&#x3C;your-domain>/grafana/d/digit-loki-logs/</code></td></tr><tr><td>Service metrics</td><td><code>https://&#x3C;your-domain>/grafana/d/digit-jvm/</code></td></tr></tbody></table>

### The other five dashboards (L2)

<table><thead><tr><th width="383.453125">Question</th><th>Dashboard</th></tr></thead><tbody><tr><td>Is the server out of disk / RAM?</td><td><code>d/node-exporter-full/</code></td></tr><tr><td>Is the database the bottleneck?</td><td><code>d/postgres-database/</code></td></tr><tr><td>Gateway, or the service behind it?</td><td><code>d/kong-gateway/</code> → <strong>Latencies</strong> row</td></tr><tr><td>Which pipeline is stuck?</td><td><code>d/kafka-consumer-lag/</code></td></tr><tr><td>Does the broker have disk?</td><td><code>d/redpanda-broker/</code> → <strong>storage</strong> row</td></tr><tr><td>Why is the supervisor dashboard slow?</td><td><code>d/pgr-analytics/</code></td></tr></tbody></table>

What each panel means: [**dashboards**](/complaints-management/deploy/operations-handbook/the-grafana-dashboards.md)

### The ladder

Steps 0–4 are the [first-response checklist ](/complaints-management/deploy/operations-handbook/l1-first-response.md)— browser only, nothing changes state. Step 5 onward is [diagnosis](/complaints-management/deploy/operations-handbook/l2-diagnosis.md).

```
 0. DETAILS   What action, what happened instead, when, which city, complaint number.
    │
    ▼
 1. SCOPE     Private window + second account + different network.
    │         Works for you? → their machine.  Fails for you too? → continue.
    ▼
 2. HEALTH    /status/     Any red tile? Screenshot it NOW — this page keeps no history.
    │                      Infrastructure red → stop, escalate.
    ▼
 3. RESTARTS  Grafana → DIGIT JVM Services → "OOM events (current range)" above 0?
    │         Then Explore → Loki, the restart query below.
    │         Same service repeating = stuck restarting. Mind the maintenance window.
    │         Record what you saw — reading the heap graphs is L2's job.
    ▼
 4. ERRORS    Grafana → DIGIT — Logs (Loki)
    │         Time range = 10 min BEFORE it started. level = ERROR.
    │         Copy the OLDEST error, not the newest.
    │
    ├──────▶  Known issue? → known-issues.md, resolve, log it.
    └──────▶  Otherwise → hand over Part A of incident-report.md.
    ▼
 5. DEEPER    Layer model, slow-vs-failing, DB pool, Kafka lag, traces  → l2-diagnosis.md
 6. HOST      Grafana → Node Exporter Full   (CPU / RAM / disk)
              Empty dashboard = host metrics not being collected. Not a fault — tell L2.
```

### Escalate immediately; don't finish the ladder

* Infrastructure red on the health dashboard
* Nobody can log in, or the site doesn't load
* No complaint can be filed by anyone
* "No space left on device" anywhere

Screenshot the health dashboard first, then hand over.

### Queries worth memorising

Grafana → **Explore** → **Loki** (needs `logs` level or higher — see below):

```
# Is anything stuck restarting? (one line per service boot)
{compose_project="digit", compose_service!="loki"} |~ `Started .+Application in`

# Which service is producing the most errors right now?
sum by (compose_service) (count_over_time({compose_project="digit"} |~ `(?i)error|exception` [5m]))
```

Grafana → **Explore** → **Prometheus** (works at every observability level):

```
# Which collectors are reporting? Expect: otel-collector, node, postgres-exporter, redpanda, kong
up

# Which service died in the last 15 minutes
count by (service_name) (jvm_thread_count offset 15m) unless count by (service_name) (jvm_thread_count)
```

> **"No data" usually means zero, not broken.** On error, deadlock and lag queries a healthy deployment returns nothing. `NaN` on a p95 means no requests in that window — widen it.

### This deployment's observability level

The level decides which of the above you actually have. **Fill it in.**

| Level                | Logs (Loki) | Traces (Tempo) |
| -------------------- | ----------- | -------------- |
| `metrics`            | ✗           | ✗              |
| `logs`               | ✓           | ✗              |
| `traces` *(default)* | ✓           | ✓              |

**Ours is: \_\_\_\_\_\_\_\_\_\_\_\_\_\_** ← ask L2 once, write it here

On `metrics` there is no log search at all: Step 4 of the checklist does not apply, and the two LogQL queries above will not run. That is a deployment decision, not a fault.

### Evidence expiry log

<table><thead><tr><th width="240.9296875">Traces</th><th>24 hours</th></tr></thead><tbody><tr><td><strong>Logs</strong></td><td><strong>72 hours</strong></td></tr><tr><td><strong>Metrics</strong></td><td><strong>15 days</strong></td></tr><tr><td><strong>Health-check history</strong></td><td>survives restarts (SQLite); ~100 results / 50 events per endpoint</td></tr></tbody></table>

### Never run

```
docker compose down -v            → deletes the database and every volume
docker system prune -a --volumes  → same
docker compose up -d              → without service names, re-seeds master data
```

### Capture before you restart

A restart usually clears the symptom and the logs together.

```
sudo docker logs --tail 500 --timestamps <container> > /tmp/evidence.log
```

### Quick Reference Pages

<table><thead><tr><th width="280.0078125">Page Links</th><th></th></tr></thead><tbody><tr><td>L1 checklist</td><td><a href="/complaints-management/deploy/operations-handbook/l1-first-response.md">L1- first response</a></td></tr><tr><td>L2 diagnosis</td><td><a href="/complaints-management/deploy/operations-handbook/l2-diagnosis.md">L2 - diagnosis</a></td></tr><tr><td>Known issues</td><td><a href="/complaints-management/deploy/operations-handbook/known-issues.md">Known issues</a></td></tr><tr><td>Report template</td><td><a href="/complaints-management/deploy/operations-handbook/incident-report.md">Incident report</a></td></tr><tr><td>Lookup / glossary</td><td><a href="/complaints-management/deploy/operations-handbook/reference.md">Reference</a></td></tr></tbody></table>

### Fill these in for your deployment

| Domain                                              |   |
| --------------------------------------------------- | - |
| Grafana account created by (admin)                  |   |
| Observability level (`metrics` / `logs` / `traces`) |   |
| L2 contact                                          |   |
| Escalation channel                                  |   |
| Vendor escalation                                   |   |
| Deployment / maintenance window                     |   |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.digit.org/complaints-management/deploy/operations-handbook/cheat-sheet.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.