> For the complete documentation index, see [llms.txt](https://docs.digit.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digit.org/health/release-notes/performance-and-benchmarking.md).

# Performance & Benchmarking

## Overview

Campaign creation and boundary ingestion were benchmarked on the **demo** environment on **28 July 2026** (campaign creation) and **5 August 2026** (parallel boundary throughput), across increasing data volumes.

### Campaign creation at scale

Three campaigns were run back-to-back, each creating projects, facilities and users in parallel, followed by resource, facility and staff mapping reconciliation.

| Task                  | 5K / 5K / 5K     | 10K / 10K / 10K         | 20K / 10K / 20K          |
| --------------------- | ---------------- | ----------------------- | ------------------------ |
| Project creation      | 5 min 53 sec     | 10 min 43 sec           | 16 min 46 sec            |
| Facility creation     | 2 min 6 sec      | 3 min 2 sec             | 8 min 59 sec             |
| User creation         | 3 min 30 sec     | 9 min 21 sec (3 failed) | 11 min 51 sec (0 failed) |
| Resource mapping      | 54 sec           | 1 min 14 sec            | 2 min 36 sec             |
| Facility mapping      | 1 min 17 sec     | 1 min 30 sec            | 2 min 4 sec              |
| Staff mapping         | 1 min 32 sec     | 1 min 43 sec            | 3 min 30 sec             |
| Credential generation | 10 sec           | 33 sec                  | 37 sec                   |
| **Total**             | **8 min 36 sec** | **14 min 50 sec**       | **24 min 21 sec**        |

Column headings are boundary/facility/user counts. Note that the facility count deliberately stays at 10,000 between the second and third runs while boundaries and users double — this isolates whether facility creation time is driven by facility count alone or by overall campaign load.

**Scaling behaviour by phase:**

| Phase             | 5K → 10K                  | 10K → 20K                 | Note                                                 |
| ----------------- | ------------------------- | ------------------------- | ---------------------------------------------------- |
| Project creation  | 1.82× (for 2× data)       | 1.56× (for 2× data)       | Sub-linear both times — the healthiest phase         |
| Facility creation | 1.45× (for 2× facilities) | 2.96× (for 1× facilities) | Dispatch stall behind Project Level 6 — see below    |
| User creation     | 2.68× (for 2× users)      | 1.27× (for 2× users)      | The 10K run's 3 failures likely added retry overhead |
| Resource mapping  | 1.36×                     | 2.11×                     | Growing faster than data volume at 20K               |
| Facility mapping  | 1.17×                     | 1.38×                     | Relatively stable                                    |
| Staff mapping     | 1.12×                     | 2.04×                     | Growing faster than data volume at 20K               |

### Boundary ingestion

End-to-end boundary creation, English-only, on the demo environment. Phases are code generation → relationship persistence → localisation upsert.

| Boundaries | Code generation | Relationship persistence | Localisation upsert | Total        |
| ---------- | --------------- | ------------------------ | ------------------- | ------------ |
| 5,000      | 2.8 sec         | 55.6 sec                 | 39.4 sec            | 1 min 38 sec |
| 10,000     | 3.9 sec         | 73.8 sec                 | 42.7 sec            | 2 min 0 sec  |
| 20,000     | 5.4 sec         | 141.5 sec                | 46.8 sec            | 3 min 14 sec |

**Localisation overhead.** Additional languages add proportional work to the upsert phase, and only to that phase — code generation and relationship persistence are unaffected by language count. A three-language campaign (English, French, Portuguese) takes 2.8–3.1× the upsert time of English-only: 111.3 sec versus 39.4 sec at 5,000 boundaries, 119.4 versus 42.7 at 10,000, and 144.6 versus 46.8 at 20,000. On the English-only path, localisation upsert completes in 39–47 sec at every scale.

**Parallel bulk-relationship throughput.** Three hierarchies of 5,000 community leaves each were ingested concurrently, with leaves pushed to `/boundary-relationships/bulk/_create` in 50 chunks of 100 (the service cap).

| Run      | Leaves | Chunks   | Leaf-ingest | Total        | Leaves/sec | Failed chunks |
| -------- | ------ | -------- | ----------- | ------------ | ---------- | ------------- |
| Stream A | 5,000  | 50 × 100 | 56.6 sec    | 2 min 57 sec | 88.4       | 0             |
| Stream B | 5,000  | 50 × 100 | 56.6 sec    | 2 min 57 sec | 88.4       | 0             |
| Stream C | 5,000  | 50 × 100 | 57.4 sec    | 2 min 58 sec | 87.1       | 0             |

All three streams completed with zero failed chunks, each sustaining approximately 88 relationships per second — roughly 264 per second in aggregate — with no contention penalty across parallel hierarchies. Total time also includes hierarchy definition, approximately 5,100 boundary entities, and ancestor relationships.

**Cluster comparison.** The same English-only test was run on both `demo` and `hcm-demo-new`. The `demo` cluster is faster at every phase and scale (relationship persistence at 5,000 is on par) and is the only cluster that completes the 20,000-boundary run, finishing in 3 min 14 sec. Code generation is 3.4× faster at 5,000, 8.5× at 10,000, and 25× at 20,000; localisation upsert is 3.6–4.0× faster. On `hcm-demo-new` the 20,000 run does not complete: code generation finishes in approximately 137 sec, but the subsequent boundary lookup against the full 20,000-code list breaks the connection, identically on retry. This confirms `demo` as the production-ready cluster for boundary operations at scale.

### Excel template generation, validation and size

Template generation time by boundary count, cold (first request) versus warm (cached):

| Boundaries in hierarchy | hcm-demo cold | hcm-demo warm | health-demo cold | health-demo warm |
| ----------------------- | ------------- | ------------- | ---------------- | ---------------- |
| 5,000                   | 0.70 sec      | 0.45 sec      | 1.68 sec         | 1.22 sec         |
| 10,000                  | 2.82 sec      | 0.42 sec      | 2.06 sec         | 1.59 sec         |
| 20,000                  | 2.47 sec      | 0.43 sec      | 6.42 sec         | 6.62 sec         |

Upload and sheet-validation time by data rows:

| Cluster     | Rows added | Pre-filled | Rows parsed | File (KB) | Upload | Validation | rows/sec |
| ----------- | ---------- | ---------- | ----------- | --------- | ------ | ---------- | -------- |
| health-demo | 5,000      | 5          | 5,005       | 285.9     | 0.34 s | 1.97 s     | 2,541    |
| health-demo | 10,000     | 5          | 10,005      | 428.8     | 0.34 s | 2.43 s     | 4,117    |
| health-demo | 20,000     | 5          | 20,005      | 710.4     | 0.60 s | 8.75 s     | 2,286    |
| hcm-demo    | 5,000      | 10,000     | 15,000      | 718.9     | 3.07 s | 4.97 s     | 3,018    |
| hcm-demo    | 10,000     | 10,000     | 20,000      | 860.1     | 2.43 s | 5.53 s     | 3,617    |
| hcm-demo    | 20,000     | 10,000     | 30,000      | 1,141.9   | 4.75 s | 7.71 s     | 3,891    |

Validation sustains approximately 2,300–4,100 rows per second; the health-demo 20,000-row set is the slowest per row at 8.75 sec for 20,005 rows.

**Template size.** The scaffold-less pre-filled template is **15.8–18.8× smaller** than the legacy blank-scaffold workbook, because it materialises only real rows instead of 5,000 empty rows per sheet. The legacy scaffold is 286.1 KB (3 sheets × 5,001 rows); the scaffold-less template is 18.1 KB where the hierarchy matches the campaign, and 15.2 KB where it does not (no boundary columns).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.digit.org/health/release-notes/performance-and-benchmarking.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.