> For the complete documentation index, see [llms.txt](https://docs.digit.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digit.org/complaints-management/release-notes/migration-guide.md).

# Migration Guide

Steps required to upgrade from the v2.11 deployment to v2.12

## 1. Overview

This guide is for an operator with a live v2.11 tenant (data, users, complaints already in production) who wants to move that deployment to v2.12-beta. If you are standing up a brand-new tenant instead, most of the "manual re-seed" items below do not apply — a fresh install auto-seeds the new master data.

## 2. Before You Start Migration

* **Full database backup** of every service schema (pgr-services, egov-user, novu-bridge if in use, etc.) before running any Flyway migration. Several new migrations do `DROP MATERIALIZED VIEW ... CASCADE` and recreate — safe by design, but back up first.
* **Back up `/opt/digit/.openbao/init.json`** immediately after your first v2.12-beta deploy, out-of-band (e.g. to a password manager or offline vault). OpenBao is initialised with a single Shamir key (`secret_shares:1, secret_threshold:1`) and this file is the only copy of the root token + unseal key. There is no recovery path if it is lost.
* If you use the Ansible deploy path for the first time (moving off a hand-rolled v2.11 compose setup), read `local-setup/ansible/runbooks/01-openbao.md` before proceeding.

## 3. Change Overview

The below-listed breaking changes require manual action.

#### 3.1 Complaint classification: ServiceDefs → ComplaintHierarchy

> **Action required:** Hand-migrate every existing `RAINMAKER-PGR.ServiceDefs` row into the new two-master model — `ComplaintHierarchyDefinition` (the hierarchy shape) + `ComplaintHierarchy` (the actual category/sub-type tree). No automated backfill ships in this repo.

* Keep each leaf node's `code` identical to the old `serviceCode` — already-filed complaints reference this value and remain valid as long as the code is reused.
* Verify `Workflow.BusinessServiceMasterConfig` has a `PGR` row (`active:true, isStatelevel:true`) — required for escalation/notification-routing features that key off `businessService`.

#### 3.2 Mobile validation: UserValidation → MobileNumberValidation

> **Action required:** `common-masters.UserValidation` is gone. Create/adjust a `common-masters.MobileNumberValidation` record for your tenant's country before upgrading `egov-user`, **and** explicitly set `default: true` on exactly one record — regardless of country.

* New shape: `{ countryCode, mobileNumberRegex, default }`, unique on `countryCode`, plus the generic MDMS-v2 `isActive` flag. Name, email, and postal-code rules live in the separate `common-masters.FormValidations` master (one row per `fieldType` with a `regex`) — see [Section 3.7](#id-3.7-form-validations-and-reopen-window-post-beta).
* Only one record ships by default, pinned to a single calling code and national-number regex (`+254`, `^0?[17][0-9]{8}$`) — and it ships with **`default: false`**. If your tenant's numbering format differs, create your own record with the right `countryCode` and `mobileNumberRegex`.
* **Set `default: true` on exactly one record.** The real consumers are `egov-user` (backend validation) and the PGR frontend `useMobileValidation` hook — the latter falls back silently (to the first active record, or a hardcoded `+91`/10-digit pattern) if nothing is flagged as default, so an unconfigured tenant on a different numbering format risks quietly validating against the wrong pattern. (`novu-bridge` has a similar-looking MDMS lookup with no fallback, but it's only reachable from a pass-through test/debug endpoint, not the real WhatsApp delivery path — see the companion changelog Section 1.4 for detail. Real WhatsApp delivery relies on the complaint's own `service.citizen.countryCode`, not this master.)
* **Bump `egov-user` too** — the old image has India's mobile-number rule hardcoded in Java; without an upgrade it never reads the `mobileNumberRegex` you just set above. Use `egovio/egov-user:2.12-87e13fe` (the 2.12 core-services merge, what `local-setup/docker-compose.egov-digit.yaml` now pins) or `registry.preview.egov.theflywheel.in/egovio/egov-user:1044-preview` (the Ansible default, `EGOV_USER_IMAGE` in `digit.env.j2`) — don't pick a tag yourself. Both also include a tenant-aware encryption fix (Digit-Core#1044) and a `SafeHtmlValidator` regression fix that otherwise blocks citizen registration (CCRS#771). **Avoid the plain `egovio/egov-user:master-e22c7c5` Docker Hub tag** — it lacks all three and is only meant as a preview-registry-unreachable fallback.

#### 3.3 Novu-bridge behaviour changes (if you already run it)

* **Default notification channel changed from WhatsApp to SMS.** If your `.env` did not explicitly pin `novu.bridge.channel`, review what channel you actually want after upgrading.
* **WhatsApp is now off by default** at the channel-enablement level too (`novu.bridge.channels.enabled` defaults to `SMS,EMAIL`) — set it explicitly to include `WHATSAPP` if you rely on it, or events will persist as `SKIPPED/NB_NO_PROVIDER`.
* Removed properties `novu.bridge.max.retries` and the `novu.bridge.config.host`/`.resolve.path`/`.search.path` trio — delete these from any custom `.env`/property overrides; they no longer do anything.
* Run the new out-of-order Flyway migration `V20260701000000__extend_dispatch_unique_key.sql` with `spring.flyway.out-of-order=true` set (already the repo default) — do not edit the older checksummed migrations to work around ordering.

#### 3.4 pgr-services default boundary host changed

> **Check your override:** `egov.boundary.host` default changed from `http://localhost:8081` to `http://boundary-service.egov:8080/` (in-cluster service name). If you already set this explicitly you are unaffected; if you relied on the old default, set it explicitly to your actual boundary-service address.

#### 3.5 RBAC / security-policy re-seed

> **Action required:** default-data-handler seeding is create-only — an already-live tenant will not automatically pick up the new `CMS_SCREENING_OFFICER` role/role-actions or the expanded `DataSecurity.SecurityPolicy` PII-visibility grants. Re-run the relevant seed step manually, or apply the equivalent MDMS records by hand, and review the new PII-visibility defaults before applying them to a tenant with its own custom security policy.

#### 3.6 Dashboard analytics now fail closed on missing HRMS department

> **Check your HRMS data:** dashboard/analytics rows are now scoped by jurisdiction (`boundary_path` prefix) and by the viewing employee's department, both resolved from HRMS — and the resolver fails closed. **Any HRMS employee with no department set will see structurally empty dashboards after upgrade**, even if they previously saw data. Audit department assignments for employees who use the dashboard before or immediately after upgrading (`dss.DashboardConfig.departmentScoping` can disable the department axis tenant-wide if needed).

#### 3.7 Form validations & reopen window (post-beta)

> **Action required (validation):** the new `common-masters.FormValidations` master (one row per `fieldType`: `postalCode`, `name`, `email`, each with a `regex`) is now the primary per-tenant rule and **outranks** the host\_vars `core_postal_configs` pattern in DIGIT Studio and all PGR create-complaint flows. Fresh tenants get a default 5-digit `postalCode` row seeded automatically; **existing tenants must seed the rows manually** (default-data-handler seeding is create-only).

* host\_vars shape change: `core_postal_configs` lost `postalCodeLength` and `postalCodeErrorMessage` — `postalCodePattern` is the only remaining knob, and the localised error message is derived from it. Delete the two removed keys from your tenant's host\_vars.

> **Check your reopen window:** `RAINMAKER-PGR.UIConstants.REOPENSLA` is now actually read — previously a hardcoded 1-hour default won regardless of this value — and the deadline is now enforced server-side too. New tenants ship 259200000 ms (72 hours); **existing tenants keep whatever they already have** (typically 432000000 = 5 days), so set `REOPENSLA` explicitly if 72 hours is what you want.
>
> **Unset `time-before-closing-complaint` in your deployment values.** It maps to `PGR_COMPLAIN_IDLE_TIME`, which used to be the enforced window and is now only a fallback. Leaving it set pins one window server-side while the UI honours the per-tenant `REOPENSLA` — the two disagree and citizens get a "Complaint is closed" error on a REOPEN button the UI still offered. Refer 5.2 of the config changelog.

### 4. New Infrastructure That's Always On After Upgrade

These come with every v2.12-beta deployment automatically — there's no setting to turn them off, whether or not you actually use the features behind them:

* **OpenTelemetry agent** — Every Java service now loads a monitoring agent (`./otel/opentelemetry-javaagent.jar`) at startup. Before running `docker compose up`, you must run `local-setup/otel/download-agent.sh` to fetch it (pinned to version 2.11.0). If you skip this step, every Java container will fail to start.
* **Full observability stack** — Tools for tracing, logging, and metrics (otel-collector, Tempo, Promtail+Loki, Grafana+Prometheus) now start automatically with every deployment. Plan for the extra RAM, CPU, and disk this uses, and make a note of the new loopback ports it opens (Loki 13100, Prometheus 19090, Tempo 13200, OTel-collector 14317/14318/13133, OpenBao 18200).
* **Dashboard performance tracking** — Two new public routes (`/otel/v1/metrics` and `/otel/v1/logs`) let the browser report how fast the dashboard is loading. If you don't want these extra public routes, turn this off with `dashboard_metrics_enabled: false` (Ansible) or `DASHBOARD_METRICS_ENABLED=false` (globalConfigs). See `docs/observability/dashboard-metrics.md` for details on what it tracks.
* ships two new **public-facing** Kong ingest routes, `/otel/v1/metrics` and `/otel/v1/logs`, so the browser can report metrics directly. Set `dashboard_metrics_enabled: false` (ansible) / `DASHBOARD_METRICS_ENABLED=false` (globalConfigs) to turn it off if you don't want the extra public routes. See `docs/observability/dashboard-metrics.md` for what it measures and how to read it (it now covers both the client and server sides).
* **OpenBao** — see the backup note in [Section 2](#id-2.-before-you-start-migration). Initialises and auto-unseals on every deploy run.
* **audit-service, db-migrations, hrms-prereq-gate, user-seed** — new always-on compose services with no profile gate.
* **egov-enc-service dependency** — pgr-services now calls this service for PII encryption on every request path that touches it; it must be deployed and reachable, or those calls fail.

Two related items are **not** in the same category — they are ordinary Spring Boot flags, default-enabled rather than unconditional, and can be turned off:

* **PGR escalation scheduler and dashboard MV refresh** — both default-enabled (`pgr.escalation.enabled=true`, `pgr.dashboard.refresh.enabled=true`) but genuinely toggleable; setting either to `false` disables that scheduler. The escalation scheduler needs the `pgr-escalation-events` Kafka topic to exist while enabled.

## 5. Required Database Migrations

Run all pending Flyway migrations for pgr-services (and novu-bridge, xstate-chatbot if in use) as part of the upgrade. They must run in order — several `DROP MATERIALIZED VIEW ... CASCADE` and recreate later in the sequence:

<table><thead><tr><th width="87.7109375">Order</th><th width="453.7109375">Migration</th><th>Service</th></tr></thead><tbody><tr><td>1</td><td><code>V20260422000000__create_dashboard_mvs.sql</code></td><td>pgr-services</td></tr><tr><td>2</td><td><code>V20260608000000__create_v2_grain_mvs.sql</code></td><td>pgr-services</td></tr><tr><td>3</td><td><code>V20260609000000__add_assignment_routing_flags_to_facts.sql</code></td><td>pgr-services</td></tr><tr><td>4</td><td><code>V20260621000000__add_extended_attributes.sql</code></td><td>pgr-services</td></tr><tr><td>5</td><td><code>V20260623120000__open_complaint_age_buckets.sql</code></td><td>pgr-services</td></tr><tr><td>6</td><td><code>V20260629000000__grain_scope_columns.sql</code></td><td>pgr-services</td></tr><tr><td>7</td><td><code>V20260708000000__sla_and_hierarchy_grains.sql</code></td><td>pgr-services</td></tr><tr><td>8</td><td><code>V20260715000000__create_hrms_projection.sql</code></td><td>pgr-services</td></tr><tr><td>9</td><td><code>V20260716000000__hier_path_null_on_dotted_node_codes.sql</code></td><td>pgr-services</td></tr><tr><td>10</td><td><code>V20260717000000__hier_path_null_on_dotted_parent_codes.sql</code></td><td>pgr-services</td></tr><tr><td>11</td><td><code>V20260731000000__repoint_grain_mvs_to_complainthierarchy.sql</code></td><td>pgr-services</td></tr><tr><td>—</td><td><code>V20260701000000__extend_dispatch_unique_key.sql</code> (out-of-order)</td><td>novu-bridge, if in use</td></tr><tr><td>—</td><td><code>V20260505000000__chat.sql</code> (new DB)</td><td>xstate-chatbot, if in use</td></tr></tbody></table>

*Before running these in production, check the new `flyway-dump-alignment` CI gate's logic (`.github/workflows/flyway-dump-alignment.yml`) — it exists specifically to catch a `SPRING_FLYWAY_TABLE` / seeded-dump mismatch that makes Flyway try to re-run already-applied migrations and crash with `42P07 relation already exists`.*

## 6. Optional Features&#x20;

The features below are optional - enable only what is required:

<table><thead><tr><th width="171.46484375">Feature</th><th>Flag(s)</th><th>Notes</th></tr></thead><tbody><tr><td>Config-driven PGR notifications (SMS/WhatsApp/Email)</td><td><code>pgr.notification.config.driven=true</code> + your own approved WhatsApp Content templates persisted into <code>RAINMAKER-PGR.NotificationProviderTemplate</code> (via Configurator's <strong>Sync WhatsApp templates</strong> UI or <code>local-setup/scripts/persist-provider-templates.py</code>)</td><td>Do <strong>not</strong> rely on <code>local-setup/db/notif-mdms-seed/seed.sh</code> — it seeds <code>TemplateBinding</code>/<code>ProviderDetail</code>, which the config-driven WhatsApp path no longer reads. The seeded reference Content SIDs are Twilio-account- and Meta-approval-bound; you must author and get your own approved on your own Twilio account. See <a href="#id-3.3-novu-bridge-behaviour-changes-if-you-already-run-it">Section 3.3</a> for channel defaults to review first</td></tr><tr><td>Supervisor dashboard</td><td>Seed <code>dss.DashboardConfig</code> (+ <code>dss.KpiDefinition</code> / <code>dss.DashboardPack</code>), setting <code>allowedRoles</code> to gate access</td><td>MV refresh is already default-enabled (<a href="#id-4.-new-infrastructure-thats-always-on-after-upgrade">Section 4</a>); this only controls route/card visibility</td></tr><tr><td>PGR Visibility V1 (My/All inbox tabs, reportee-scoped inbox)</td><td><code>pgr.visibility.enabled=true</code> / <code>PGR_VISIBILITY_ENABLED</code> + <code>RAINMAKER-PGR.InboxVisibilityConfig</code> record</td><td>Needs the new <code>eg_pgr_hrms_projection</code> table (migration #8) and HRMS Kafka topics</td></tr><tr><td>digit-ui-v2 citizen SPA</td><td><code>enable_digit_ui_v2</code></td><td>Serves alongside the existing citizen UI at <code>/citizen/</code></td></tr><tr><td>Elasticsearch-backed inbox v2</td><td><code>enable_search_stack</code></td><td>~3GB additional RAM</td></tr><tr><td>Real OTP delivery</td><td><code>enable_otp_services</code></td><td>Kong mocks a fixed OTP otherwise</td></tr><tr><td>Turbopass / self-hosted Overpass</td><td><code>enable_turbopass</code> / <code>enable_overpass</code></td><td>OSM-based location autocomplete for Configurator</td></tr></tbody></table>

## 7. Steps

Follow the steps below to upgrade:

* Backup all service databases and, once deployed, the OpenBao `init.json` ([Section 2](#id-2.-before-you-start-migration)).
* Create/verify country-specific master data before flipping any validation-related service: `common-masters.MobileNumberValidation` for your locale ([Section 3.2](#id-3.2-mobile-validation-uservalidation-mobilenumbervalidation)), `core_postal_configs`/`core_mobile_configs` in your tenant's `host_vars/<tenant>.yml`.
* Migrate `ServiceDefs` → `ComplaintHierarchyDefinition` + `ComplaintHierarchy` for your tenant's complaint types ([Section 3.1](#id-3.1-complaint-classification-servicedefs-complainthierarchy)), and confirm the `Workflow.BusinessServiceMasterConfig` `PGR` row exists.
* Re-seed / manually apply the new RBAC role, role-actions, and security-policy rows ([Section 3.5](#id-3.5-rbac-security-policy-re-seed)).
* Run `local-setup/otel/download-agent.sh` before bringing up the new Compose stack ([Section 4](#id-4.-new-infrastructure-thats-always-on-after-upgrade)).
* Deploy v2.12-beta (Ansible: populate `host_vars/<tenant>.yml` from `_example.yml`, run `deploy.sh <tenant>`; Compose: use `docker-compose.egov-digit.yaml`).
* Let Flyway apply the migrations in [Section 5](#id-5.-required-database-migrations) (in order); verify with the app logs / `flyway_schema_history` table that all listed migrations show `success=true`.
* Review `novu-bridge` channel defaults if you already used it ([Section 3.3](#id-3.3-novu-bridge-behaviour-changes-if-you-already-run-it)) before assuming WhatsApp still works unchanged.
* Opt into any features from [Section 6](#id-6.-optional-features) as needed.
* Run through the [Section 8](#id-8.-verification-checklist) verification checklist.

## 8. Verification Checklist

Verify the following post-upgrade:

* [x] &#x20;Complaint create/assign/resolve/escalate flow works end-to-end for at least one complaint type per (migrated) hierarchy branch.
* [x] &#x20;Employee inbox loads and paginates; if Visibility V1 is enabled, the My/All tabs return the expected scoped results.
* [x] &#x20;Mobile-number entry accepts your tenant's real-number format on both the citizen and employee create-complaint forms.
* [x] &#x20;Dashboard loads without errors and KPI tiles show non-null values (confirms the new materialised views populated correctly). If tiles are unexpectedly empty for a given employee, check their HRMS department is set (Section 1.3 of the companion changelog — the new scoping fails closed).
* [x] &#x20;If notifications are enabled: `drive-test-complaint.py` (local-setup/scripts) completes and a real SMS/WhatsApp message is received using your **own** approved Content templates, not the seeded reference SIDs.
* [x] &#x20;Grafana/Prometheus/Loki/Tempo are reachable on their new ports and receiving data from at least one Java service; if dashboard render-lag metrics are enabled, see `docs/observability/dashboard-metrics.md` for how to read them.
* [x] &#x20;The backed-up OpenBao `init.json` matches what is currently on disk at `/opt/digit/.openbao/init.json`.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.digit.org/complaints-management/release-notes/migration-guide.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.