> For the complete documentation index, see [llms.txt](https://docs.digit.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digit.org/complaints-management/deploy/configure/notifications/setup-notifications.md).

# Setup Notifications

Setting up and running notifications

## Overview

The operator guide: from nothing to a delivered SMS, WhatsApp or email, then day-to-day operation and troubleshooting. Section 2 needs server access; everything else is in the Configurator under **Notifications**.

Upgrading a 2.12 deployment? Read migration.md first.

<table><thead><tr><th width="80.19921875">Step</th><th width="484.953125">What you do</th><th>Where</th></tr></thead><tbody><tr><td>1</td><td>Collect accounts and roles</td><td>—</td></tr><tr><td>2</td><td>Deploy the notification stack</td><td>Server</td></tr><tr><td>3</td><td>Add a provider and its credentials</td><td>Providers</td></tr><tr><td>4</td><td>Select the provider for a channel and switch it on</td><td>Channels</td></tr><tr><td>5</td><td>Review events, routing, templates; validate</td><td>Configure</td></tr><tr><td>6</td><td>Send a test, read the result</td><td>Providers, Logs</td></tr><tr><td>7</td><td>Checklist and troubleshooting</td><td>—</td></tr><tr><td>8</td><td>Deployment settings, receipts, per-channel server steps, new workspaces</td><td>Server</td></tr></tbody></table>

### 1. Before you start

**Accounts, per channel.** Channels are independent; set up one, two or all three.

<table><thead><tr><th width="147.421875">Channel</th><th>You need</th></tr></thead><tbody><tr><td>SMS</td><td>An account with Twilio, SMSCountry (legacy bulk API, panel login), or an Ozeki gateway you run. India: DLT-registered templates (§8.4).</td></tr><tr><td>WhatsApp</td><td>A Twilio account with a WhatsApp-enabled sender <strong>and Meta-approved message templates</strong>. Nothing sends without them — start approval early.</td></tr><tr><td>Email</td><td>An SMTP mailbox. Gmail / Microsoft 365 need an <strong>app password</strong>, not the account password.</td></tr></tbody></table>

**Citizen sign-in codes do not go through notifications yet.** In 2.20, citizens sign in through the Identity BFF, which sends its codes with its own sender (`http` to a gateway the deployment runs, or `log` for development); connecting it to the bridge's `messages/_send` is a follow-up. **SMS carries the codes of DIGIT's older OTP service**, where a deployment still uses it (`enable_otp_services: true`, which requires `enable_novu: true`; the deploy refuses otherwise): they go through the same SMS channel, provider and log. Switching SMS off stops that OTP login (rows read `SKIPPED / NB_NO_PROVIDER`, event name `CORE.SMS.OTP`); changing the SMS provider changes who sends those codes.

**Roles.** Two tiers, set at deploy time:

<table><thead><tr><th width="256.77734375">To</th><th width="314.140625">You need a role from</th><th>Default</th></tr></thead><tbody><tr><td>Use the screens: logs, providers, preferences; <strong>Check status</strong></td><td><code>novu_bridge_proxy_allowed_roles</code></td><td><code>EMPLOYEE,SUPERUSER,GRO,PGR_LME,MDMS_ADMIN</code></td></tr><tr><td>Create a provider, rotate its credentials, disable or delete it, <strong>Test</strong> it; <code>POST /dispatch/_dry-run</code> with <code>"send": true</code></td><td><code>novu_bridge_proxy_admin_roles</code>, <strong>held at a state that owns the providers</strong> (below)</td><td><code>SUPERUSER,MDMS_ADMIN,ACCOUNT_ADMIN</code></td></tr><tr><td>Preview one tenant's events: <code>POST /dispatch/_resolve</code>, <code>/dispatch/_dry-run</code> without <code>send</code> (what <code>migrate-notifications.py</code> uses for its before/after preview)</td><td><code>novu_bridge_proxy_admin_roles</code>, held at <strong>the event tenant's own state root</strong>, or at a state that owns the providers</td><td><code>SUPERUSER,MDMS_ADMIN,ACCOUNT_ADMIN</code></td></tr></tbody></table>

Without an admin role at a state tenant, those calls answer `403 NB_ADMIN_ROLE_REQUIRED`: a provider serves the whole deployment, so an admin role held only at a city (`ke.bomet`) does not count. The state must also **own** the deployment's providers: the deployment's `state_root` (novu-bridge reads it from `NOVU_BRIDGE_CORE_SMS_DEFAULT_TENANT`, which the deploy sets to `state_root` and checks on the running container), plus any state listed in `novu_bridge_provider_admin_tenants` (`NOVU_BRIDGE_PROVIDER_ADMIN_TENANTS`, comma-separated, default empty). An admin of any other root on the same box — a second state root or an onboarded workspace (#1999) — gets `403 NB_TENANT_NOT_ALLOWED` on the provider calls; list that root there if its admins should manage providers. That admin may still preview its own root's events, so `migrate-notifications.py plan/apply` for a root runs logged in at that root; only `--create-provider` / `--create-smscountry-provider` needs an owning state (see migration.md). With neither set, every provider call is refused (providers.md). An admin role also satisfies the first tier. Logs and the config-source report answer only for your own tenant — for a user at the state tenant, the state and its cities — and `403 NB_TENANT_NOT_ALLOWED` for any other. Editing Channels / Routing / Templates / Provider Templates is governed by the ordinary MDMS roles (`MDMS_ADMIN`, `ACCOUNT_ADMIN`, `SUPERUSER`); the Channels card's switches and provider choice, and **Sync WhatsApp templates**, are only offered to those roles. Every Configurator save — the Channels card and a provider's Disable/Delete included — runs the checker first, blocks on an error it would cause, and waits until the configuration it checks against has loaded.

### 2. Turn the stack on

1. In `local-setup/ansible/inventory/host_vars/mycity.yml` set `enable_novu: true` (`seed_notifications` follows it). Other settings: §8.1.
2. Deploy and wait for `failed=0`:

   ```bash
   cd local-setup/ansible
   ./deploy.sh mycity
   ```
3. Verify the Novu API key — a green recap is not enough, the provider bootstrap may fail without failing the deploy:

   ```bash
   export NOVU_BASE_URL='http://127.0.0.1:14002'
   export NOVU_API_KEY="$(sudo sed -n 's/^NOVU_API_KEY=//p' /opt/digit/.env | tail -1)"
   test -n "$NOVU_API_KEY" && test "$NOVU_API_KEY" != changeme
   curl -fsS -H "Authorization: ApiKey $NOVU_API_KEY" "$NOVU_BASE_URL/v1/integrations" >/dev/null
   ```

   Stop if any command fails: every provider operation goes through `novu-bridge`, which cannot work with an empty, placeholder or invalid key.

The deploy starts novu-bridge, Novu and digit-user-preferences-service; mints the Novu API key into `/opt/digit/.env`; creates the Novu workflows `complaints-sms`, `complaints-whatsapp`, `complaints-email`; creates the Kafka topics (kafka-events.md); grants the access-control actions (restarting `egov-accesscontrol` when it added any; a role the tenant does not have gets no grant and is listed as `ACL-ROLES-ABSENT`, and an action whose usual id is already another endpoint's at the tenant is created at a free id instead, `ACL-ID-COLLISION`: neither is a failure, migration.md) and then seeds the `NOTIFICATIONS.*` masters. A tenant with no channel rows gets one row per channel, **only if `novu_bridge_channels_enabled` lists it** (unset: all three off) and with no provider selected; a tenant that has rows keeps them (migration.md). It also syncs `local-setup/kong/kong.yml`; if you edit that file by hand, apply it with `sudo docker exec kong-gateway kong reload`.

> Never set `NOVU_BRIDGE_PROXY_AUTH_ENABLED=false` on a reachable deployment: Kong delegates authentication for these routes to novu-bridge, so provider creation and test-send become unauthenticated ([#1942](https://github.com/egovernments/Citizen-Complaint-Resolution-System/issues/1942)).

### 3. Add a provider

**Notifications → Providers → Add Provider**: pick the type, give a name, fill the credential fields, **Create Provider**.

A workspace with its own notification account (tenant-accounts.md; a banner on the page says so) adds providers to that account, and its own admins manage them. Everywhere else the providers are the deployment's shared ones, managed by the admins of the state that owns them.

<table><thead><tr><th width="165.94140625">Type</th><th width="135.40625">Channel</th><th>Fields</th></tr></thead><tbody><tr><td>Twilio SMS</td><td>SMS</td><td>Account SID (<code>AC…</code>), Auth token, From number (E.164, e.g. <code>+14155238886</code>)</td></tr><tr><td>Twilio WhatsApp</td><td>WHATSAPP</td><td>Account SID, Auth token, WhatsApp sender (<code>whatsapp:+14155238886</code>; the sandbox number works after <code>join &#x3C;code></code> from your handset)</td></tr><tr><td>Email (SMTP)</td><td>EMAIL</td><td>SMTP host, SMTP port (<code>587</code>), Username, Password (app password), From address (usually = username), From name, Use TLS on connect (port 465)</td></tr><tr><td>SMSCountry ¹</td><td>SMS</td><td>Panel username, Panel password, Registered sender id, Gateway URL (blank = standard bulk endpoint). Legacy bulk API only; a panel showing AuthKey/AuthToken is the unsupported REST API</td></tr><tr><td>Ozeki SMS Gateway ¹</td><td>SMS</td><td>HTTP API URL (e.g. <code>http://ozeki.example.org:9509/api?action=sendmsg</code>), Username (the gateway's HTTP API user), Password, Sender id (optional)</td></tr><tr><td>Jasmin SMS Gateway ¹</td><td>SMS</td><td>Send URL (e.g. <code>http://jasmin.example.org:1401/send</code>), Username, Password, Sender id (optional). Text outside the GSM alphabet (Amharic, emoji) goes as UCS-2: <strong>70 characters per SMS segment, not 160</strong></td></tr></tbody></table>

¹ These three are DIGIT's providers, mounted into Novu's worker by the deploy; there is nothing to set. The worker refuses to boot if they fail to load, with a `[digit-novu-providers]` error in `docker logs novu-worker`. Without them, the provider would save, **Test** and dispatch would report the message as sent, and every send would fail inside Novu (`Sms handler for provider … is not found` in Novu's activity feed). See providers.md for how they load and for the gateway specifics.

Email traps:

* Gmail / Microsoft 365 with 2FA reject the account password with `535-5.7.8 Username and Password not accepted`. Use a 16-character app password.
* **Use TLS on connect** means TLS from the first byte: tick it for port **465**, leave it unticked for **587** (STARTTLS). Ticked with 587 hangs or fails the handshake.

Credentials are stored **only in Novu**: never in MDMS, `/opt/digit/.env` or logs, and never returned to the browser.

Row actions on the Providers list:

<table><thead><tr><th width="220.7109375">Action</th><th>What it does</th></tr></thead><tbody><tr><td><strong>Check status</strong></td><td>Confirms the integration exists and is enabled. Proves there are <strong>no</strong> credentials of any type.</td></tr><tr><td><strong>Test</strong></td><td>Sends one real message (see §6). The only credential proof. Admin role at the state tenant, or at the workspace when it has its own account.</td></tr><tr><td><strong>Rotate credentials</strong></td><td>Asks for every field again — the store overwrites, it does not merge.</td></tr><tr><td><strong>Rename</strong></td><td>Display name only.</td></tr><tr><td><strong>Disable</strong> / <strong>Enable</strong></td><td>Switches the Novu integration off/on. <strong>Disable</strong> is guarded like <strong>Delete</strong>. With <code>NOVU_BRIDGE_DIGIT_WORKER_PROVIDERS=false</code>, an SMSCountry / Ozeki / Jasmin provider cannot be re-enabled, rotated or tested (<code>400 NB_PROVIDER_TYPE_UNAVAILABLE</code>); disabling, renaming and deleting still work.</td></tr><tr><td><strong>Delete</strong></td><td>Refused with <code>409 NB_PROVIDER_IN_USE</code> while a channel still selects it (checked in MDMS at that moment, for your state and every state this deployment has sent for), <strong>or while it is the last active integration on its Novu channel</strong> (<code>sms</code> carries SMS and WhatsApp, <code>email</code> Email) <strong>and some enabled channel on it has no provider selected</strong> — such a channel sends through Novu's default integration: a legacy channel row, a state running on <code>novu_bridge_channels_enabled</code>, or any tenant while channel policy is off. Also refused when those channel rows cannot be read — nothing is deleted; try again. Point the channel elsewhere (or add another active integration) first.</td></tr><tr><td><strong>Delivery workflows</strong></td><td>Read-only list of Novu workflows for the channel — plumbing, not message text.</td></tr></tbody></table>

The page also has **Sync WhatsApp templates** (§5.5). How the types work and how to add a new one: providers.md.

### 4. Switch the channel on

**Notifications → Channels** (MDMS `NOTIFICATIONS.Channel`, one row per SMS / WHATSAPP / EMAIL, held at the **state** tenant). Select the provider and use **Enable** / **Disable**. Only providers of that channel are offered. Changes apply within about a minute (60 s cache).

* **One provider per channel, per state.** Selecting another replaces it. No fan-out, no failover, no per-city provider.
* You must be logged in at the state tenant; scoped to a city, Enable/Disable are disabled ("You are scoped to … switch to the state tenant to change channel policy").
* A channel with no provider selected falls back to the deployment's env settings (§8.1); Validate flags it (`channel-needs-provider`).

Status card messages:

<table><thead><tr><th width="279.5703125">It says</th><th>Do this</th></tr></thead><tbody><tr><td>"… is on and delivering through <em>Name</em>"</td><td>Nothing.</td></tr><tr><td>"No channel policy row for …"</td><td>Switch the channel on so the choice is explicit.</td></tr><tr><td>"… is off. Every event on this channel is recorded SKIPPED / NB_NO_PROVIDER …"</td><td>Switch on once its provider exists.</td></tr><tr><td>"… is on but no provider is configured for it."</td><td>Add a provider. Until Novu holds an active integration for the channel (a workspace with its own notification account: in that account), every message on it is <code>SKIPPED / NB_PROVIDER_UNAVAILABLE</code> (never a false <code>SENT</code>).</td></tr><tr><td>"… is on but no provider is selected."</td><td>Select one.</td></tr><tr><td>"… selected provider <em>Name</em> no longer exists / is disabled / does not serve …"</td><td>Re-enable it or select another. Meanwhile every message is <code>SKIPPED / NB_PROVIDER_UNAVAILABLE</code> (never a false <code>SENT</code>). The same happens to an SMSCountry / Ozeki / Jasmin provider while <code>NOVU_BRIDGE_DIGIT_WORKER_PROVIDERS=false</code> (the Novu worker does not load DIGIT's providers — providers.md).</td></tr><tr><td>"… the Novu workflow complaints-… is missing"</td><td>Deployment job: re-run <code>./deploy.sh</code>.</td></tr></tbody></table>

### 5. Configure what is sent

A message goes out only when channel, routing, template and provider all line up; anything missing becomes a `SKIPPED` row on Logs with the reason.

<table><thead><tr><th width="183.765625">Screen</th><th width="307.5625">Holds</th><th>Stored in</th></tr></thead><tbody><tr><td>Providers</td><td>Gateway accounts and credentials</td><td>Novu</td></tr><tr><td>Channels</td><td>On/off and selected provider per channel</td><td><code>NOTIFICATIONS.Channel</code></td></tr><tr><td>Configure</td><td>Guided editor: pick a <strong>Module</strong>, edit routing + templates per event, <strong>Validate</strong>; OTP wording (§5.6)</td><td>— (OTP wording: localization)</td></tr><tr><td>Events</td><td>Events, their actors, placeholders and allowed channels. <strong>Read-only</strong></td><td><code>NOTIFICATIONS.EventCatalogue</code></td></tr><tr><td>Templates</td><td>Message text per event × audience × channel × locale</td><td><code>NOTIFICATIONS.Template</code></td></tr><tr><td>Routing</td><td>Who is told, per event × audience × channel</td><td><code>NOTIFICATIONS.Routing</code></td></tr><tr><td>Provider Templates (WhatsApp)</td><td>Approved WhatsApp template per routing key</td><td><code>NOTIFICATIONS.ProviderTemplate</code></td></tr><tr><td>Logs</td><td>Every attempt and its outcome</td><td><code>nb_dispatch_log</code></td></tr><tr><td>User Preferences</td><td>Each user's language and per-channel consent (read-only)</td><td>digit-user-preferences-service</td></tr></tbody></table>

Use **Configure** day to day; it edits a routing row and its templates together. The masters live at the state tenant. Legacy `RAINMAKER-PGR.Notification*` masters appear read-only under **Advanced** as "Legacy (PGR) …"; a tenant not yet copied shows the banner *"This tenant has not been migrated yet — shown read-only"* — see migration.md. The Configurator decides which namespace is live exactly as novu-bridge does: routing and templates from whether the tenant has any `NOTIFICATIONS.Routing` row (active or not), channel policy from whether it has channel rows. While a tenant is on its legacy masters, creating a raw `NOTIFICATIONS.Routing` or `NOTIFICATIONS.Channel` row is refused (`namespace-switch`, below).

Shipped defaults for complaints: 14 events, 24 routing rows (citizen on SMS/WhatsApp/email for APPLY, ASSIGN, REASSIGN, REJECT, RESOLVE, REOPEN; assignee on all three when assigned and when rated), 42 templates (24 `en_IN` + 18 `hi_IN`), 14 WhatsApp provider templates (7 events × `en_IN`/`hi_IN`, **belonging to the reference demo Twilio account — they will not work on yours**), 3 channel rows (off, unless `novu_bridge_channels_enabled` listed the channel when the tenant was seeded).

The default wording is deliberately neutral, so it reads right in any country and on a single-level workspace: no sender signature, no honorific, and only tokens every complaint can fill (the assignee is addressed as "Dear {emp\_name}", with no office or district line). Add your own signature, office name or helpline in the templates. Changing the defaults never rewrites a tenant's existing rows: a tenant seeded before keeps its wording until you edit it.

#### 5.1 Events

Events are declared by the producing module (CMS is generated from its workflow), named like `COMPLAINTS.WORKFLOW.ASSIGN.PENDINGATLME`. The Events screen is the authority for which actor names an audience may use and which `{tokens}` a template may use. You cannot create events here.

#### 5.2 Routing and audiences

One row per *(event, audience, channel)*: "when **X** happens, tell **audience** by **channel**".

<table><thead><tr><th width="207.57421875">Audience (Kind on Configure)</th><th width="194.6328125">Stored as</th><th>Means</th></tr></thead><tbody><tr><td>Actor on the event</td><td><code>ACTOR:&#x3C;name></code></td><td>A person the event names; complaints carry <code>citizen</code> and <code>assignee</code></td></tr><tr><td>Everyone with a role</td><td><code>ROLE:&#x3C;code></code></td><td>Every holder of the role in the tenant</td></tr><tr><td>Contacts on the event</td><td><code>EVENT_RECIPIENTS</code></td><td>Contacts carried on the event (account-less, e.g. OTP)</td></tr><tr><td>fallback chain</td><td><code>A|B</code></td><td>Try A; if it names nobody, try B. Added with <strong>+ add a fallback …</strong>, shown as <em>or, if empty:</em></td></tr></tbody></table>

`AUTO_ESCALATE` and `SYSTEM` are dropped (not people). Legacy bare names still work: `CITIZEN` = `ACTOR:citizen`, `EMPLOYEE` = `ACTOR:assignee`. A person reached through two audiences gets one message per channel.

#### 5.3 Templates and placeholders

One template per *(event, audience, channel, locale)*. A routing row without a template sends nothing.`NB_NO_TEMPLATE`). **Every routing row needs a `en_IN` template** — it is the fallback for recipients whose language has none.

* SMS: body only. 160 GSM-7 characters per segment (153 when split); one non-GSM character (curly quote, em dash, Devanagari, accented letters) switches the whole message to UCS-2: 70 per segment (67 when split).
* Email: subject + body; a blank subject becomes `Complaint <id>`.
* WhatsApp: the body is **not** sent — see §5.5.

Placeholders use **single braces**: `{id}`. `{{id}}` and `{ id }` are sent literally. A placeholder with no value is delivered **as its braces** (`{emp_name}`), except `{download_link}`, which is blanked. On WhatsApp, a missing value is an empty string.

Complaint placeholders:

| Placeholder                                             | Becomes                                            | Empty when                    |
| ------------------------------------------------------- | -------------------------------------------------- | ----------------------------- |
| `{id}`                                                  | Complaint number                                   | never                         |
| `{complaint_type}`                                      | Category, translated where possible                | never                         |
| `{status}`                                              | Current status, translated                         | before the first transition   |
| `{date}`                                                | Filing date, `dd/MM/yyyy`                          | never                         |
| `{additional_comments}`                                 | Employee's comment on this action                  | no comment                    |
| `{rating}`                                              | Citizen's rating 1–5                               | not rated yet                 |
| `{citizen_name}`                                        | Filer's name                                       | filed without a name          |
| `{download_link}`                                       | Short app link                                     | shortener unavailable         |
| `{ulb}`                                                 | City / district, translated                        | no district on the address    |
| `{ao_designation}`                                      | Assigning officer's designation label              | label not translated          |
| `{emp_name}` / `{emp_department}` / `{emp_designation}` | Current assignee's name / department / designation | not assigned yet (e.g. APPLY) |

Values are resolved once per event in one locale; the template text is chosen per recipient. Recipients get their profile's `preferredLanguage` where a template exists, else `en_IN`.

#### 5.4 Validation rules

**Configure → Validate** answers **All checks passed**, **Passed · N warning(s)** or **N error(s)**; **Show details** lists findings by rule id. Every save runs the same check on the configuration as it would be after the save: errors your change causes block it (*"This change cannot be saved until the following is fixed:"*); warnings and pre-existing errors on other rows do not. Removing the last template of an active routing row is refused.

<table><thead><tr><th width="162.6875">Rule</th><th width="150.54296875">Level</th><th>Meaning → fix</th></tr></thead><tbody><tr><td><code>audience-role-exists</code></td><td>error</td><td>Actor not declared by the event, or role code unknown → use one the Events screen lists / an existing role</td></tr><tr><td><code>audience-scheme</code></td><td>error</td><td>Unknown audience scheme (<code>NB_UNKNOWN_AUDIENCE_SCHEME</code>) → use <code>ACTOR:</code>, <code>ROLE:</code>, <code>EVENT_RECIPIENTS</code> or a chain</td></tr><tr><td><code>routing-has-template</code></td><td>error</td><td>Active routing row has no active <code>en_IN</code> template → add it or deactivate the row</td></tr><tr><td><code>channel-allowed</code></td><td>error</td><td>Channel not SMS / WHATSAPP / EMAIL</td></tr><tr><td><code>transition-exists</code></td><td>error</td><td>Event not in the catalogue → pick one from Events</td></tr><tr><td><code>channel-gateway-mismatch</code></td><td>error</td><td>Legacy direct gateway cannot carry the channel (<code>smscountry</code> is SMS only) → gateway <code>novu</code> + a provider</td></tr><tr><td><code>channel-needs-provider</code></td><td>error / warn</td><td>Enabled channel, no provider (error if routing uses it) → select one on Channels</td></tr><tr><td><code>channel-provider-missing</code></td><td>error / warn</td><td>Selected provider no longer exists → select another</td></tr><tr><td><code>channel-provider-inactive</code></td><td>error / warn</td><td>Selected provider disabled → re-enable or replace</td></tr><tr><td><code>placeholder-braces</code></td><td>error</td><td><code>{{id}}</code>, unclosed <code>{</code>, stray <code>}</code> → exactly <code>{id}</code></td></tr><tr><td><code>template-needs-body</code></td><td>error</td><td>Active template with empty body</td></tr><tr><td><code>whatsapp-variable-unmapped</code></td><td>error</td><td>Body placeholder missing from the provider template's <strong>Variables (ordered)</strong> → add it in the approved position or remove it</td></tr><tr><td><code>namespace-switch</code></td><td>error</td><td>A raw <code>NOTIFICATIONS.Routing</code> or <code>NOTIFICATIONS.Channel</code> create while the tenant is still served from its legacy masters: the first such row would stop every legacy route (or the legacy channel policy) at once → move the tenant with <code>migrate-notifications.py</code> (migration.md)</td></tr><tr><td><code>channel-in-event</code></td><td>warn</td><td>Channel not declared on the event's catalogue row → usually fix the routing row</td></tr><tr><td><code>no-orphan-template</code></td><td>warn</td><td>Template with no active routing row → harmless</td></tr><tr><td><code>non-notifiable-audience</code></td><td>warn</td><td><code>AUTO_ESCALATE</code> / <code>SYSTEM</code> never send</td></tr><tr><td><code>channel-enabled</code></td><td>warn</td><td>Routing on a channel that is off or has no row (expected on a fresh install)</td></tr><tr><td><code>unknown-token</code></td><td>warn</td><td><code>{token}</code> not declared by the event; ships literally</td></tr><tr><td><code>email-needs-subject</code></td><td>warn</td><td>No subject; <code>Complaint &#x3C;id></code> is used</td></tr><tr><td><code>email-subject-length</code></td><td>warn</td><td>Subject over 150 characters</td></tr><tr><td><code>sms-length</code></td><td>warn</td><td>Estimated over 3 segments. Estimate adds 12 characters per placeholder; the shipped Hindi bodies trigger it</td></tr><tr><td><code>whatsapp-needs-template</code></td><td>warn</td><td>WhatsApp routing without an approved provider template (<code>NB_TEMPLATE_NOT_APPROVED</code>)</td></tr><tr><td><code>whatsapp-variable-unfilled</code></td><td>warn</td><td>Provider template variable the event cannot fill; sent empty</td></tr></tbody></table>

Expected warnings on the shipped defaults: `channel-enabled` (channels ship off unless the allowlist named them), `whatsapp-needs-template` (the assignee WhatsApp rows have no approved template), `sms-length` (Hindi bodies).

#### 5.5 WhatsApp provider templates

WhatsApp sends an approved template (Twilio Content SID, `HX…`) plus ordered values, not your body. Without one, the message is `SKIPPED / NB_TEMPLATE_NOT_APPROVED`.

1. Get your own templates approved in Twilio, with names and ordered variables matching your messages.
2. **Providers → Sync WhatsApp templates**: it matches approved templates to routing rows; review and save the rows you want. They land on **Provider Templates (WhatsApp)**.
3. Check routing and templates on **Configure**, then trigger a real complaint transition.

#### 5.6 Changing the OTP wording

**Configure → Login and registration OTP (SMS)**, at the end of the page, edits the text of the SMS of **DIGIT's older OTP service** (`user-otp`), where a deployment still uses it (`enable_otp_services`). It does not change the codes of 2.20's citizen sign-in (the Identity BFF), which do not go through notifications yet, nor the fixed text of `messages/_send`. It changes the wording only: `user-otp` builds the SMS itself and novu-bridge delivers it, so on/off and provider stay on **Channels** (SMS, §4). The OTP has no event, routing or template row.

The text is three localisation messages in the module `egov-user`:

| Code                    | OTP type       | Built-in text, used when the language has no `egov-user` message    |
| ----------------------- | -------------- | ------------------------------------------------------------------- |
| `sms.login.otp.msg`     | login          | `Dear Citizen, Your Login OTP is %s.`                               |
| `sms.register.otp.msg`  | register       | `Dear Citizen, Your OTP to complete your DIGIT Registration is %s.` |
| `sms.pwd.reset.otp.msg` | password reset | `Dear Citizen, Your OTP for recovering password is %s.`             |

* **`%s` is the code** and must appear exactly once. Without it, the SMS goes out with no code; with two the OTP request fails. Write `%%` for a literal `%`. The screen refuses those saves (`otp-code-slot`, `otp-format`, `otp-needs-text`) and warns, without blocking, when the SMS with a 6-digit code is over one segment (`otp-sms-length`) or the language is not in the tenant's language list (`otp-locale-unused`).
* **Tenant and language.** The OTP service looks the text up at the request's tenant minus its last segment (`mz.maputo` → `mz`; a state tenant `mz` stays `mz`), in the language the citizen's app is set to, `en_IN` when the request names none.
* **All three or none.** The built-in text is used only while the language has *no* `egov-user` message at all. Once it has one, a missing code makes that OTP type fail, so a save always writes all three codes. **Reset to default** deletes them when the other two are at their built-in text, and nothing else in `egov-user` exists for that language; otherwise it stores the built-in text.
* **When it applies.** The next OTP. The OTP service reads localisation for every OTP and keeps no copy. Saving here also calls `POST /localization/messages/cache-bust`: an upsert alone does not replace a cached *empty* answer, so the first custom wording for a language would otherwise go unseen. Do the same after writing these codes any other way.
* **Who can save.** A role with `/localization/messages/v1/_upsert` (on the shipped seed `LOC_ADMIN`, `ACCOUNT_ADMIN`, `SUPERUSER`); everyone else sees the section read-only.

### 6. Send a test and read the logs

**Providers → Test** on a row (admin role at the state tenant), enter a recipient you may message, **Send Test**, then **View Notification Logs**. A test is a real message and a real log row at your tenant, flagged as a test and hidden unless the **Test sends** filter is set to *Show test sends*. It proves the provider and credentials only — not routing, templates or the channel switch.

A test that nothing could deliver is refused with a reason instead of being sent: the provider is disabled in Novu or doesn't carry the channel you picked, or (no provider named) Novu has no enabled provider for the channel. It is the check that marks a real message `Skipped` with `NB_PROVIDER_UNAVAILABLE`; for a test, the bridge answers `409 NB_PROVIDER_UNAVAILABLE`, the Configurator shows the reason, and nothing is sent or logged. Fix what it names and test again. On a workspace with its own notification account (tenant-accounts.md) the same check reads that workspace's own providers, never the shared account's. If novu-bridge cannot read Novu's provider list at that moment, the test is sent unchecked, as a real message would be, and the result says so.

**Logs** filters: **Complaint #**, **Channel**, **Status**, **Produced by**, **Test sends**. Recipients are masked server-side, with one rule: a user UUID is an ID and is shown as it is; a phone keeps its last three digits (`+***678`), an email its first character and domain (`c***@example.org`). There are no retries: a failed or skipped message is one row, and re-enabling a channel does not resend what was skipped.

<table><thead><tr><th width="247.2890625">Status</th><th>Meaning</th></tr></thead><tbody><tr><td>Sent (accepted by transport)</td><td>The gateway <strong>accepted</strong> it — not proof of delivery</td></tr><tr><td>Queued (provider had not answered)</td><td>A sign-in code (<code>messages/_send</code>) the delivery engine took, whose provider had not answered within a few seconds. It is not updated later</td></tr><tr><td>Delivered / Bounced</td><td>A delivery receipt reported it (§8.3)</td></tr><tr><td>Failed</td><td>Gateway refused it or a receipt reported failure; reason in the Error column</td></tr><tr><td>Skipped</td><td>A deliberate decision not to send; read the code</td></tr><tr><td>Rejected (bad event)</td><td>Malformed or uncatalogued event — a producer fault</td></tr><tr><td>Received (dry run)</td><td>Validation-only run</td></tr></tbody></table>

Rows with Channel *No channel* are decisions taken before any channel (nobody routed, nobody resolved); filter **No channel (nothing sent)**. **Produced by** is *Sent as finished message* (pre-rendered by the producer) or *Routed by notifications* (resolved by the bridge).

| Code                               | Cause                                                                                                                                                                                                                                                                                      | Fix                                                                                  |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ |
| `NB_NO_PROVIDER`                   | Channel off for this tenant (expected on a new city)                                                                                                                                                                                                                                       | §4                                                                                   |
| `NB_PROVIDER_UNAVAILABLE`          | Selected provider missing, disabled or wrong channel (a Twilio SMS provider selected for WhatsApp, or the reverse, included) — or no provider selected and Novu has no active integration for the channel — or SMSCountry / Ozeki / Jasmin with `NOVU_BRIDGE_DIGIT_WORKER_PROVIDERS=false` | §3 / §4; for the last, select a Novu-native provider or turn the worker providers on |
| `NB_NO_ROUTING`                    | No routing row for the event                                                                                                                                                                                                                                                               | Add one (§5.2)                                                                       |
| `NB_NO_TEMPLATE`                   | No template in the recipient's language or `en_IN`                                                                                                                                                                                                                                         | Add it                                                                               |
| `NB_NO_RECIPIENTS`                 | Every audience named nobody                                                                                                                                                                                                                                                                | Check the role has holders in this tenant                                            |
| `NB_CONTACT_MISSING`               | Recipient has no phone / email                                                                                                                                                                                                                                                             | Fix the record or route another channel                                              |
| `NB_CONTACT_INVALID`               | A phone without a country code, at a tenant with no mobile rule and no deployment fallback                                                                                                                                                                                                 | §7 Phone numbers                                                                     |
| `NB_CONFIG_UNAVAILABLE` (`FAILED`) | A phone without a country code, and the tenant's mobile rule could not be read from MDMS                                                                                                                                                                                                   | Replay from the DLQ once MDMS answers (§7 Phone numbers)                             |
| `NB_TEMPLATE_NOT_APPROVED`         | WhatsApp without an approved template                                                                                                                                                                                                                                                      | §5.5                                                                                 |
| `NB_PREFERENCE_DENIED`             | Recipient has not consented to the channel                                                                                                                                                                                                                                                 | Nothing — consent working                                                            |
| `NB_UNKNOWN_AUDIENCE_SCHEME`       | Audience prefix unknown                                                                                                                                                                                                                                                                    | Fix the routing row                                                                  |
| `NB_RECIPIENT_LIMIT_EXCEEDED`      | Fan-out over the cap (1000); nothing sent                                                                                                                                                                                                                                                  | Check role assignments                                                               |
| `NB_EVENT_NOT_IN_CATALOGUE`        | Producer sent an undeclared event                                                                                                                                                                                                                                                          | Report it; not an operator fix                                                       |

Every code: contract/error-codes.md.

### 7. Going live and troubleshooting

Checklist:

* [ ] Each channel you use is on, a provider is selected, and the card says "on and delivering".
* [ ] **Validate** reports zero errors.
* [ ] Every routing row has an `en_IN` template, plus each language you serve.
* [ ] WhatsApp: **your own** approved templates are synced and saved.
* [ ] India SMS: every body is DLT-registered, per language (§8.4).
* [ ] A real complaint transition arrived on a handset / in a mailbox — not just a `SENT` row.
* [ ] Where DIGIT's older OTP service is used: a phone login through it works since SMS was switched on.
* [ ] Only the right people hold an admin role.

<table><thead><tr><th width="171.2421875">Symptom</th><th width="247.0625">Likely cause</th><th>Fix</th></tr></thead><tbody><tr><td>Nothing arrives on any channel</td><td>Channels are off (they ship off unless <code>novu_bridge_channels_enabled</code> named them)</td><td>§4; Logs full of <code>NB_NO_PROVIDER</code></td></tr><tr><td>Nothing on Logs at all</td><td>No event reached the bridge</td><td>Confirm the complaint moved; check services, then the DLQ (kafka-events.md)</td></tr><tr><td>WhatsApp rows skipped</td><td><code>NB_TEMPLATE_NOT_APPROVED</code></td><td>§5.5</td></tr><tr><td><code>SENT</code> but nothing arrives</td><td>Gateway dropped it later</td><td>Gateway's own delivery report: sender id, DLT template, barred number; email: spam, SPF/DKIM</td></tr><tr><td>403 saving in Configurator</td><td>Missing MDMS role, or access-control rows never seeded, or a truncated tenant bootstrap</td><td>Hold <code>MDMS_ADMIN</code>/<code>ACCOUNT_ADMIN</code>/<code>SUPERUSER</code>; if everyone gets 403: <code>./deploy.sh mycity --tags notifications</code>. If the deploy printed <code>master-repair — ACTION</code> (the tenant has exactly 500 role-actions), read the <code>master-repair — result</code> rows, then <code>./deploy.sh mycity -e repair_tenant_masters=true --tags master-repair,notifications</code></td></tr><tr><td>The deploy fails at <code>notif-seed — fail: state_root could not be seeded</code> and names <code>ACL-ID-TAKEN</code> or <code>ACL-ACTION-INACTIVE</code></td><td>No free id for a notification action after five tries, or a notification action the data phase needs is deactivated at that tenant</td><td>migration.md: Access-control actions</td></tr><tr><td>403 on every Notifications save after a 2.20 deploy failed with a 403 on <code>RAINMAKER-PGR.NotificationChannel</code></td><td>The tenant's own actions already used ids 4623–4639; a 2.20 build older than <a href="https://github.com/egovernments/Citizen-Complaint-Resolution-System/pull/2603">#2603</a> granted the notification roles to those other endpoints</td><td>Redeploy (<code>--tags notifications</code>); review the <code>ACL-STALE-GRANT</code> lines it prints under <code>notif-seed — WARNING: access-control rows to review</code> and deactivate the grants that should not be there (migration.md)</td></tr><tr><td>"This needs one of these roles held at a state tenant…" (<code>NB_ADMIN_ROLE_REQUIRED</code>)</td><td>No admin role, or one held only at a city</td><td>Be granted one at the state tenant</td></tr><tr><td><code>403 NB_TENANT_NOT_ALLOWED</code> on Logs, or on Disable / Delete</td><td>Looking at another state's tenant, or managing a provider for a state you are not an admin of</td><td>Log in at the tenant you mean</td></tr><tr><td><code>403 NB_TENANT_NOT_ALLOWED</code> on any provider action, as an admin of your own state</td><td>Your state does not own the deployment's providers (it is not <code>state_root</code>, and not in <code>novu_bridge_provider_admin_tenants</code>)</td><td>Add it to <code>novu_bridge_provider_admin_tenants</code> and redeploy, or use an admin of <code>state_root</code></td></tr><tr><td><code>migrate-notifications.py</code> stops with <code>REFUSED: provider creation refused by novu-bridge (403 NB_TENANT_NOT_ALLOWED …)</code></td><td><code>--create-provider</code> run logged in at a root that does not own the providers; nothing was written</td><td>Create the provider as an admin of <code>state_root</code> (Configurator → Providers), then re-run for your root with <code>--provider SMS=&#x3C;identifier></code>; or list the root in <code>novu_bridge_provider_admin_tenants</code></td></tr><tr><td>The deploy prints <code>notif-seed — WARNING: a complaint root other than state_root could not be seeded</code></td><td>That root (often the stock <code>pg</code> demo complaints of a restored dump) refused a write (exit 3) or could not be read (exit 2). Only a failure at <code>state_root</code> fails the deploy</td><td>Follow the warning: grant the seeding admin <code>MDMS_ADMIN</code> at that root and re-run <code>--tags notifications</code>, seed it by hand (§8.2), or add it to <code>notifications_seed_exclude</code></td></tr><tr><td>The deploy prints <code>notification stack — WARNING: the retired OTP senders were kept</code></td><td>novu-bridge had not taken over <code>egov.core.notification.sms</code> within 10 minutes (not healthy, its listener off, or its consumer group owns / has committed none of the topic's partitions), so <code>egov-notification-sms</code> and <code>otp-publisher</code> were left running. A login OTP can arrive twice meanwhile</td><td>Fix novu-bridge (<code>docker logs novu-bridge</code>; <code>docker exec digit-redpanda rpk group describe novu-bridge</code>), then re-run the deploy</td></tr><tr><td><code>409 NB_PROVIDER_IN_USE</code> on Disable / Delete</td><td>A channel still selects the provider; or it is the last active integration on its Novu channel and a channel with no provider selected sends through it; or MDMS could not be read</td><td>Point the channel at another provider (or add another active integration first); retry if MDMS was down</td></tr><tr><td><code>400 NB_PROVIDER_TYPE_UNAVAILABLE</code> creating / testing an SMSCountry, Ozeki or Jasmin provider</td><td>The Novu worker does not load DIGIT's providers (<code>NOVU_BRIDGE_DIGIT_WORKER_PROVIDERS=false</code>)</td><td>Use a Novu-native provider, or turn them on: Helm <code>global.novuWorkerDigitProviders: true</code> (providers.md)</td></tr><tr><td>Banner "not been migrated yet"</td><td>Tenant still on its 2.12 configuration</td><td><code>migrate-notifications.py plan --tenant mycity</code>, review, then <code>apply</code> (migration.md)</td></tr><tr><td>Banner "No notification configuration on this tenant"</td><td>Defaults never installed</td><td>Fresh install: <code>./deploy.sh mycity --tags notifications</code>. A tenant with complaints (the deploy printed <code>notif-seed — ACTION: this tenant has no notification configuration</code>): <code>migrate-notifications.py plan --tenant mycity --adopt-defaults</code>, then <code>apply --tenant mycity --adopt-defaults --yes</code> (migration.md)</td></tr><tr><td>OTP login of DIGIT's older OTP service stopped</td><td>SMS off or its provider broke</td><td>§4; look for <code>CORE.SMS.OTP</code> rows. Citizen sign-in through the Identity BFF does not use notifications: check its own OTP sender</td></tr><tr><td>The deploy prints <code>notif-seed — ACTION: a self-serve workspace has no notification configuration</code></td><td>A workspace provisioned before pgr-services seeded notification defaults at signup, or whose step failed</td><td>Seed it once with its admin's access token (§8.6)</td></tr><tr><td><code>{emp_name}</code> in a message</td><td>Placeholder has no value for that event</td><td>Use tokens the Events screen lists for it</td></tr><tr><td><code>SKIPPED / NB_NO_ROUTING</code> for a complaint in <code>pg.citya</code></td><td>The complaint's state root (<code>pg</code>) has no configuration — the deploy reported it <code>none</code> (<code>notif-seed — result per state root</code>), or could not log in there (<code>WARNING: could not log in at a complaint root</code>)</td><td>Install its defaults (<code>migrate-notifications.py plan/apply --tenant pg --adopt-defaults</code>), or seed it by hand (§8.2); then configure it logged in at <code>pg</code></td></tr></tbody></table>

Not supported: provider failover, retries, per-city providers within a state, resending skipped messages. The Novu dashboard (`/novu`) is for debugging only.

#### Phone numbers

DIGIT stores a mobile number nationally (`0712345678`); every SMS and WhatsApp message goes out in E.164 (`+254712345678`). The bridge completes a number that has no `+` / `00` with, in order:

1. the user's own `countryCode` (egov-user), when the record carries one;
2. the **tenant's** `common-masters.MobileNumberValidation` rule — the `default: true` one, read at the tenant and then its state root. Workspace signup writes it from the signup country, so a deployment with workspaces in Kenya, India, Ethiopia and Mozambique sends each its own code;
3. `novu_bridge_core_sms_country_code` (`NOVU_BRIDGE_CORE_SMS_COUNTRY_CODE`), the deployment-wide last resort.

None of them: the message is recorded `SKIPPED / NB_CONTACT_INVALID` and not sent — never as `+` followed by the national number, which is someone else's number or nobody's. Fix it by adding the tenant's mobile rule (Configurator → Mobile Number Validation) or setting the deployment fallback. The same rule applies to the codes of DIGIT's older OTP service (`CORE.SMS.*` rows).

The deployment fallback is used only where MDMS **answered** that the tenant and its root have no rule. A rule MDMS cannot be asked for is not "no rule": the bridge uses the last rule it read for that tenant, and with none the message is recorded `FAILED / NB_CONFIG_UNAVAILABLE` and dead-lettered for replay — never sent with another country's code. Such a failure is remembered for 5 seconds, so a short MDMS outage costs a few dead-lettered messages, not one MDMS call each.

Digits that start with the country code (`254712345678`) are either already international or a national number that happens to start with it; the rule's `mobileNumberRegex` decides when exactly one reading matches it. Without one (the deployment fallback) a number that leaves 9 or more digits after the code is read as already carrying it. A bracketed trunk `(0)` (`+254 (0) 712345678`) is dropped.

### 8. Deployment reference

#### 8.1 Deployment settings

Ansible `host_vars/<tenant>.yml` (re-run `./deploy.sh` after changing):

<table><thead><tr><th width="132.88671875">Setting</th><th width="476.44921875">Meaning</th><th>Default</th></tr></thead><tbody><tr><td><code>enable_novu</code></td><td>Starts the notification stack</td><td><code>false</code></td></tr><tr><td><code>seed_notifications</code></td><td>Create the notification schemas and access-control rows, channel rows for a tenant with none, and the shipped defaults on a fresh install (no configuration, no complaint ever filed), at <code>state_root</code> and every state root with complaints — except self-serve workspaces, which pgr-services seeds at signup (§8.6). Never copies legacy rows — that is migrate-notifications.py</td><td><code>enable_novu</code></td></tr><tr><td><code>notifications_adopt_defaults</code></td><td>Also seed the shipped defaults into a state root with no configuration that already has complaints. Only for roots whose complaints are demo data (a restored dump); otherwise use <code>migrate-notifications.py plan/apply --adopt-defaults</code>. <code>true</code> = every root the deploy seeds; a list (<code>[pg]</code>) = those roots only. Never touches a root that has configuration</td><td><code>false</code></td></tr><tr><td><code>notifications_seed_exclude</code></td><td>Regex of state roots the seed skips (and lists) even though they have complaints. The seed covers <code>state_root</code> and <strong>every state root with complaints</strong> — novu-bridge reads a complaint's configuration at its own root (#1943)</td><td><code>(?i)^(PW_|pwt)</code> (test-suite junk)</td></tr><tr><td><code>notif_seed_user</code> / <code>notif_seed_pass</code></td><td>The admin the seed logs in as, <strong>at each root</strong> it seeds. At a root other than <code>state_root</code>, any failure — the user does not exist there, a write refused with 403, MDMS unreadable — is a warning with the command that finishes it; only <code>state_root</code> fails the deploy</td><td><code>ADMIN</code> / <code>eGov@123</code></td></tr><tr><td><code>novu_bridge_provider_admin_tenants</code></td><td>Extra state tenants whose admins may manage providers, besides <code>state_root</code> (#1999 multi-root boxes)</td><td>blank</td></tr><tr><td><code>enable_otp_services</code></td><td>Real OTP login through DIGIT's older OTP service (<code>user-otp</code>), not the Identity BFF's citizen sign-in; requires <code>enable_novu</code></td><td><code>false</code></td></tr><tr><td><code>novu_bridge_proxy_allowed_roles</code> / <code>novu_bridge_proxy_admin_roles</code></td><td>The two role tiers (§1)</td><td>see §1</td></tr><tr><td><code>novu_admin_email</code> / <code>novu_admin_password</code></td><td>Novu's first account, which novu-bridge also signs in as to manage tenant organizations. With <code>novu_tenant_accounts</code> on, leave the password unset (the deploy keeps one in OpenBao and refuses the published default), unless the admin has a custom password you want to keep: then pin it here</td><td>—</td></tr><tr><td><code>novu_tenant_accounts</code></td><td>Each signup workspace gets its own Novu organization and manages its own providers (tenant-accounts.md). Unset, it is on only if the deploy can sign in to Novu as the admin with the password the bridge gets; otherwise off for that deploy, with an <code>ACTION</code> line. <code>true</code> makes a refused sign-in stop the deploy (tenant-accounts.md)</td><td><code>enable_novu</code> (verified)</td></tr><tr><td><code>novu_admin_password_legacy</code></td><td>The Novu admin's password before tenant accounts were turned on; the deploy changes it to the OpenBao one. Pin a hand-changed password here before upgrading</td><td><code>Digit@12345</code></td></tr><tr><td><code>novu_tenant_accounts_backfill</code></td><td>Existing tenants to move onto their own organization on this deploy</td><td><code>[]</code></td></tr><tr><td><code>novu_api_key</code></td><td>Leave unset; the deploy mints it</td><td>—</td></tr><tr><td><code>notification_stack_tag</code></td><td>Image tag of pgr-services, pgr-services-db, novu-bridge and novu-bridge-db — one build (migration.md) — and of digit-user-preferences-service and its -db image. Pin a <code>develop-&#x3C;sha8></code> or release tag; the deploy warns while it is rolling</td><td><code>nightly-develop</code> (rolling; a stopgap until the release pins one)</td></tr><tr><td><code>verify_tenant_masters</code> / <code>repair_tenant_masters</code></td><td>Non-<code>pg</code> state roots: compare the tenant's access-control rows with <code>pg</code>'s and report the gap / also copy the missing rows. Copying is opt-in: a gap can be deliberate, and a copied grant cannot be removed</td><td><code>true</code> / <code>false</code></td></tr><tr><td><code>novu_bridge_channels_enabled</code></td><td><strong>Fallback only</strong>, for a tenant with no channel rows, e.g. <code>"SMS"</code>. The seed turns it into rows for such a tenant</td><td>unset = nothing sent</td></tr><tr><td><code>novu_bridge_receipts_secret</code></td><td>Enables delivery receipts (§8.3); a secret</td><td>blank = off</td></tr><tr><td><code>novu_bridge_preference_enabled</code> / <code>novu_bridge_preference_fail_open</code></td><td>Consent gate; allow delivery when the preference service is down</td><td><code>false</code> / <code>true</code></td></tr><tr><td><code>novu_bridge_core_sms_country_code</code></td><td>Deployment-wide <strong>last resort</strong> country code for a phone number that arrives without one (codes of DIGIT's older OTP service and complaint notifications alike), e.g. <code>+254</code> (<code>254</code> works too); a leading trunk <code>0</code> is dropped. The user's own <code>countryCode</code> and then the tenant's <code>common-masters.MobileNumberValidation</code> rule come first, so workspaces in different countries each get their own code (§7). Blank, at a tenant with no rule: the message is not sent (<code>SKIPPED / NB_CONTACT_INVALID</code>), and the bridge warns at startup</td><td>blank</td></tr><tr><td><code>twilio_account_sid</code> / <code>twilio_auth_token</code> / <code>twilio_whatsapp_from</code></td><td>Bootstrap the <code>twilio-whatsapp</code> Novu integration at deploy</td><td>—</td></tr><tr><td><code>novu_bridge_workflow_id_sms</code> / <code>_whatsapp</code> / <code>_email</code></td><td>Novu workflow ids</td><td><code>complaints-*</code></td></tr><tr><td><code>novu_bridge_integration_id_whatsapp</code></td><td>Only if a second Twilio integration exists</td><td>blank</td></tr><tr><td><code>novu_bridge_sms_provider</code> / <code>novu_bridge_sms_sender_id</code> / <code>novu_bridge_smscountry_user</code> / <code>novu_bridge_smscountry_password</code></td><td>Legacy direct-SMSCountry route (bypasses Novu) — use <strong>either</strong> this <strong>or</strong> an SMSCountry provider, not both</td><td>blank</td></tr></tbody></table>

novu-bridge environment. On Compose, the deploy renders these into `/opt/digit/.env` from the host\_vars above (they are interpolated into the `novu-bridge` service in `local-setup/docker-compose.egov-digit.yaml`; `./deploy.sh` regenerates `.env` on every run, so set them in host\_vars, not in `.env`); on Helm set them in `devops/deploy-as-code/charts/common-services/novu-bridge/values.yaml`.

<table><thead><tr><th width="223.515625">Env</th><th width="367.125">Meaning</th><th>Default</th></tr></thead><tbody><tr><td><code>NOVU_BRIDGE_RECEIPTS_SECRET</code></td><td>Enables delivery receipts (§8.3)</td><td>blank = off</td></tr><tr><td><code>NOVU_BRIDGE_PREFERENCE_ENABLED</code> / <code>NOVU_BRIDGE_PREFERENCE_FAIL_OPEN</code></td><td>Consent gate; allow delivery when the preference service is down</td><td>Compose <code>false</code> / <code>true</code></td></tr><tr><td><code>NOVU_BRIDGE_CORE_SMS_COUNTRY_CODE</code></td><td>Last-resort country code for numbers sent without one, after the user's and the tenant's (see <code>novu_bridge_core_sms_country_code</code>)</td><td>blank</td></tr><tr><td><code>NOVU_BRIDGE_CORE_SMS_DEFAULT_TENANT</code></td><td>Tenant of a code from DIGIT's older OTP service that carries none, <strong>and</strong> the state that owns the providers. Compose: the deploy rewrites <code>pg</code> to <code>state_root</code> and fails if the running bridge has anything else. Helm: egov-config <code>state-level-tenant-id</code></td><td><code>state_root</code></td></tr><tr><td><code>NOVU_BRIDGE_PROVIDER_ADMIN_TENANTS</code></td><td>Extra states whose admins may manage providers (from <code>novu_bridge_provider_admin_tenants</code>; Helm <code>provider-admin-tenants</code>)</td><td>blank</td></tr><tr><td><code>NOVU_BRIDGE_TENANT_ACCOUNTS_ENABLED</code></td><td>Per-tenant Novu organizations (tenant-accounts.md); from <code>novu_tenant_accounts</code> (Helm <code>tenant-accounts.enabled</code>)</td><td><code>false</code></td></tr><tr><td><code>NOVU_BRIDGE_NOVU_ADMIN_EMAIL</code> / <code>_PASSWORD</code>, <code>NOVU_BRIDGE_TENANT_KEY_ENCRYPTION_KEY</code> (<code>_PREVIOUS</code>), <code>NOVU_BRIDGE_INTERNAL_ADMIN_TOKEN</code>, <code>NOVU_BRIDGE_INTERNAL_SEND_TOKEN</code></td><td>Its secrets: from OpenBao on Compose, from the <code>novu-bridge-tenant-accounts</code> Secret on Helm. Never in host_vars</td><td>blank</td></tr><tr><td><code>NOVU_BRIDGE_DIGIT_WORKER_PROVIDERS</code></td><td>Whether novu-worker loads DIGIT's SMSCountry / Ozeki / Jasmin providers; <code>false</code> hides and refuses them. Compose always mounts them, so it is fixed at <code>true</code> there. Helm: <code>global.novuWorkerDigitProviders</code> in <code>env.yaml</code> sets it <strong>and</strong> the worker mount (<code>worker.digitProviders.enabled</code>) together</td><td><code>true</code></td></tr></tbody></table>

Any other property in `backend/novu-bridge/src/main/resources/application.properties` must be added to the service's `environment:` block — Compose reads `.env` only for interpolation. One worth knowing: channel policy is cached for `novu.bridge.channel.policy.cache.ttl.ms` (`NOVU_BRIDGE_CHANNEL_POLICY_CACHE_TTL_MS`, 60 s), so a Channels change or a seed takes up to a minute to apply. Leave `NOVU_BRIDGE_CHANNEL_POLICY_SCHEMA` unset.

#### 8.2 WhatsApp server side

If `twilio_*` was set before the deploy, the `twilio-whatsapp` integration and `complaints-whatsapp` workflow already exist. To add them later without a full deploy (the script does not update an existing integration's credentials):

```bash
export TWILIO_ACCOUNT_SID='<SID>' TWILIO_AUTH_TOKEN='<token>' TWILIO_WHATSAPP_FROM='whatsapp:+<sender>'
export NOVU_ENV_FILE=/dev/null NOVU_INTEGRATION_NAME=twilio-whatsapp NOVU_INTEGRATION_ID=twilio-whatsapp
export NOVU_WORKFLOW_ID=complaints-whatsapp NOVU_WORKFLOW_NAME=complaints-whatsapp
export NOVU_SMS_BODY='Complaint {{payload.complaintNo}} status is {{payload.status}}'
bash backend/novu-bridge/config/bootstrap-novu-whatsapp.sh
unset TWILIO_ACCOUNT_SID TWILIO_AUTH_TOKEN TWILIO_WHATSAPP_FROM NOVU_SMS_BODY
```

Verify (`twilio-whatsapp` active, `complaints-whatsapp` present):

```bash
curl -fsS -H "Authorization: ApiKey $NOVU_API_KEY" "$NOVU_BASE_URL/v1/integrations" \
  | jq '[.data[] | select(.identifier=="twilio-whatsapp") | {identifier,channel,active,primary}]'
curl -fsS -H "Authorization: ApiKey $NOVU_API_KEY" "$NOVU_BASE_URL/v2/workflows?limit=100&page=0" \
  | jq '[.data.workflows[] | select(.workflowId=="complaints-whatsapp") | .workflowId]'
```

Configuration resolves at the complaint tenant's state root (`pg.citya` → `pg`), which on a dump-based install may differ from the deployment's `state_root` ([#1943](https://github.com/egovernments/Citizen-Complaint-Resolution-System/issues/1943)). The Ansible deploy therefore seeds `state_root` **and every state root that has complaints**, and reports each one (`notif-seed — result per state root`). On the stock quickstart (`state_root: pg`, the dump's demo complaints under `pg`) that root is `none` — it has complaints, so it is not provably fresh — until `notifications_adopt_defaults: true` (or `[pg]`) or a reviewed `migrate-notifications.py apply --tenant pg --adopt-defaults --yes` installs the defaults. Seed a root by hand when the deploy could not (its admin does not exist at that root, so the deploy printed `WARNING: could not log in at a complaint root`), or when you use `enable-notifications.sh`, which seeds only `NOTIF_TENANT` and names the other roots. Then log in to the Configurator at that root:

```bash
export NOTIF_TENANT=pg DIGIT_URL='http://127.0.0.1:18000' DIGIT_USERNAME='ADMIN' \
       DIGIT_PASSWORD='<bootstrap_password>' DIGIT_LOGIN_TENANT=pg
cd /opt/digit/notification-seed
NOTIF_SEED_PHASE=access python3 seed-notifications.py
# printed ACL-CHANGED? then: sudo docker restart egov-accesscontrol, and wait for
# curl -sf http://127.0.0.1:18000/access/health before the next command
export NOTIF_CHANNELS_ALLOWLIST="$(sudo docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' novu-bridge | sed -n 's/^NOVU_BRIDGE_CHANNELS_ENABLED=//p')"
SCHEMA_FILE=RAINMAKER-PGR.json NOTIF_SCHEMA_FILE=NOTIFICATIONS.json DATA_DIR=. \
  NOTIF_SEED_PHASE=data python3 seed-notifications.py
unset DIGIT_PASSWORD NOTIF_CHANNELS_ALLOWLIST
```

No admin with a DIGIT password at that root (a Keycloak-only admin)? Replace `DIGIT_USERNAME` / `DIGIT_PASSWORD` / `DIGIT_LOGIN_TENANT` with `DIGIT_ACCESS_TOKEN=<that admin's DIGIT access token>` (§8.6 says where to find it). Exit 3 means a write was refused with 403: restart `egov-accesscontrol` and run the data phase again. Exit 4 (`NOTIF-LOGIN-REFUSED`) means the login itself was refused: that user does not exist at `DIGIT_LOGIN_TENANT`, the password differs, or the access token has expired; nothing was read or written. Without `NOTIF_CHANNELS_ALLOWLIST` a tenant with no channel rows gets none (it keeps following the env allowlist). Omitting `NOTIF_SCHEMA_FILE` seeds only the legacy masters. Without `NOTIF_TENANT_COMPLAINTS=0` a root with no configuration is left `none` (the seed cannot tell it from a live 2.12 tenant): install its defaults after a review with `migrate-notifications.py plan/apply --tenant pg --adopt-defaults`, or pass `NOTIF_ADOPT_DEFAULTS=1` when its complaints are demo data.

#### 8.3 Delivery receipts

`SENT` moves to `DELIVERED` / `BOUNCED` / `FAILED` only when a provider reports back. Set `NOVU_BRIDGE_RECEIPTS_SECRET` and point the provider at:

<table><thead><tr><th width="185.8125">Provider</th><th width="353.2109375">URL</th><th>Auth</th></tr></thead><tbody><tr><td>Novu webhook</td><td><code>POST &#x3C;public>/novu-bridge/novu-adapter/v1/receipts/novu</code></td><td>header <code>X-Receipt-Secret</code></td></tr><tr><td>SMSCountry DR callback</td><td><code>GET/POST &#x3C;public>/novu-bridge/novu-adapter/v1/receipts/smscountry?secret=&#x3C;secret></code></td><td>query <code>secret</code></td></tr></tbody></table>

Blank secret = endpoint answers 403. Late or duplicate reports never move a row backwards. Until receipts are wired, judge delivery by the gateway's own report — SMSCountry returns a job ID even for messages it later drops.

#### 8.4 SMS India DLT registration

Every message must match a template registered against your sender ID; unregistered ones are accepted and then dropped (they still get a job ID). Matching is on content; no template ID is sent. Register each template from **Configure**, replacing `{variable}` with `{#var#}`, once per language (Hindi registers separately).

<table><thead><tr><th width="137.45703125">Audience</th><th width="212.671875">Transition</th><th>Variables</th></tr></thead><tbody><tr><td>Citizen</td><td>submitted / re-opened</td><td>complaint type, id, date</td></tr><tr><td>Citizen</td><td>assigned / re-assigned</td><td>complaint type, id, date, employee name, designation, department</td></tr><tr><td>Citizen</td><td>rejected</td><td>complaint type, id, date, rejection reason (free text; DLT caps a variable at ~30 characters — drop it if enforced)</td></tr><tr><td>Citizen</td><td>resolved</td><td>complaint type, id, date, employee name</td></tr><tr><td>Employee</td><td>assigned to you</td><td>employee name, id, complaint type</td></tr><tr><td>Employee</td><td>feedback received</td><td>employee name, complaint type, id, rating</td></tr></tbody></table>

#### 8.5 Testing email without a real mailbox

[Ethereal](https://ethereal.email) issues throwaway SMTP accounts that capture mail:

```bash
curl -sS -X POST https://api.nodemailer.com/user -H 'Content-Type: application/json' \
  -d '{"requestor":"ccrs","version":"1.0.0"}'
```

Put the returned `host`, `port`, `user`, `pass` into an Email (SMTP) provider, trigger a transition, read the mail at ethereal.email. It proves wiring, not deliverability (SPF/DKIM).

#### 8.6 New workspaces

A self-serve workspace (signup → provisioning) gets its notification configuration from pgr-services, at signup: onboarding step `NOTIFICATION_DEFAULTS`, right after the platform baseline. It writes what the deploy writes for a fresh tenant — the notification schemas, the access-control rows the **Notifications** screens need, the event catalogue, the default templates, WhatsApp provider templates and routing — from the same files (`backend/pgr-services/docs/onboarding-workspace-contract.md`). It writes **no channel rows**: the workspace follows `novu_bridge_channels_enabled` until its admin switches channels on (§4), and it has no provider until one is added (§3).

With per-tenant notification accounts on (tenant-accounts.md), the last step of the same signup (`NOTIFICATION_ACCOUNT`) also gives the workspace its own Novu organisation, which starts with **no providers**. Its messages then go only through that organisation: until its admin adds a provider there, every message on an enabled channel is recorded `SKIPPED / NB_PROVIDER_UNAVAILABLE` and a **Test** is refused with the reason. The deployment's shared providers are never used.

The deploy does **not** seed workspaces: their root has no `ADMIN`, and the founder signs in through Keycloak only, so has no DIGIT password. It lists them instead (`notif-seed — roots`: *Self-serve workspaces, NOT seeded by the deploy*, with each one's `NOTIFICATIONS.Routing` row count) and prints `notif-seed — ACTION: a self-serve workspace has no notification configuration` for one without routing: provisioned before the step existed, or its step failed (pgr-services logs `Workspace <T>: notification defaults were NOT fully seeded (<code>)`; the signup still completes). Seed such a workspace once with the **DIGIT access token** of one of its admins — the founder's: in a signed-in Configurator session it is the `authToken` of any `/mdms-v2` request in the browser's network tab. It expires; take a fresh one.

```bash
cd /opt/digit/notification-seed
export DIGIT_URL='http://127.0.0.1:18000' NOTIF_TENANT=<workspace>
read -rs DIGIT_ACCESS_TOKEN && export DIGIT_ACCESS_TOKEN     # paste the token; it is never printed
NOTIF_SEED_PHASE=access python3 seed-notifications.py
# printed ACL-CHANGED? then: sudo docker restart egov-accesscontrol, and wait for
# curl -sf http://127.0.0.1:18000/access/health before the next command
export NOTIF_CHANNELS_ALLOWLIST="$(sudo docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' novu-bridge | sed -n 's/^NOVU_BRIDGE_CHANNELS_ENABLED=//p')"
SCHEMA_FILE=RAINMAKER-PGR.json NOTIF_SCHEMA_FILE=NOTIFICATIONS.json DATA_DIR=. NOTIF_ADOPT_DEFAULTS=1 \
  NOTIF_SEED_PHASE=data python3 seed-notifications.py
unset DIGIT_ACCESS_TOKEN NOTIF_CHANNELS_ALLOWLIST
```

`NOTIF_ADOPT_DEFAULTS=1` is right for a workspace even when it has complaints: a workspace never ran 2.12's hard-coded notifications, so there is no wording of its own to preserve. The token is checked with egov-user (`/user/_details`) before anything is read; a refused one is exit 4 (`NOTIF-LOGIN-REFUSED`: expired, or not a DIGIT token of a user at that workspace) and nothing is read or written. `migrate-notifications.py` takes the same `DIGIT_ACCESS_TOKEN`.

**Channels differ by how the workspace was seeded.** One seeded at signup has **no channel rows**: it follows the deployment's `novu_bridge_channels_enabled` as it is now and as it changes, until its admin switches a channel on the **Channels** screen. One seeded by hand as above gets channel rows **written once** from `NOTIF_CHANNELS_ALLOWLIST` (the bridge's setting at that moment): a later change of `novu_bridge_channels_enabled` does not reach it; switch its channels on the **Channels** screen.

The signup step runs only where the notification stack does: Ansible renders `PGR_ONBOARDING_NOTIFICATION_DEFAULTS_ENABLED` from `enable_novu` (override: `pgr_onboarding_notification_defaults_enabled`), Helm from `onboarding.notificationDefaultsEnabled`. A retryable failure (MDMS or egov-user unavailable, a row not yet visible) is retried by the signup on its back-off, up to 10 times; after that, or at once for a failure that retrying cannot fix, the step gives up without stopping the signup and pgr-services logs `notification defaults were NOT fully seeded`.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.digit.org/complaints-management/deploy/configure/notifications/setup-notifications.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
