# Deploy Complaints Management ## Overview This section walks you through the end-to-end process of configuring and deploying DIGIT (CMS – Complaints Management) in a production environment—whether on-premise or on any commercial cloud platform. The installer can be run on a laptop or in a virtual machine, but **deploying DIGIT directly on a laptop is not recommended**. DIGIT deployment consists of three key phases: 1. **Infrastructure Provisioning -** Creating the Kubernetes cluster and provisioning all required resources. 2. **DIGIT Deployment -** Deploying all DIGIT microservices onto the provisioned infrastructure. 3. **Configuration -** Applying environment-specific configurations and preparing the services for production use. While infrastructure provisioning varies across cloud providers and on-prem environments, the DIGIT deployment process remains consistent once the infrastructure is ready. ### Before you begin… 1. Make sure a domain name is ready, and that you have admin access to the FQDN you plan to use for the DIGIT deployment. Example - [mydigit.org](http://mydigit.org) 2. Ensure SSL certificates are available for the domain 3. SMS gateway URL & credentials (If you are planning on integrating with an SMS provider) 4. Logo, banner, header, footer and other assets for the UI (Optional. Can be configured later as well) 5. Ensure a GitHub account (or other equivalent tool) is set up under an organisation umbrella. 6. Teams are created (optional) 7. Admin & other users have access to the GitHub repository with the correct access. 8. Ensure you have the requisite access, credentials and permissions to deploy. 9. Ensure the following tools are installed locally on the machine where the installer will run. These tools are used in the DIGIT DevOps stack: 1. [Helm](https://helm.sh/docs/intro/install/) 2. [Helmfile](https://helmfile.readthedocs.io/en/latest/) 3. Git (for SCM and configuration synchronisation) 4. sops + age - recommended for secure secrets management ## Steps {% stepper %} {% step %} ### Infra Provisioning For infrastructure provisioning, follow the steps in [the document here](https://docs.digit.org/platform/guides/installation-guide/infrastructure-setup) for the specified cloud provider. The provisioning process generates key artefacts, which are displayed on the console and stored in an [`outputs.tf`](http://outputs.tf/) file.\ Before proceeding to **Step 2: DIGIT Deployment**, ensure that you have access to the [`outputs.tf`](http://outputs.tf/) file created during the provisioning step. #### Checkpoints: * **Verify successful execution**\ Ensure the console shows no errors and that all provisioning steps have completed successfully. Resolve any issues before moving forward. * **Confirm the presence of `outputs.tf`**\ Check that the `outputs.tf` file is available under the `` and that all required variables are populated. This file will be required in **Step 2**. * **Download the Kubernetes configuration file**\ Make sure you have downloaded the Kubernetes config (`kubeconfig`) and have access to it for interacting with the cluster. {% endstep %} {% step %} ### Modify Configuration for DIGIT CMS Deployment Before starting Step 2, ensure the following: 1. **Step 1 (Infrastructure Provisioning) has been completed successfully**\ No errors in the console. 2. **You have the following files** * `outputs.tf` (from infra provisioning) * `kubeconfig` (for cluster access) 3. **Helm installed**\ Helmfile uses Helm internally, so make sure Helm is installed and configured. #### **2.1: Fork the DIGIT CMS Repository** Fork the CMS repository into your GitHub organisation and name it appropriately. #### **2.2: Clone the Fork** Clone your forked repository onto your local machine. ``` git clone -b master https://github.com/{forked_git_org_name}/Citizen-Complaint-Resolution-System.git ``` #### **2.3: Open the DevOps Configuration Folder** This folder contains all Helmfiles, chart values, and environment configurations. ``` cd Citizen-Complaint-Resolution-System/devops ``` #### **2.4: Configure Environment Values** Locate the **`env.yaml`** file and: * Make a copy named **`production.yaml`**. ``` deploy-as-code/charts/environments/env.yaml ``` * Replace all placeholder values (DB hostname, bucket names, domain, etc.).\ Many of these values come from the `outputs.tf` file. Configure the items given below: **2.4.1 Configure the domain** This FQDN will be used to access the DIGIT instance once all setup is complete. **Where:** ``` global: domain: egov-services-fqdn-name: https:/// host-map: "{'':'https:///'}" ``` **What:** Public DNS name pointing to the ingress LoadBalancer. **Example:** crs-demo.mydomain.org For on-premise setup only (not required for cloud): * Ensure the domain is mapped to a public IP. * Set up managed TLS certificates for secure communication (recommended for production servers). * DNS: Map the CMS domain → LoadBalancer external IP. * Network/Firewall: allow HTTPS to the LoadBalancer IP; allow egress to DB/Kafka/ES/SMTP/SMS/etc. **2.4.2 Configure database** Locate the database host name from the [outputs.tf](http://outputs.tf) file. Any database name without special characters or underscores can be entered below, and it will be created as part of the service deployment. **Where:** ``` db-host: db-url: jdbc:postgresql:/// db-otel-url: jdbc:otel:postgresql:/// db-name: ``` **What:** PostgreSQL server and main application database. **Examples:** * Host: * egov-demo.postgres.database.azure.com * pgr-demo-db.ap-south-1.rds.amazonaws.com * Database: * egovdemo * pgrdemodb **2.4.3 Configure Tenant** The tenant is the account ID that will host the applications and configurations. For more info on tenancy, please refer to <**TBD**>. Tenant IDs on digit can be any string. An example would be "et" for Ethiopia or "in" for India. {% hint style="info" %} **Note:** egov-state-level-tenant-id and state-level-tenant-id need to be the same string. {% endhint %} **Where:** ``` egov-state-level-tenant-id: state-level-tenant-id: host-map: "{'':'https:///'}" egov-user: egov-state-level-tenant-id: egov-enc-service: state-level-tenant-id: egov-state-level-tenant-id: ``` **Example Tenant IDs:** ``` in → India ke → Kenya et → Ethiopia ``` {% hint style="info" %} ⚠️ If tenant ID changes: * Update [globalConfigsPGR.js](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/master/configs/assets/globalConfigsPGR.js) * Upload the updated file to S3 or other storage where static assets are stored * Update UI reference
{% endhint %} **2.4.4 Configure Filestore / S3** This sets up the S3 or blob storage or NFS backing the filestore service. This needs to be configured to ensure all files are stored securely. The bucket has been created as part of the infra provisioning, so locate the filestore\_s3\_bucket from the output console and update it. Filestore Bucket - `fixed-bucketname: ` For on-premise deployment, follow the [Minio Setup.](/complaints-management/complaints-resolution-v2.10/deploy/setup/production-setup/setup-infrastructure/on-premise/deploy-minio.md) **2.4.5 Confiture UI Assets Bucket** {% hint style="info" %} **Note:** This can be configured at a later point post-deployment as well. {% endhint %} Create a [bucket](https://docs.digit.org/complaints-resolution/configure/aws-s3-bucket) to store UI assets that will be used to display in the UI. Ensure you have the right permissions configured for this. This will be used to store static assets such as banners, logos, and configuration JS. Download [this](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/master/configs/assets/globalConfigsPGR.js) global configuration and modify the following key values: ``` var stateTenantId = ""; ``` Replace \ with the tenant ID defined in Step 2.4.3 – Tenant Configuration. This is a critical step, as the tenant ID configured in this file is used to fetch localisation and master data required for rendering the login page correctly. **2.4.6 Configure Git Indexer & Persister** Provide the link to the forked repository (Step 2.1) and the branch to be used for configurations to be pulled in. ``` repo: "git@github.com:/Citizen-Complaint-Resolution-System" branch: "master" ``` **2.4.7 Authenticate OAuth/Github (Optional)** {% hint style="info" %} Note: This is not required if the observability stack is not deployed. Not needed for dev environments. Can be configured later as required. {% endhint %} This is required for authorising access to the monitoring tools & dashboards. In the env.yaml file, search for the “oauth2-proxy:” section. Replace the values below: ``` github_org = github_team = ``` **Examples:** * Org: egovernments, my-org * Team: platform-admins, devops **2.4.8 Configure Storage Class (On-Premise only)** ``` minio: persistence: storageClass: ``` **Examples** * gp3 * standard * managed-premium {% endstep %} {% step %} ### Configure Secure Secrets Management Every deployment has DB, GitHub and other passwords that need to be configured. These need to be managed securely. Below are the steps to secure secrets management. * Create a copy of the env-secret.yaml file: deploy-as-code/charts/environments/env-secrets.yaml and call it \-secrets.yaml. Example: production.yaml will have a production-secrets.yaml * Encrypt credentials and secrets using[ sops](https://github.com/mozilla/sops#updatekeys-command) and create a[ \-secret.yaml](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/master/devops/deploy-as-code/charts/environments/env-secrets.yaml) separately * SOPS expects an encryption key to encrypt/decrypt a specified plaintext and keep the details secured. The following are the options to generate the encryption key - * Option 1: Generate PGP keys[ https://fedingo.com/how-to-generate-pgp-key-in-ubuntu](https://fedingo.com/how-to-generate-pgp-key-in-ubuntu/) * Option 2:[ Create AWS KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html) when you want to use the AWS cloud provider. * Option 3:[ Create Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/quick-create-portal) when you want to use the MS Azure cloud provider. * Once you generate your encryption key, create a .sops.yaml configuration file under the /helm directory of the cloned repo to define the keys used for specific files. Refer to the SOP[ doc](https://github.com/mozilla/sops#211using-sopsyaml-conf-to-select-kmspgp-for-new-files) for more details. {% hint style="info" %} **Note:** For demo purposes, you can use the[ env-secrets.yaml](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/master/devops/deploy-as-code/charts/environments/env-secrets.yaml) as it is without SOPS configuration, but make sure you update your specific details like Git SSH, URL, etc. When you decide to push these configurations into any git or public space, make sure you follow the SOPS configuration mentioned in this article[ sops](https://github.com/mozilla/sops#updatekeys-command) to encrypt your secrets. {% endhint %} Configure the following secrets in the env-secret.yaml file **3.1 Database passwords** {% hint style="info" %} **Note:** Database and Flyway passwords must be the same. {% endhint %} ``` db: username: password: flywayUsername: flywayPassword: ``` **3.2 Git Sync - Config Repository Access** Generate SSH key pairs using the method below: * Generate a new [SSH key pair](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for GitHub authentication. * [Add the public SSH key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account) to the relevant GitHub user account to grant read access to the configuration repository. * Update the private SSH key in the deployment [secrets](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/d8a4d6c1f36ca5d84ec30a99253ebe8a9280a2fb/devops/deploy-as-code/charts/environments/env-secrets.yaml#L77) file:\ deploy-as-code/charts/environments/\-secrets.yaml * Paste the generated private SSH key under the git-sync section in the file. {% hint style="info" %} **Note:** * When copying and pasting the SSH private key into the secrets file (deploy-as-code/charts/environments/env-secrets.yaml), ensure the indentation is exactly as defined in the YAML structure. * Ensure there are no extra spaces or blank characters at the end of any line. * Incorrect indentation or trailing spaces may cause YAML parsing errors and result in git sync failures in the persister and indexer services. * Generate an SSH authentication key, generate a new[ SSH key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) and[ add it to the above user account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account). * Add the SSH private key that you generated in the [env-secrets.yaml](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/d8a4d6c1f36ca5d84ec30a99253ebe8a9280a2fb/devops/deploy-as-code/charts/environments/env-secrets.yaml#L77) under the git-sync section. {% endhint %} ``` git-sync: ssh: |- -----BEGIN RSA PRIVATE KEY----- ``` **Best Practices** * Use a dedicated bot user (e.g., egov-bot) * Grant read-only access * Add key under GitHub → SSH Keys * known\_hosts should remain unchanged. **3.3 Encryption Secrets - Mandatory Change** The encryption service encrypts/decrypts data using encryption keys. These are configurable below. Change the values before the first deployment. For more information, refer to [this](https://core.digit.org/platform/core-services/encryption-service#configuration-details) document. ``` egov-enc-service: master_password: master_salt: master_initialvector: ``` {% hint style="info" %} ⚠️ Critical Security Requirement * Change values before the first deployment * Do not change after go-live * Losing or changing these values will make the encrypted data unreadable {% endhint %} {% endstep %} {% step %} ### DIGIT CMS Deployment Once all configurations are completed. Run the command below to install DIGIT CMS successfully. Must run the command below from the [devops](https://github.com/egovernments/Citizen-Complaint-Resolution-System/tree/localisation-script/devops) folder. ``` $ helmfile -f deploy-as-code/digit-helmfile.yaml -e {env_filename} apply ``` {env\_filename} - the [env.yaml](https://github.com/egovernments/Citizen-Complaint-Resolution-System/blob/localisation-script/devops/deploy-as-code/charts/environments/env.yaml) if you created a copy of env.yaml, then that filename should be replaced here. This will deploy DIGIT CMS using Helmfile in a controlled and auditable manner. {% endstep %} {% step %} ### Domain name and CNAME mapping At the end of the infra provisioning step, the Kubernetes configuration file will be available for download. Download the configuration file and ensure it is active. Run the below command to get the loadbalancer ID from the EKS cluster: ``` kubectl get svc ingress-nginx-controller -n backbone -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' ``` Example output: `ae210873da6f.ap-south-1.elb.amazonaws.com` Add this as a CNAME record in your domain provider settings. At the end of this step, you should be able to access the DIGIT CMS UI - citizen and employee portals - by typing in the `https:///digit-ui/citizen` {% endstep %} {% step %} ### Troubleshooting Deployment **6.1: Check Pod Status** After deployment, first verify that all pods in the target namespace are running correctly. List all pods in the namespace: `kubectl get pods -n ` Ensure that the STATUS column shows only: * Running * Completed There should be no pods in the following states: * CrashLoopBackOff * Init:Error * Init:ConfigMapKeyMissing **Example:** kubectl get pods -n egov If any pod is not running as expected, refer to the FAQ / Troubleshooting section before proceeding further. **6.2 Check if basic configuration data is loaded** The default-data-handler service is responsible for seeding essential system data during initialisation. This includes: * Core and common MDMS schemas and master data for DIGIT CRS * Default English localisation data If any dependent services (such as MDMS, Localisation, or Boundary) are not running at the time of initialisation, the default data will not be loaded. Once the required services are available, the default-data-handler service must be restarted to trigger data seeding again. **6.3 Restart Gateway Service** In some cases, the gateway service may start before all other services are fully available.\ When this happens, the gateway may not register newly started services, resulting in 404 errors when accessing their APIs. To resolve this issue, restart the gateway service: `kubectl rollout restart deployment gateway -n egov` This ensures all downstream services are correctly registered and accessible. {% endstep %} {% step %} ### Set up CMS Notebook Refer to the [CMS Data Setup](/complaints-management/complaints-resolution-v2.10/deploy/setup/production-setup/deploy-complaints-management/data-setup.md) section for detailed steps. {% endstep %} {% step %} ### Get Started Use the URL below to log in to the employee dashboard: https\://\/digit-ui/employee {% endstep %} {% endstepper %} ## Destroy Cluster Use the command below to destroy the deployed DIGIT CMS cluster. `$ helmfile -f digit-helmfile.yaml -e pgr-sdc-prd destroy` ### Support & Best Practices * Always version control env.yaml and env-secrets.yaml * Never commit plain secrets to public repositories; always encrypt and push into a git repo * Test deployments in a non-prod environment first * Maintain separate values files for dev/staging/prod --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.digit.org/complaints-management/complaints-resolution-v2.10/deploy/setup/production-setup/deploy-complaints-management.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.