1 of 40

Backend Developer Guide

DIGIT backend development guide

This guide provides detailed steps for developers to create a new microservice on top of DIGIT. At the end of this guide, you will be able to run the provided (code provided), test it out locally and also deploy it using CI/CD to your DIGIT environment.

Steps to create a microservice:

Set up your development environment
Develop the registries, services, and APIs for a voter registration module that were described in the
Integrate with an existing DIGIT environment and re-use a lot of the common services using Kubernetes port forwarding
Test the new module and debug
Build and deploy the new service in the DIGIT environment

The guide is divided into multiple sections for ease of use. Click on the section cards below to follow the development steps.

Access the sample module . Download and run this in the local environment.

Section 0: Prep

This section provides the complete details of the system pre-requisites, prepping and setting up the system for development.

Follow the steps in the docs resources below:

Development Pre-requisites

Design Inputs

Artefacts required before beginning the development phase

The are the inputs to the development phase.

The docs below guide you through the steps and the resources required to build and design the module.

High Level Design

Available design artifacts

API Specs

Find below the Open API specifications defined for this guide:

Birth Registration

Process Workflow Diagram

Registries

As a part of this guide, we are going to build a single birth registry. We will re-use the user registry from the DIGIT core. This will capture the mother and father details and store them in the user registry. The baby's details will remain in the birth registry.

Services

A single birth service will manage the registry.

System Architecture Diagram

Low Level Design

This page describes all the low level design artifacts available for the service

DB Schema Diagram

The DB tables provide the registry maps as given below:

Table Name

Description

Development Environment Setup

Overview

Follw the steps outlined on this page to setup the DIGIT development environment.

Steps

To setup the DIGIT development environment -

Run the Kafka and PostgreSQL on the development machine and re-use other services from the DIGIT development environment. The following tools are required for development:
- Install Git
  Git for windows
  Git for linux
- Install JDK8
  JDK8 for windows
  JDK8 for linux
- Install IDE - For creating SpringBoot/Java applications we recommend using IntelliJ IDE. IntelliJ can be downloaded from the following links -
  IntelliJ for windows
  IntelliJ for linux
- Install the Lombok plugins for IntelliJ as we use Lombok annotations in this module.
- Install Kafka (version 3.2.0 which is the latest version) - To install and run Kafka locally, follow the following links -
  Kafka for windows
  Kafka for linux
- Install Postman - To install postman, follow the following links -
  Postman for windows
  Postman for linux
- Install Kubectl - Kubectl is the tool that we use to interact with services deployed on our sandbox environment -
  https://kubernetes.io/docs/tasks/tools/install-kubectl-windows/https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
- Install aws-iam-authenticator - (if the DIGIT development environment is in AWS) - https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html
- Install PostgreSQL v10 locally
Add configuration - Post installation of kubectl, add the following configuration in the Kubernetes config file to allow access to resources present in our sandbox environment. Kubernetes config file is usually present in the user's home directory (which varies from OS to OS).
For example, on a Mac, it is present at:/Users/xyz/.kube/config
Once the config file is created, add the following content to it.

Note that you will have to replace the placeholder keys and tokens in the below snippet with your AWS-specific keys. Please contact your system administrator for help with this.

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01EVXhNekV6TlRReE5sb1hEVE13TURVeE1URXpOVFF4Tmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTkJyClN6aHJjdDNORE1VZVF5TENTYWhwbEgyajJ1bkdYSWk1QThJZjF6OTgwNEZpSjZ6OS9qUHVpY3FjaTB1VURJQnUKS3hjdVFJRkozMG1MRWg3RGNiQlh2dDRnUlppZWtlZzVZNGxDT2NlTWZFZkFHY01KdDE1RVVCUFVzdlYyclRMcQp6a0ovRzVRUUFXMmhwREJLaFBoblZJTktYN1YzOU9tMUtuTklTbllPWERsZ1g3dW9Wa3I1OFhzREFHWEVsdC9uClpyc3laM2pkMWplWS8rMXlQQzlxbkorT0QwZlRQVGdCV1hMQlFwMHZKdHVzNE1JV2JLdkhlcUZ5eWtGd2V5MmoKSzk5eU1Yb0oraUpCaFJvWGllU3ZrNnFYdG44S2l4bVJtOXZPQk1hcWpuNkwwTjc3UWNCNjVRaHNKb0tWKzBiMQp5VVpJTHVTWWVTY0Yra3h6TzFVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFNdnF3THl6d2RUL05OWlkvanNzb0lmQmIyNDgKZ3oxSHRuSXJ4UGhaY3RrYjBSMExxeTYzRFZBMFNSN0MrWk90aTNNd3BHMkFSVHVzdG1vYm9HV3poUXlXRk16awpVMVNIZSt6S3poeGcweUpjUjliZnlxM1ZtQVVCZlQyTVV5cVl2OVg0aWxpbmV0SURQaFBuWnlPMERQTHJITGoyCkcxZy8vWmZYbmFCT2k3dlZLSXFXUUR6RlltWGkwME9vOEVoalVyMU5sQ3FISnF1dUo3TlRWQWk1cXA0Qm1xWU8KUTBrbTVxTVVHbG9ZdkNmN1lHQWREWTVnWGg4dzFVMVdaNWNub0Q4WWc3aEtlSjRMRzRram1adlNucGZrS3VxNApiVDdUSjEwUEZlWFJkek8xa2FkQ3VMQSttUlg3OEd5WEw0UTZnOFdPUlhOVDYzdXN3MnlpMXVVN1lMTT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    server: https://3201E325058272AA0990C04346DA6E82.yl4.ap-south-1.eks.amazonaws.com
  name: eks_egov-dev
contexts:
- context:
    cluster: eks_egov-dev
    namespace: egov
    user: eks_egov-dev
  name: dev
current-context: dev
kind: Config
preferences: {}
users:
- name: eks_egov-dev
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - token
      - -i
      - egov-dev
      command: aws-iam-authenticator
      env:
      - name: AWS_ACCESS_KEY
        value: {AWS_ACCESS_KEY_PLACEHOLDER}
      - name: AWS_SECRET_ACCESS_KEY
        value: {AWS_SECRET_ACCESS_KEY_PLACEHOLDER}
      - name: AWS_REGION
        value: {AWS_REGION_PLACEHOLDER}

Once the configuration file is created, access the pods running in the environment by typing: kubectl get pods

This lists all the pods in the development environment.

In case you run into an error stating “Error: You must be logged in to the server (Unauthorized)”, add sudo before the command. For example, “sudo kubectl get pods”. That should resolve the error.

Section 1: Create Project

This section showcases how to create a basic Spring Boot project using an API spec and configure the database.

Follow the steps detailed as detailed below:

Generate Project Using API Specs

Generate Project Stub

Overview

This page provides detailed steps for generating projects using the given API specifications.

Steps

Build A Microservice

Prepare Swagger contracts that detail all the APIs that the service is going to expose for external consumption. eGov uses a customised Swagger Codegen tool.
Download the jar file and make sure it is available in the classpath. Use the Swagger Codegen tool to generate client SDKs using these Swagger contracts.

Refer to the following tutorials to understand the creation of Swagger contracts -

OpenAPI 3.0 Tutorial| Swagger Tutorial For Beginners | Design REST API Using Swagger Editor

Generate API Skeleton

Use the generic command below to create an API skeleton for any Swagger contract:

java -jar codegen-1.0-SNAPSHOT-jar-with-dependencies.jar 
-l -t -u {CONTRACT_PATH } -a project_name -b /path/to/code/folder

The following sequence is used to generate the API skeleton using codegen jar:

Navigate to the folder where you have downloaded the codegen jar.
Execute the following command:

java -jar codegen-1.0-SNAPSHOT-jar-with-dependencies.jar -l -t -u https://github.com/egovernments/DIGIT-Dev/blob/birth-registration-service/municipal-services/birth-registration/birth-registration-api-spec.yaml -a birth-registration -b digit

Download the contract available here and save it in a file locally. Run the following command to generate the skeleton code from the contract.

java -jar codegen-1.0-SNAPSHOT-jar-with-dependencies.jar -l -t -u file:///{ABSOLUTE_PATH_OF_FILE} -a birth-registration -b digit

Rename the output folder to birth-registration.
Import it in Eclipse or VS Code.
Update the spring-boot-starter-parent to 2.2.6-RELEASE in pom.xml.
Perform a maven update once the spring boot version is updated.
Make sure the following dependency is present in the pom.xml. If the version is different, make sure to update it to the version given below:

<dependency>
	<groupId>org.egov.services</groupId>
	<artifactId>services-common</artifactId>
	<version>2.0.0-SNAPSHOT</version>
</dependency>

Put a slash in front of server.contextPath and add this property to the application.properties file which helps request handlers to serve requests -

server.contextPath=/birth-registration
server.servlet.context-path=/birth-registration

Add the below external dependencies to pom.xml:

<dependency>
   <groupId>org.flywaydb</groupId>
   <artifactId>flyway-core</artifactId>
</dependency>
<dependency>
   <groupId>org.egov</groupId>
   <artifactId>mdms-client</artifactId>
   <version>0.0.2-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
   <version>42.2.2.jre7</version>
</dependency>

Create Database

Overview

Once PostgreSQL (v10) has been installed and the basic setup is done, we use Flyway to create the tables.

Steps

Enable Flyway Migration

Configure the below properties in the application.properties file to enable flyway migration:

Add the Flyway SQL scripts in the following structure under resources/db/migration/main:

Add the migration files to the main folder. Follow the specified nomenclature while naming the file. The file name should be in the following format:

Example: V20180920110535__tl_tradelicense_ddl.sql

For this sample service, use the following SQL script to create the required tables.

Configure Application Properties

Overview

The application.properties file is already populated with default values. Read on to learn how to customise and add extra values for your application (if required).

Steps

Kafka topics for the module that need to be added are detailed here.

There are three ways to access services:

a. Run the code locally.

b. Access the service in a DIGIT environment.

c. Access the service locally via port forwarding. This bypasses Zuul's authentication and authorization.

Wherever the localhost is in the URL, the Kubernetes port forwarding has been set up from the development environment to the specified port. In your setup, modify the URLs for the various services depending on whether you are using them from an environment or running them locally or accessing them via port-forwarding.

For example, if no port forwarding has been done, you will have to provide the FQDN of your DIGIT install instead of localhost. Also, without port forwarding, you will have to update the auth tokens in your .aws profile file periodically.

Include all the necessary service host URLs and API endpoints in the "application.properties" file.

This guide specifically references the User, Localisation, HRMS, IDGen, MDMS, and Workflow services that are operational within the DIGIT development environment.

Configure Application Properties

Add the following properties to the application.properties file.

#Localization config
egov.localization.host=https://dev.digit.org
egov.localization.workDir.path=/localization/messages/v1
egov.localization.context.path=/localization/messages/v1
egov.localization.search.endpoint=/_search
egov.localization.statelevel=true

#mdms urls
egov.mdms.host=https://dev.digit.org
egov.mdms.search.endpoint=/egov-mdms-service/v1/_search

#hrms urls
egov.hrms.host=https://dev.digit.org
egov.hrms.search.endpoint=/egov-hrms/employees/_search

#User config
#egov.user.host=https://dev.digit.org
egov.user.host=http://localhost:8284/
egov.user.context.path=/user/users
egov.user.create.path=/_createnovalidate
egov.user.search.path=/user/_search
egov.user.update.path=/_updatenovalidate

#Idgen Config
egov.idgen.host=http://localhost:8285/
egov.idgen.path=egov-idgen/id/_generate

#Workflow config
is.workflow.enabled=true
egov.workflow.host=http://localhost:8280
egov.workflow.transition.path=/egov-workflow-v2/egov-wf/process/_transition
egov.workflow.businessservice.search.path=/egov-workflow-v2/egov-wf/businessservice/_search
egov.workflow.processinstance.search.path=/egov-workflow-v2/egov-wf/process/_search

The following properties must be added for configuring the database and Kafka server. Make sure to use the default values to tune in the Kafka server that can be overwritten during deployment.

Include the following properties in the "application.properties" file to configure the database and Kafka for development purposes once you have completed adding the external dependencies to the "pom.xml" file and reloading the Maven changes.

#DATABASE CONFIGURATION
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/birthregn
spring.datasource.username=birth
spring.datasource.password=birth

Configure Kafka

# KAFKA SERVER CONFIGURATIONS
kafka.config.bootstrap_server_config=localhost:9092
spring.kafka.consumer.value-deserializer=org.egov.tracer.kafka.deserializer.HashMapDeserializer
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.group-id={PLACEHOLDER_PUT_KAFKA_CONSUMER_NAME}
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer
spring.kafka.listener.missing-topics-fatal=false
spring.kafka.consumer.properties.spring.json.use.type.headers=false

# KAFKA CONSUMER CONFIGURATIONS
kafka.consumer.config.auto_commit=true
kafka.consumer.config.auto_commit_interval=100
kafka.consumer.config.session_timeout=15000
kafka.consumer.config.auto_offset_reset=earliest

# KAFKA PRODUCER CONFIGURATIONS
kafka.producer.config.retries_config=0
kafka.producer.config.batch_size_config=16384
kafka.producer.config.linger_ms_config=1
kafka.producer.config.buffer_memory_config=33554432

Kafka Topics For Module

Create & Update Kafka Topics For The Module

Append Kafka configurations as per the specific requirements of the DIGIT services. Each module may use different configurations to manage its topics.

# Birth registration Kafka config
btr.kafka.create.topic=save-bt-application
btr.kafka.update.topic=update-bt-application
btr.default.offset=0
btr.default.limit=10
btr.search.max.limit=50

Import Core Models

Overview

The pom.xml typically includes most of the dependencies listed below at project generation time. Review and ensure that all of these dependencies are present. Update the pom.xml in case any are missing.

Steps

Models/POJOs of the dependent service can be imported from the digit-core-models library (work on creating the library is ongoing). These models are used to integrate with the dependent services.

These are pre-written libraries which contain tracer support, common models like MDMS, Auth and Auth and the capability to raise custom exceptions.

Once these core models are imported, it is safe to delete the RequestInfo, and ResponseInfo classes from the models folder and use the ones present in the common contract that is imported.

Import Core Models

Before starting development, create/update the following classes as given below.

Create RequestInfoWrapper POJO within the Models package -

Update the Applicant POJO with the following content -

Create the BTRConfiguration class within the configuration package. The MainConfiguration class should already exist inside the config package.

The core models are imported.

Implement Repository Layer

Overview

Methods in the service layer, upon performing all the business logic, call methods in the repository layer to persist or lookup data i.e. it interacts with the configured data store. For executing the queries, JdbcTemplate class is used. JdbcTemplate takes care of the creation and release of resources such as creating and closing the connection etc. All database operations namely insert, update, search and delete can be performed on the database using methods of JdbcTemplate class.

Steps

On DIGIT the create and update operations are handled asynchronously.

The persister service listens on the topic to which service applications are pushed for insertion and updation. Persister then takes care of executing insert and update operations on the database without clogging the application’s threads.

The execution of search queries on the database returns applications as per the search parameters provided by the user.

Implement Repository Layer

Define POJOs - The Address object is defined in the common contract (refer to the API spec). Link it to the birth registration table via the registrationId as defined in the DB schema.
Update the Address POJO using the below code:

Define a BirthApplicationSearchCriteria POJO to take care of search requests in the DB.

Create packages - Add thequerybuilder and rowmapper packages within the repository folder.
Create a class - by the name of BirthApplicationQueryBuilder in querybuilder folder and annotate it with @Component annotation.
Insert the following content in BirthApplicationQueryBuilder class -

Create a class - by the name of BirthApplicationRowMapper within the rowmapper package and annotate it with @Component.
Add the following content to the class.

Create a class - by the name of BirthRegistrationRepository within the repository folder and annotate it with @Repository annotation.
Add the following content to the class.

The repository layer is implemented.

Create Validation & Enrichment Layers

Overview

Find the steps to create the and on DIGIT.

Validation Layer

All business validation logic should be added to this class. For example, verifying the values against the master data, ensuring non-duplication of data etc.

Steps

Follow the steps below to create the validation layer.

Create a package called validators. This ensures the validation logic is separate so that the code is easy to navigate through and readable.
Create a class by the name of BirthApplicationValidator
Annotate the class with @Component annotation and insert the following content in the class -

NOTE: For the sake of simplicity the above-mentioned validations are implemented. Required validations will vary on a case-to-case basis.

Enrichment Layer

This layer enriches the request. System-generated values like id, auditDetails etc. are generated and added to the request. In the case of this module, since the applicant is the parent of a baby and a child cannot be a user of the system directly, both the parents' details are captured in the User table. The user ids of the parents are then enriched in the application.

Steps

Follow the steps below to create the enrichment layer.

Create a package under DIGIT by the name of enrichment. This ensures the enrichment code is separate from business logic so that the codebase is easy to navigate through and readable.
Create a class by the name of BirthApplicationEnrichment
Annotate the class with @Component and add the following methods to the class -

NOTE: For the sake of simplicity the above-mentioned enrichment methods are implemented. Required enrichment will vary on a case-to-case basis.

Implement Service Layer

Overview

The service layer performs business logic on the RequestData and prepares the Response to be returned back to the client.

Steps

Follow the steps below to create the service layer.

Create a new package called Service.
Create a new class in this folder by the name BirthRegistrationService.
Annotate the class with @Service annotation.
Add the following content to the class -

NOTE: At this point, your IDE must be showing a lot of errors but do not worry we will add all dependent layers as we progress through this guide and the errors will go away.

Build The Web Layer

Implementing the controller layer in Spring

Overview

The web/controller layer handles all the incoming REST requests to a service.

Steps

The @RequestMapping("/v1") annotation is added on top of the controller class. This contains the version of the API (this becomes a part of the API endpoint URL).

Steps to setup the request handler in the controller layer

Follow the steps below to set up the request handler in the controller layer.

Make a call to the method in the Service Layer and get the response back from it.
Build the responseInfo.
Build the final response to be returned to the client.

The controller class here contains the below content -

package digit.web.controllers;

import com.fasterxml.jackson.databind.ObjectMapper;
import digit.service.BirthRegistrationService;
import digit.utils.ResponseInfoFactory;
import digit.web.models.*;
import io.swagger.annotations.ApiParam;
import io.swagger.annotations.Tag;
import lombok.ToString;
import lombok.extern.slf4j.Slf4j;
import org.egov.common.contract.response.ResponseInfo;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

import javax.servlet.http.HttpServletRequest;
import javax.validation.Valid;
import java.util.Collections;
import java.util.List;
@javax.annotation.Generated(value = "org.egov.codegen.SpringBootCodegen", date = "2022-07-26T12:39:05.988+05:30")
@Slf4j
@ToString
@Controller
@RequestMapping("/birth-services")
public class V1ApiController{

    private final ObjectMapper objectMapper;

    private final HttpServletRequest request;

    private BirthRegistrationService birthRegistrationService;

    @Autowired
    private ResponseInfoFactory responseInfoFactory;

    @Autowired
    public V1ApiController(ObjectMapper objectMapper, HttpServletRequest request, BirthRegistrationService birthRegistrationService) {
        this.objectMapper = objectMapper;
        this.request = request;
        this.birthRegistrationService = birthRegistrationService;
    }

    @RequestMapping(value="/v1/registration/_create", method = RequestMethod.POST)
    public ResponseEntity<BirthRegistrationResponse> v1RegistrationCreatePost(@ApiParam(value = "Details for the new Birth Registration Application(s) + RequestInfo meta data." ,required=true )  @Valid @RequestBody BirthRegistrationRequest birthRegistrationRequest) {
        List<BirthRegistrationApplication> applications = birthRegistrationService.registerBtRequest(birthRegistrationRequest);
        ResponseInfo responseInfo = responseInfoFactory.createResponseInfoFromRequestInfo(birthRegistrationRequest.getRequestInfo(), true);
        BirthRegistrationResponse response = BirthRegistrationResponse.builder().birthRegistrationApplications(applications).responseInfo(responseInfo).build();
        return new ResponseEntity<>(response, HttpStatus.OK);
    }

    @RequestMapping(value="/v1/registration/_search", method = RequestMethod.POST)
    public ResponseEntity<BirthRegistrationResponse> v1RegistrationSearchPost(@RequestBody RequestInfoWrapper requestInfoWrapper, @Valid @ModelAttribute BirthApplicationSearchCriteria birthApplicationSearchCriteria) {
        List<BirthRegistrationApplication> applications = birthRegistrationService.searchBtApplications(requestInfoWrapper.getRequestInfo(), birthApplicationSearchCriteria);
        ResponseInfo responseInfo = responseInfoFactory.createResponseInfoFromRequestInfo(requestInfoWrapper.getRequestInfo(), true);
        BirthRegistrationResponse response = BirthRegistrationResponse.builder().birthRegistrationApplications(applications).responseInfo(responseInfo).build();
        return new ResponseEntity<>(response,HttpStatus.OK);
    }

    @RequestMapping(value="/v1/registration/_update", method = RequestMethod.POST)
    public ResponseEntity<BirthRegistrationResponse> v1RegistrationUpdatePost(@ApiParam(value = "Details for the new (s) + RequestInfo meta data." ,required=true )  @Valid @RequestBody BirthRegistrationRequest birthRegistrationRequest) {
        BirthRegistrationApplication application = birthRegistrationService.updateBtApplication(birthRegistrationRequest);

        ResponseInfo responseInfo = responseInfoFactory.createResponseInfoFromRequestInfo(birthRegistrationRequest.getRequestInfo(), true);
        BirthRegistrationResponse response = BirthRegistrationResponse.builder().birthRegistrationApplications(Collections.singletonList(application)).responseInfo(responseInfo).build();
        return new ResponseEntity<>(response, HttpStatus.OK);
    }

}

NOTE: At this point, your IDE must be showing a lot of errors but do not worry we will add all dependent layers as we progress through this guide and the errors will go away.

Since the Codegen jar creates the search API with search parameters annotated with @RequestParam rather than taking request parameters as a POJO. For this, we will create a POJO by the name of BirthApplicationSearchCriteria within the Models package. Insert the following content in POJO.

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;

import java.util.List;

@Data
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
public class BirthApplicationSearchCriteria {

    @JsonProperty("tenantId")
    private String tenantId;

    @JsonProperty("status")
    private String status;

    @JsonProperty("ids")
    private List<String> ids;

    @JsonProperty("applicationNumber")
    private String applicationNumber;

}

The web layer is now setup.

Section 2: Integrate Persister & Kafka

This section contains information on steps for integrating with the Persister Service and Kafka.

Code of developer guide till this stage is available here. Postman collection corresponding to this stage is available here.

Steps to integrate Persister Service and Kafka:

Add Kafka Configuration

Overview

Follow the steps detailed below to add the Kafka configurations.

Steps

Add the below code in application.properties to configure Kafka.

# BTR config
btr.kafka.create.topic=save-bt-application
btr.kafka.update.topic=update-bt-application
btr.default.offset=0
btr.default.limit=10
btr.search.max.limit=50

Implement Kafka Producer & Consumer

Overview

Follow the steps detailed below to implement Kafka Producer & Consumer.

Producer

Producer classes help in pushing data from the application to Kafka topics. DIGIT has a custom implementation of KafkaTemplate class in the tracer library called CustomKafkaTemplate. This implementation of the Producer class does not change across services of DIGIT.

Steps

Access the producer implementation details here - Producer Implementation.
The Codegen jar already has created a Producer class. We will continue using it.
Make sure the tracer dependency version in the pom.xml is 2.0.0-SNAPSHOT.

Consumer

For our guide, we will be implementing a notification consumer in the following section.

Once an application is created/requested or progresses further in the workflow, notifications can be triggered as each of these events is pushed onto Kafka topics which can be listened to and an sms/email/in-app notification can be sent to the concerned user(s).

For our guide, we will be implementing a notification consumer which will listen to the topic on which birth registration applications are created. Create a customised message and send it to the notification service (sms/email) to trigger notifications to the concerned users.

Sending SMS notifications to the customer:

Once an application is created/updated the data is pushed on Kafka topic. We trigger notifications by consuming data from this topic. Whenever any message is consumed the service will call the localisation service to fetch the SMS template. It will then replace the placeholders in the SMS template with the values in the message it consumed.

(For example, It will replace the {NAME} placeholder with the owner name from the data consumed). Once the SMS text is ready, the service pushes this data on the notification topic. SMS service consumes data from notification topic and triggers SMS.

Steps

Open Kafka/NotificationConsumer.java and paste the following code:

package digit.kafka;

import com.fasterxml.jackson.databind.ObjectMapper;
import digit.service.NotificationService;
import digit.web.models.BirthRegistrationRequest;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.handler.annotation.Header;
import org.springframework.stereotype.Component;

import java.util.HashMap;

@Component
@Slf4j
public class NotificationConsumer {

    @Autowired
    private ObjectMapper mapper;

    @Autowired
    private NotificationService notificationService;

    @KafkaListener(topics = {"${egov.bt.registration.create.topic}"})
    public void listen(final HashMap<String, Object> record, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {

        try {

            BirthRegistrationRequest request = mapper.convertValue(record, BirthRegistrationRequest.class);
            //log.info(request.toString());
            notificationService.prepareEventAndSend(request);

        } catch (final Exception e) {

            log.error("Error while listening to value: " + record + " on topic: " + topic + ": ", e);
        }
    }

}

Create a POJO by the name of SMSRequest in the web.models package and add the following content to it:

package digit.web.models;

import lombok.*;

@Getter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
public class SMSRequest {
    private String mobileNumber;
    private String message;
}

Create a class by the name of NotificationService under service folder to handle preparation of customised messages and pushing the notifications.
Add the following content to it -

package digit.service;

import digit.config.BTRConfiguration;
import digit.kafka.Producer;
import digit.web.models.BirthRegistrationApplication;
import digit.web.models.BirthRegistrationRequest;
import digit.web.models.SMSRequest;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import java.util.ArrayList;
import java.util.List;
@Slf4j
@Service
public class NotificationService {

    @Autowired
    private Producer producer;

    @Autowired
    private BTRConfiguration config;

    @Autowired
    private RestTemplate restTemplate;

    private static final String smsTemplate = "Dear {FATHER_NAME} and {MOTHER_NAME} your birth registration application has been successfully created on the system with application number - {APPNUMBER}.";

    public void prepareEventAndSend(BirthRegistrationRequest request){
        List<SMSRequest> smsRequestList = new ArrayList<>();
        request.getBirthRegistrationApplications().forEach(application -> {
            SMSRequest smsRequestForFather = SMSRequest.builder().mobileNumber(application.getFatherMobileNumber()).message(getCustomMessage(smsTemplate, application)).build();
            SMSRequest smsRequestForMother = SMSRequest.builder().mobileNumber(application.getMotherMobileNumber()).message(getCustomMessage(smsTemplate, application)).build();
            smsRequestList.add(smsRequestForFather);
            smsRequestList.add(smsRequestForMother);
        });
        for (SMSRequest smsRequest : smsRequestList) {
            producer.push(config.getSmsNotificationTopic(), smsRequest);
            log.info("Messages: " + smsRequest.getMessage());
        }
    }

    private String getCustomMessage(String template, BirthRegistrationApplication application) {
        template = template.replace("{APPNUMBER}", application.getApplicationNumber());
        template = template.replace("{FATHER_NAME}", application.getFather().getName());
        template = template.replace("{MOTHER_NAME}", application.getMother().getName());
        return template;
    }

}

Add Persister Configuration

Overview

This page provides the steps on how to add persister configuration.

The persister configuration is written in a YAML format. The INSERT and UPDATE queries for each table are added in prepared statement format, followed by the jsonPaths of values that have to be inserted/updated.

For example, for a table named studentinfo with id, name, age, and marks fields, the following configuration will get the persister ready to insert data into studentinfo table -

serviceMaps:
 serviceName: student-management-service
 mappings:
 - version: 1.0
   description: Persists student details in studentinfo table
   fromTopic: save-student-info
   isTransaction: true
   queryMaps:
       - query: INSERT INTO studentinfo( id, name, age, marks) VALUES (?, ?, ?, ?);
         basePath: Students.*
         jsonMaps:
          - jsonPath: $.Students.*.id

          - jsonPath: $.Students.*.name

          - jsonPath: $.Students.*.age

          - jsonPath: $.Students.*.marks

Steps

Add Persister Configuration

Fork the configs repo. Ignore if done already. Clone configs repo in the local environment.

git clone -o upstream https://github.com/<your_configs_repo>/configs

Persister configurations for all modules are present under the egov-persister folder. Create a file by the name of btr-persister.yml (or any other name) under the egov-persister folder.
Add the following content to it -

serviceMaps:
  serviceName: btr-services
  mappings:
    - version: 1.0
      description: Persists birth details in tables
      fromTopic: save-bt-application
      isTransaction: true	
      queryMaps:

        - query: INSERT INTO eg_bt_registration(id,tenantid,applicationnumber,babyfirstname,babylastname,fatherid,motherid,doctorname,hospitalname,placeofbirth,timeofbirth,createdby,lastmodifiedby,createdtime, lastmodifiedtime) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,?,?,?);
          basePath: BirthRegistrationApplications.*
          jsonMaps:
            - jsonPath: $.BirthRegistrationApplications.*.id

            - jsonPath: $.BirthRegistrationApplications.*.tenantId

            - jsonPath: $.BirthRegistrationApplications.*.applicationNumber
          
            - jsonPath: $.BirthRegistrationApplications.*.babyFirstName
            
            - jsonPath: $.BirthRegistrationApplications.*.babyLastName
          
            - jsonPath: $.BirthRegistrationApplications.*.fatherOfApplicant.id
            
            - jsonPath: $.BirthRegistrationApplications.*.motherOfApplicant.id
                                           
            - jsonPath: $.BirthRegistrationApplications.*.doctorName
            
            - jsonPath: $.BirthRegistrationApplications.*.hospitalName
            
            - jsonPath: $.BirthRegistrationApplications.*.placeOfBirth

            - jsonPath: $.BirthRegistrationApplications.*.timeOfBirth
            
            - jsonPath: $.BirthRegistrationApplications.*.auditDetails.createdBy

            - jsonPath: $.BirthRegistrationApplications.*.auditDetails.lastModifiedBy

            - jsonPath: $.BirthRegistrationApplications.*.auditDetails.createdTime
            
            - jsonPath: $.BirthRegistrationApplications.*.auditDetails.lastModifiedTime

        - query: INSERT INTO eg_bt_address(id, tenantid, doorno, latitude, longitude, buildingname, addressid, addressnumber, type, addressline1, addressline2, landmark, street, city, locality, pincode, detail, registrationid, createdby, lastmodifiedby, createdtime, lastmodifiedtime) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);
          basePath: BirthRegistrationApplications.*
          jsonMaps:
            - jsonPath: $.BirthRegistrationApplications.*.address.id

            - jsonPath: $.BirthRegistrationApplications.*.address.tenantId

            - jsonPath: $.BirthRegistrationApplications.*.address.doorNo

            - jsonPath: $.BirthRegistrationApplications.*.address.latitude

            - jsonPath: $.BirthRegistrationApplications.*.address.longitude

            - jsonPath: $.BirthRegistrationApplications.*.address.buildingName

            - jsonPath: $.BirthRegistrationApplications.*.address.addressId

            - jsonPath: $.BirthRegistrationApplications.*.address.addressNumber

            - jsonPath: $.BirthRegistrationApplications.*.address.type

            - jsonPath: $.BirthRegistrationApplications.*.address.addressLine1

            - jsonPath: $.BirthRegistrationApplications.*.address.addressLine2

            - jsonPath: $.BirthRegistrationApplications.*.address.landmark

            - jsonPath: $.BirthRegistrationApplications.*.address.street

            - jsonPath: $.BirthRegistrationApplications.*.address.city

            - jsonPath: $.BirthRegistrationApplications.*.address.locality.name

            - jsonPath: $.BirthRegistrationApplications.*.address.pincode

            - jsonPath: $.BirthRegistrationApplications.*.address.detail

            - jsonPath: $.BirthRegistrationApplications.*.id

            - jsonPath: $.BirthRegistrationApplications.*.address.createdBy

            - jsonPath: $.BirthRegistrationApplications.*.address.lastModifiedBy

            - jsonPath: $.BirthRegistrationApplications.*.address.createdTime

            - jsonPath: $.BirthRegistrationApplications.*.address.lastModifiedTime

    - version: 1.0
      description: Update birth registration applications in table
      fromTopic: update-bt-application
      isTransaction: true
      queryMaps:
        - query: UPDATE eg_bt_registration SET tenantid = ?,babyFirstName = ?, timeOfBirth = ? WHERE id=?;
          basePath: BirthRegistrationApplications.*
          jsonMaps:
            - jsonPath: $.BirthRegistrationApplications.*.tenantId

            - jsonPath: $.BirthRegistrationApplications.*.babyFirstName
           
            - jsonPath: $.BirthRegistrationApplications.*.timeOfBirth

            - jsonPath: $.BirthRegistrationApplications.*.id

Run Persister Locally

Import all the core-services projects as Maven projects into your IDE. It is assumed that you have already cloned the DIGIT code locally.
Modify the application.properties file in the egov-persister project and set the following property:

egov.persist.yml.repo.path=file:///Users/subha/Code/configs/egov-persister/btr-persister.yml

Note: You can set a comma-separated list of files as the value of the above property. If you are running multiple services locally, then this has to be a comma-separated list of persister config files. Make sure you always give the absolute path.

Make sure the Spring DB configurations and Flyway config reflect the same database as what has been set in the module itself. Otherwise, we will see failures in the persister code.

spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/birthregistration3
spring.datasource.username=yourusername
spring.datasource.password=yourpassword

Make sure the Kafka is running locally. Now, go ahead and run the EgovPersistApplication from the IDE. Check the console to make sure it is listening to the right topics as configured in your module's application.properties file.

The persister is now ready for use.

Deploy Persister Configuration

Note: Below steps are for when you deploy your code to the DIGIT env, not for local development. You may choose to do this when you build and deploy.

Push the code to the appropriate branch from which your environment will read it.
Navigate to your fork of the DIGIT-DevOps repository. Under the deploy-as-code/helm/environments directory, find the deployment helm chart that was used to deploy DIGIT.
In the deployment helm chart (which was used to set up the DIGIT environment), find "egov-persister". Find the "persist-yml-path" property and add the path to your new persister file here.
In the snippet below, file:///work-dir/configs/egov-persister/birth-module-developer-guide.yml

egov-persister:
  replicas: 6
  persist-yml-path: "file:///work-dir/configs/egov-persister/privacy-audit.yml,file:///work-dir/configs/egov-persister/pgr-migration-batch.yml,file:///work-dir/configs/egov-persister/pgr-services-persister.yml,file:///work-dir/configs/egov-persister/pdf-filestoreid-update.yml,file:///work-dir/configs/egov-persister/chatbot.yml,file:///work-dir/configs/egov-persister/pt-mutation-calculator-persister.yml,file:///work-dir/configs/egov-persister/apportion-persister.yml,file:///work-dir/configs/egov-persister/property-services-registry.yml,file:///work-dir/configs/egov-persister/billing-services-persist.yml,file:///work-dir/configs/egov-persister/egf-bill.yaml,file:///work-dir/configs/egov-persister/egov-user-event-persister.yml,file:///work-dir/configs/egov-persister/egov-workflow-v2-persister.yml,file:///work-dir/configs/egov-persister/firenoc_persiter.yaml,file:///work-dir/configs/egov-persister/hrms-employee-persister.yml,file:///work-dir/configs/egov-persister/pdf-generator.yml,file:///work-dir/configs/egov-persister/pg-service-persister.yml,file:///work-dir/configs/egov-persister/pgr.v3.yml,file:///work-dir/configs/egov-persister/property-services.yml,file:///work-dir/configs/egov-persister/pt-calculator-v2-persister.yml,file:///work-dir/configs/egov-persister/pt-drafts.yml,file:///work-dir/configs/egov-persister/pt-persist.yml,file:///work-dir/configs/egov-persister/tl-billing-slab-persister.yml,file:///work-dir/configs/egov-persister/tl-calculation-persister.yml,file:///work-dir/configs/egov-persister/tradelicense.yml,file:///work-dir/configs/egov-persister/uploader-persister.yml,file:///work-dir/configs/egov-persister/collection-migration-persister.yml,file:///work-dir/configs/egov-persister/water-persist.yml,file:///work-dir/configs/egov-persister/water-meter.yml,file:///work-dir/configs/egov-persister/assessment-persister.yml,file:///work-dir/configs/egov-persister/sewerage-persist.yml,file:///work-dir/configs/egov-persister/bpa-persister.yml,file:///work-dir/configs/egov-persister/property-services-migration-temp-config.yml,file:///work-dir/configs/egov-persister/assessment-persister-migration-temp.yml,file:///work-dir/configs/egov-persister/migration-batch-count-persister.yml,file:///work-dir/configs/egov-persister/land-persister.yml,file:///work-dir/configs/egov-persister/noc-persister.yml,file:///work-dir/configs/egov-persister/fsm-persister.yaml,file:///work-dir/configs/egov-persister/vehicle-persister.yaml,file:///work-dir/configs/egov-persister/vendor-persister.yaml,file:///work-dir/configs/egov-persister/fsm-calculator-persister.yaml,file:///work-dir/configs/egov-persister/echallan.yml,file:///work-dir/configs/egov-persister/egov-document-upload-persister.yml,file:///work-dir/configs/egov-persister/egov-survey-service-persister.yml,file:///work-dir/configs/egov-persister/firenoc-calculator-persister.yml,file:///work-dir/configs/egov-persister/bulk-bill-generation-audit.yml,file:///work-dir/configs/egov-persister/nss-persister.yml,file:///work-dir/configs/egov-persister/birth-death.yml,file:///work-dir/configs/egov-persister/bulk-bill-generator-ws.yml,file:///work-dir/configs/egov-persister/bulk-bill-generator-sw.yml,file:///work-dir/configs/egov-persister/audit-service-persister.yml,file:///work-dir/configs/egov-persister/birth-module-developer-guide.yml"

Raise a PR to the appropriate branch of the DevOps repo (master, in egov case) which was forked/used to create the deployment. Once that is merged, restart the indexer service in your environment so it will pick up this new config for the module.

Enable Signed Audit

Integration with signed audit

Overview

Enabling signed audit for a module ensures that all transactions - creates, updates, deletes - are recorded in a digitally signed fashion. Learn more about the signed audit service here.

Enabled signed audit is optional but highly recommended to ensure data security.

Steps

Enable Signed Audit

Add the following lines of code to the birth registration persister after the fromTopic attribute under mappings:

isTransaction: true
isAuditEnabled: true
module: BTR
objecIdJsonPath: $.id
tenantIdJsonPath: $.tenantId
transactionCodeJsonPath: $.applicationNumber
auditAttributeBasePath: $.BirthRegistrationApplications

Run Application

Overview

We have completed integration with key services and we can now test out the REST APIs we have built.

Steps

Run & Test Sample Application

Ensure that Kafka and Persister are running in the local environment.
Run birth-registration-service that we just created.
Import the postman collection of the APIs that this sample service exposes from here: Birth Registration Postman Collection
Test the following APIs:
- Hit the _create API request to create a birth registration application. You should see entries in the local database.
- Hit the _search API request to search for the created birth registration applications.
- Hit _update API request to update your application by changing the baby's firstName field. You should be able to see the updates in the DB.

Section 3: Integrate Microservices

Integration with other DIGIT services

Overview

A separate class should be created for integrating with each dependent microservice. Only one method from that class should be called from the main service class for integration.

This guide showcases the steps to integrate our microservices with other microservices like

IdGen Service
User Service
MDMS Service
Workflow Service
URL Shortening Service

Common Resources

Code for developer guide till this stage is available here. Postman collection corresponding to this stage is available here.

For interacting with other microservices, we can create and implement the following ServiceRequestRepository class under repository package -

package digit.repository;


import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import lombok.extern.slf4j.Slf4j;
import org.egov.tracer.model.ServiceCallException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Repository;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.client.RestTemplate;

import java.util.Map;


@Repository
@Slf4j
public class ServiceRequestRepository {

    private ObjectMapper mapper;

    private RestTemplate restTemplate;


    @Autowired
    public ServiceRequestRepository(ObjectMapper mapper, RestTemplate restTemplate) {
        this.mapper = mapper;
        this.restTemplate = restTemplate;
    }


    public Object fetchResult(StringBuilder uri, Object request) {
        mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
        Object response = null;
        try {
            response = restTemplate.postForObject(uri.toString(), request, Map.class);
        }catch(HttpClientErrorException e) {
            log.error("External Service threw an Exception: ",e);
            throw new ServiceCallException(e.getResponseBodyAsString());
        }catch(Exception e) {
            log.error("Exception while fetching from searcher: ",e);
        }

        return response;
    }
}

Integrate IDGen Service

Describes how to integrate with DIGIT's ID Gen service

Overview

This page provides the steps to integrate with the IDGen Service. Each application needs to have a unique ID. The IDGen service generates these unique IDs. ID format can be customised via configuration in MDMS.

Steps

Add the ID format that needs to be generated in this file - Id Format Mdms File. The following config has been added for this module:

{
"format": "PB-BTR-[cy:yyyy-MM-dd]-[SEQ_EG_BTR_ID]",
"idname": "btr.registrationid"
}

Restart the IDGen service and MDMS service and port-forward IDGen service to port 8285:

kubectl port-forward <IDGEN_SERVICE_POD_NAME> 8285:8080

Note that you can set the ID format in the application.properties file of IDGen service and run the service locally if you don't have access to a DIGIT environment.

Hit the below curl to verify that the format is added properly. The "ID" name needs to match exactly with what was added in MDMS.

curl --location --request POST 'http://localhost:8285/egov-idgen/id/_generate' \
--header 'Content-Type: application/json' \
--data-raw '{
    "RequestInfo": {
        "apiId": "string",
        "ver": "string",
        "ts": null,
        "action": "string",
        "did": "string",
        "key": "string",
        "msgId": "string",
        "authToken": "6456b2cf-49ca-47c7-b7b6-c179f19614c7",
        "correlationId": "e721639b-c095-40b3-86e2-acecb2cb6efb",
        "userInfo": {
            "id": 23299,
            "uuid": "e721639b-c095-40b3-86e2-acecb2cb6efb",
            "userName": "9337682030",
            "name": "Abhilash Seth",
            "type": "EMPLOYEE",
            "mobileNumber": "9337682030",
            "emailId": "abhilash.seth@gmail.com",
            "roles": [
                {
                    "id": 281,
                    "name": "Employee"
                }
            ]
        }
    },
    "idRequests": [
        {
            "tenantId": "pb.amritsar",
            "idName": "btr.registrationid"
        }
    ]
}'

Once verified, we can call the ID generation service from within our application and generate the registrationId.
Add the following model POJOs under models folder:
IdGenerationRequest.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.egov.common.contract.request.RequestInfo;

import java.util.List;

@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class IdGenerationRequest {

    @JsonProperty("RequestInfo")
    private RequestInfo requestInfo;

    private List<IdRequest> idRequests;

}

IdGenerationResponse.java

package digit.web.models;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.egov.common.contract.response.ResponseInfo;

import java.util.List;


@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class IdGenerationResponse {

    private ResponseInfo responseInfo;

    private List<IdResponse> idResponses;

}

IdRequest.java

package digit.web.models;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

import javax.validation.constraints.NotNull;


@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class IdRequest {

    @JsonProperty("idName")
    @NotNull
    private String idName;

    @NotNull
    @JsonProperty("tenantId")
    private String tenantId;

    @JsonProperty("format")
    private String format;

}

IdResponse.java

package digit.web.models;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;


@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class IdResponse {

    private String id;

}

In the BirthApplicationEnrichment class, update the enrichBirthApplication method as shown below:

public void enrichBirthApplication(BirthRegistrationRequest birthRegistrationRequest) {
        //Retrieve list of IDs from IDGen service
        List<String> birthRegistrationIdList = idgenUtil.getIdList(birthRegistrationRequest.getRequestInfo(), birthRegistrationRequest.getBirthRegistrationApplications().get(0).getTenantId(), "btr.registrationid", "", birthRegistrationRequest.getBirthRegistrationApplications().size());
        Integer index = 0;
        for(BirthRegistrationApplication application : birthRegistrationRequest.getBirthRegistrationApplications()) {
            // Enrich audit details
            AuditDetails auditDetails = AuditDetails.builder().createdBy(birthRegistrationRequest.getRequestInfo().getUserInfo().getUuid()).createdTime(System.currentTimeMillis()).lastModifiedBy(birthRegistrationRequest.getRequestInfo().getUserInfo().getUuid()).lastModifiedTime(System.currentTimeMillis()).build();
            application.setAuditDetails(auditDetails);

            // Enrich UUID
            application.setId(UUID.randomUUID().toString());

            // Set application number from IdGen
            application.setApplicationNumber(birthRegistrationIdList.get(index++));
            
            // Enrich registration Id
            application.getAddress().setRegistrationId(application.getId());

            // Enrich address UUID
            application.getAddress().setId(UUID.randomUUID().toString());
        }
    }

Make sure below ID generation host configuration is present in the application.properties file. Make sure to fill in the correct values for the host.

#Idgen Config
egov.idgen.host=http://localhost:8285/ #REPLACE
egov.idgen.path=egov-idgen/id/_generate

Integrate User Service

Overview

The user service provides the capabilities of creating a user, searching for a user and retrieving the details of a user. This module will search for a user and if not found, create that user with the user service.

DIGIT's user service masks PII that gets stored in the database using the encryption service.

Steps

Create the following POJOs under the model directory:

CreateUserRequest.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import org.egov.common.contract.request.RequestInfo;

@AllArgsConstructor
@Getter
@NoArgsConstructor
public class CreateUserRequest {

    @JsonProperty("requestInfo")
    private RequestInfo requestInfo;

    @JsonProperty("user")
    private User user;

}

UserSearchRequest.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Getter;
import lombok.Setter;
import org.egov.common.contract.request.RequestInfo;

import java.util.Collections;
import java.util.List;


@Getter
@Setter
public class UserSearchRequest {

    @JsonProperty("RequestInfo")
    private RequestInfo requestInfo;

    @JsonProperty("uuid")
    private List<String> uuid;

    @JsonProperty("id")
    private List<String> id;

    @JsonProperty("userName")
    private String userName;

    @JsonProperty("name")
    private String name;

    @JsonProperty("mobileNumber")
    private String mobileNumber;

    @JsonProperty("aadhaarNumber")
    private String aadhaarNumber;

    @JsonProperty("pan")
    private String pan;

    @JsonProperty("emailId")
    private String emailId;

    @JsonProperty("fuzzyLogic")
    private boolean fuzzyLogic;

    @JsonProperty("active")
    @Setter
    private Boolean active;

    @JsonProperty("tenantId")
    private String tenantId;

    @JsonProperty("pageSize")
    private int pageSize;

    @JsonProperty("pageNumber")
    private int pageNumber = 0;

    @JsonProperty("sort")
    private List<String> sort = Collections.singletonList("name");

    @JsonProperty("userType")
    private String userType;

    @JsonProperty("roleCodes")
    private List<String> roleCodes;

}

UserDetailResponse.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.ToString;
import org.egov.common.contract.response.ResponseInfo;

import java.util.List;

@AllArgsConstructor
@NoArgsConstructor
@Getter
@ToString
public class UserDetailResponse {
    @JsonProperty("responseInfo")
    ResponseInfo responseInfo;

    @JsonProperty("user")
    List<User> user;
}

Create a class by the name of UserService under service folder and add the following content to it:

UserService.java

package digit.service;

import digit.config.BTRConfiguration;
import digit.utils.UserUtil;
import digit.web.models.*;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang.StringUtils;
import org.egov.common.contract.request.RequestInfo;
import org.egov.tracer.model.CustomException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.CollectionUtils;

import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;


@Service
@Slf4j
public class UserService {
    private UserUtil userUtils;

    private BTRConfiguration config;

    @Autowired
    public UserService(UserUtil userUtils, BTRConfiguration config) {
        this.userUtils = userUtils;
        this.config = config;
    }

    /**
     * Calls user service to enrich user from search or upsert user
     * @param request
     */
    public void callUserService(BirthRegistrationRequest request){
        request.getBirthRegistrationApplications().forEach(application -> {
            if(!StringUtils.isEmpty(application.getFather().getId()))
                enrichUser(application, request.getRequestInfo());
            else {
                User user = createFatherUser(application);
                application.getFather().setId(upsertUser(user, request.getRequestInfo()));
            }
        });

        request.getBirthRegistrationApplications().forEach(application -> {
            if(!StringUtils.isEmpty(application.getMother().getId()))
                enrichUser(application, request.getRequestInfo());
            else {
                User user = createMotherUser(application);
                application.getMother().setId(upsertUser(user, request.getRequestInfo()));
            }
        });
    }

    private User createFatherUser(BirthRegistrationApplication application){
        FatherApplicant father = application.getFather();
        User user = User.builder().userName(father.getUserName())
                .name(father.getName())
                .mobileNumber(father.getMobileNumber())
                .emailId(father.getEmailId())
                .altContactNumber(father.getAltContactNumber())
                .tenantId(father.getTenantId())
                .type(father.getType())
                .roles(father.getRoles())
                .build();
//        String tenantId = father.getTenantId();
        return user;
    }

    private User createMotherUser(BirthRegistrationApplication application){
        MotherApplicant mother = application.getMother();
        User user = User.builder().userName(mother.getUserName())
                .name(mother.getName())
                .mobileNumber(mother.getMobileNumber())
                .emailId(mother.getEmailId())
                .altContactNumber(mother.getAltContactNumber())
                .tenantId(mother.getTenantId())
                .type(mother.getType())
                .roles(mother.getRoles())
                .build();
//        String tenantId = father.getTenantId();
        return user;
    }
    private String upsertUser(User user, RequestInfo requestInfo){

        String tenantId = user.getTenantId();
        User userServiceResponse = null;

        // Search on mobile number as user name
        UserDetailResponse userDetailResponse = searchUser(userUtils.getStateLevelTenant(tenantId),null, user.getMobileNumber());
        if (!userDetailResponse.getUser().isEmpty()) {
            User userFromSearch = userDetailResponse.getUser().get(0);
            log.info(userFromSearch.toString());
            if(!user.getUserName().equalsIgnoreCase(userFromSearch.getUserName())){
                userServiceResponse = updateUser(requestInfo,user,userFromSearch);
            }
            else userServiceResponse = userDetailResponse.getUser().get(0);
        }
        else {
            userServiceResponse = createUser(requestInfo,tenantId,user);
        }

        // Enrich the accountId
       // user.setId(userServiceResponse.getUuid());
        return userServiceResponse.getUuid();
    }


    private void enrichUser(BirthRegistrationApplication application, RequestInfo requestInfo){
        String accountIdFather = application.getFather().getId();
        String accountIdMother = application.getMother().getId();
        String tenantId = application.getTenantId();

        UserDetailResponse userDetailResponseFather = searchUser(userUtils.getStateLevelTenant(tenantId),accountIdFather,null);
        UserDetailResponse userDetailResponseMother = searchUser(userUtils.getStateLevelTenant(tenantId),accountIdMother,null);
        if(userDetailResponseFather.getUser().isEmpty())
            throw new CustomException("INVALID_ACCOUNTID","No user exist for the given accountId");

        else application.getFather().setId(userDetailResponseFather.getUser().get(0).getUuid());

        if(userDetailResponseMother.getUser().isEmpty())
            throw new CustomException("INVALID_ACCOUNTID","No user exist for the given accountId");

        else application.getMother().setId(userDetailResponseMother.getUser().get(0).getUuid());

    }

    /**
     * Creates the user from the given userInfo by calling user service
     * @param requestInfo
     * @param tenantId
     * @param userInfo
     * @return
     */
    private User createUser(RequestInfo requestInfo,String tenantId, User userInfo) {

        userUtils.addUserDefaultFields(userInfo.getMobileNumber(),tenantId, userInfo);
        StringBuilder uri = new StringBuilder(config.getUserHost())
                .append(config.getUserContextPath())
                .append(config.getUserCreateEndpoint());

        CreateUserRequest user = new CreateUserRequest(requestInfo, userInfo);
        log.info(user.getUser().toString());
        UserDetailResponse userDetailResponse = userUtils.userCall(user, uri);

        return userDetailResponse.getUser().get(0);

    }

    /**
     * Updates the given user by calling user service
     * @param requestInfo
     * @param user
     * @param userFromSearch
     * @return
     */
    private User updateUser(RequestInfo requestInfo,User user,User userFromSearch) {

        userFromSearch.setName(user.getName());
        userFromSearch.setActive(true);

        StringBuilder uri = new StringBuilder(config.getUserHost())
                .append(config.getUserContextPath())
                .append(config.getUserUpdateEndpoint());


        UserDetailResponse userDetailResponse = userUtils.userCall(new CreateUserRequest(requestInfo, userFromSearch), uri);

        return userDetailResponse.getUser().get(0);

    }

    /**
     * calls the user search API based on the given accountId and userName
     * @param stateLevelTenant
     * @param accountId
     * @param userName
     * @return
     */
    public UserDetailResponse searchUser(String stateLevelTenant, String accountId, String userName){

        UserSearchRequest userSearchRequest =new UserSearchRequest();
        userSearchRequest.setActive(true);
        userSearchRequest.setUserType("CITIZEN");
        userSearchRequest.setTenantId(stateLevelTenant);

        if(StringUtils.isEmpty(accountId) && StringUtils.isEmpty(userName))
            return null;

        if(!StringUtils.isEmpty(accountId))
            userSearchRequest.setUuid(Collections.singletonList(accountId));

        if(!StringUtils.isEmpty(userName))
            userSearchRequest.setUserName(userName);

        StringBuilder uri = new StringBuilder(config.getUserHost()).append(config.getUserSearchEndpoint());
        return userUtils.userCall(userSearchRequest,uri);

    }

    /**
     * calls the user search API based on the given list of user uuids
     * @param uuids
     * @return
     */
    private Map<String,User> searchBulkUser(List<String> uuids){

        UserSearchRequest userSearchRequest =new UserSearchRequest();
        userSearchRequest.setActive(true);
        userSearchRequest.setUserType("CITIZEN");


        if(!CollectionUtils.isEmpty(uuids))
            userSearchRequest.setUuid(uuids);


        StringBuilder uri = new StringBuilder(config.getUserHost()).append(config.getUserSearchEndpoint());
        UserDetailResponse userDetailResponse = userUtils.userCall(userSearchRequest,uri);
        List<User> users = userDetailResponse.getUser();

        if(CollectionUtils.isEmpty(users))
            throw new CustomException("USER_NOT_FOUND","No user found for the uuids");

        Map<String,User> idToUserMap = users.stream().collect(Collectors.toMap(User::getUuid, Function.identity()));

        return idToUserMap;
    }

}

Changes to BirthApplicationEnrichment.java

Add the below methods to the enrichment class we created. When we search for an application, the code below will search for the users associated with the application and add in their details to the response object.

enrichFatherApplicantOnSearch

public void enrichFatherApplicantOnSearch(BirthRegistrationApplication application) {
        UserDetailResponse fatherUserResponse = userService.searchUser(userUtils.getStateLevelTenant(application.getTenantId()),application.getFather().getId(),null);
        User fatherUser = fatherUserResponse.getUser().get(0);
        log.info(fatherUser.toString());
        FatherApplicant fatherApplicant = FatherApplicant.builder().aadhaarNumber(fatherUser.getAadhaarNumber())
                .accountLocked(fatherUser.getAccountLocked())
                .active(fatherUser.getActive())
                .altContactNumber(fatherUser.getAltContactNumber())
                .bloodGroup(fatherUser.getBloodGroup())
                .correspondenceAddress(fatherUser.getCorrespondenceAddress())
                .correspondenceCity(fatherUser.getCorrespondenceCity())
                .correspondencePincode(fatherUser.getCorrespondencePincode())
                .gender(fatherUser.getGender())
                .id(fatherUser.getUuid())
                .name(fatherUser.getName())
                .type(fatherUser.getType())
                .roles(fatherUser.getRoles()).build();
        application.setFather(fatherApplicant);
    }

    public void enrichMotherApplicantOnSearch(BirthRegistrationApplication application) {
        UserDetailResponse motherUserResponse = userService.searchUser(userUtils.getStateLevelTenant(application.getTenantId()),application.getMother().getId(),null);
        User motherUser = motherUserResponse.getUser().get(0);
        log.info(motherUser.toString());
        MotherApplicant motherApplicant = MotherApplicant.builder().aadhaarNumber(motherUser.getAadhaarNumber())
                .accountLocked(motherUser.getAccountLocked())
                .active(motherUser.getActive())
                .altContactNumber(motherUser.getAltContactNumber())
                .bloodGroup(motherUser.getBloodGroup())
                .correspondenceAddress(motherUser.getCorrespondenceAddress())
                .correspondenceCity(motherUser.getCorrespondenceCity())
                .correspondencePincode(motherUser.getCorrespondencePincode())
                .gender(motherUser.getGender())
                .id(motherUser.getUuid())
                .name(motherUser.getName())
                .type(motherUser.getType())
                .roles(motherUser.getRoles()).build();
        application.setMother(motherApplicant);
    }

Changes to BirthRegistrationService.java

Add in a userService object:

 @Autowired
 private UserService userService;

And enhance the following two methods in BirthRegistrationService.java:

registerBtRequest

public List<BirthRegistrationApplication> registerBtRequest(BirthRegistrationRequest birthRegistrationRequest) {
        birthApplicationValidator.validateBirthApplication(birthRegistrationRequest);
        birthApplicationEnrichment.enrichBirthApplication(birthRegistrationRequest);

        // Enrich/Upsert user in upon birth registration
        userService.callUserService(birthRegistrationRequest);

        // Push the application to the topic for persister to listen and persist
        producer.push(configuration.getCreateTopic(), birthRegistrationRequest);

        // Return the response back to user
        return birthRegistrationRequest.getBirthRegistrationApplications();
    }

searchBtApplications

 public List<BirthRegistrationApplication> searchBtApplications(RequestInfo requestInfo, BirthApplicationSearchCriteria birthApplicationSearchCriteria) {
        // Fetch applications from database according to the given search criteria
        List<BirthRegistrationApplication> applications = birthRegistrationRepository.getApplications(birthApplicationSearchCriteria);

        // If no applications are found matching the given criteria, return an empty list
        if(CollectionUtils.isEmpty(applications))
            return new ArrayList<>();

        // Enrich mother and father of applicant objects
        applications.forEach(application -> {
            birthApplicationEnrichment.enrichFatherApplicantOnSearch(application);
            birthApplicationEnrichment.enrichMotherApplicantOnSearch(application);
        });

        // Otherwise, return the found applications
        return applications;
    }

Add the following properties in application.properties file:

Note that if you are port-forwarding using k8s, you will use localhost. Else, if you have a valid auth token, please provide the name of the host here.

#User config
egov.user.host=http://localhost:8284/
egov.user.context.path=/user/users
egov.user.create.path=/_createnovalidate
egov.user.search.path=/user/_search
egov.user.update.path=/_updatenovalidate

Add MDMS Configuration

Overview

MDMS data is the master data used by the application. New modules with master data need to be configured inside the /data/<tenant>/ folder of the MDMS repo. Each tenant should have a unique ID and sub-tenants can be configured in the format state.cityA, state.cityB etc..Further hierarchies are also possible with tenancy.

If you already have a DIGIT environment configured with a tenant and CITIZEN/EMPLOYEE roles, that is sufficient data to get this module running locally. Configuring role-action mapping is necessary during deployment of the app in the DIGIT environment. It will not be needed to run the application locally.

For more information on MDMS see here. To read about how to design for MDMS, please see the design guide.

In the birth registration use case, we use the following master data:

tenantId = "pb"
User roles - CITIZEN and EMPLOYEE roles configured in roles.json (see below section for more info)
Actions - URIs to be exposed via Zuul (see below section for more info)
Role-action mapping - for access control (see below section for more info)

Make sure to add data to the correct branch of the MDMS repository. i.e. if you have setup CD CI to deploy the DEV branch of the repository to the dev environment (default), then make sure to add the information in the DEV branch. If you are testing in staging or some other environment, make sure to add the master data to the corresponding branch of MDMS.

Steps

Create a folder called "pb" in the data folder of the MDMS repository "DEV" branch. You will have a new folder path as follows:

<MDMS repo URL path>/data/pb

Restart the MDMS service in the development environment where DIGIT is running once data is added to the MDMS repository. This loads the newly added/updated MDMS configs.

A sample MDMS config file can be viewed here - Sample MDMS data file.

API Access Control Configuration

URIs (actions), roles and URI-role mapping will be defined in MDMS. These will apply when the module is deployed into an environment where Zuul is involved (not while locally running the app). In this sample app, we have used "pb" as a tenantId. In your environment, you can choose to define a new one or work with an existing one.

All folders mentioned below need to be created under the data/pb folder in MDMS.

You can choose to use some other tenantId. Make sure to change the tenant ID everywhere.

Actions

Actions need to be defined inside the /data/pb/ACCESS-CONTROL-ACTIONS/actions.json file. Below are the actions for the birth registration module. Append this to the bottom of the actions.json file. Make sure the "id" field in the JSON is incremented. It needs to be unique in your environment.

// Some code
 {
      "id": 2382,
      "name": "Birth registration module create",
      "url": "/birth-registration/birth-services/v1/registration/_create",
      "parentModule": "",
      "displayName": "Birth registration",
      "orderNumber": 0,
      "enabled": true,
      "serviceCode": "BTR",
      "code": "null",
      "path": ""
    },
  {
    "id": 2383,
    "name": "Birth registration module update",
    "url": "/birth-registration/birth-services/v1/registration/_update",
    "parentModule": "",
    "displayName": "Birth registration",
    "orderNumber": 0,
    "enabled": true,
    "serviceCode": "BTR",
    "code": "null",
    "path": ""
  },
  {
    "id": 2384,
    "name": "Birth registration module search",
    "url": "/birth-registration/birth-services/v1/registration/_search",
    "parentModule": "",
    "displayName": "Birth registration",
    "orderNumber": 0,
    "enabled": true,
    "serviceCode": "BTR",
    "code": "null",
    "path": ""
  }

Note that the IDs in the actions.json config are generated manually.

Roles Configuration

Roles config happens at a state level. For birth registration, we need only CITIZEN and EMPLOYEE roles in the /data/pb/ACCESSCONTROL-ROLES/roles.json file.Here are some sample roles that can be defined in an environment. If these roles are already present in the file, then there is no need to add them in again.

Role-action Mapping

Append the below code to the "roleactions" key in the /data/pb/ACCESSCONTROL-ROLEACTIONS/roleactions.json. Note that other role-action mappings may already be defined in your DIGIT environment. So please make sure to append the below. The actionid refers to the URI ID defined in the actions.json file.

    {
      "rolecode": "CITIZEN",
      "actionid": 2382,
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "CITIZEN",
      "actionid": 2383,
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "CITIZEN",
      "actionid": 2384,
      "actioncode": "",
      "tenantId": "pb"
    }

Integrate MDMS Service

Overview

We will call into MDMS deployed in the sandbox environment. All MDMS config data needs to be uploaded into the MDMS repository (DEV branch if you are deploying/testing in your dev environment).

Steps

Integration with MDMS requires the following steps to be followed:

Add a new MDMS file in MDMS repo. For this guide, a sample MDMS file has already been added available here. Copy this file into your repository.
Restart MDMS service after adding the new file via Jenkins build UI.
Once restarted, hit the curl mentioned below to verify that the new file has been properly added .

curl --location --request POST 'https://yourserver.digit.org/egov-mdms-service/v1/_search' \
--header 'Content-Type: application/json' \
--data-raw '{
    "RequestInfo": {
        "apiId": "asset-services",
        "ver": null,
        "ts": null,
        "action": null,
        "did": null,
        "key": null,
        "msgId": "search with from and to values",
        "authToken": "{{devAuth}}"
    },
    "MdmsCriteria": {
        "tenantId": "pb",
        "moduleDetails": [
            {
                "moduleName": "BTR",
                "masterDetails": [
                    {
                        "name": "RegistrationCharges"
                    }
                ]
            }
        ]
    }
}'

Call the MDMS service post verification from within our application and fetch the required master data. For this, create a Java class by the name of MdmsUtil under utils folder. Annotate this class with @Component and put the following content in the class -

package digit.utils;

import com.jayway.jsonpath.JsonPath;
import org.egov.common.contract.request.RequestInfo;
import org.egov.mdms.model.MasterDetail;
import org.egov.mdms.model.MdmsCriteria;
import org.egov.mdms.model.MdmsCriteriaReq;
import org.egov.mdms.model.ModuleDetail;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;


@Component
public class MdmsUtil {
    @Autowired
    private RestTemplate restTemplate;

    @Value("${egov.mdms.host}")
    private String mdmsHost;

    @Value("${egov.mdms.search.endpoint}")
    private String mdmsUrl;


    public Integer fetchRegistrationChargesFromMdms(RequestInfo requestInfo, String tenantId) {
        StringBuilder uri = new StringBuilder();
        uri.append(mdmsHost).append(mdmsUrl);
        MdmsCriteriaReq mdmsCriteriaReq = getMdmsRequestForCategoryList(requestInfo, tenantId);
        Object response = new HashMap<>();
        Integer rate = 0;
        try {
            response = restTemplate.postForObject(uri.toString(), mdmsCriteriaReq, Map.class);
            rate = JsonPath.read(response, "$.MdmsRes.BTR.RegistrationCharges.[0].amount");
        }catch(Exception e) {
            return  null;
        }
        return rate;
    }

    private MdmsCriteriaReq getMdmsRequestForCategoryList(RequestInfo requestInfo, String tenantId) {
        MasterDetail masterDetail = new MasterDetail();
        masterDetail.setName("RegistrationCharges");
        List<MasterDetail> masterDetailList = new ArrayList<>();
        masterDetailList.add(masterDetail);

        ModuleDetail moduleDetail = new ModuleDetail();
        moduleDetail.setMasterDetails(masterDetailList);
        moduleDetail.setModuleName("BTR");
        List<ModuleDetail> moduleDetailList = new ArrayList<>();
        moduleDetailList.add(moduleDetail);

        MdmsCriteria mdmsCriteria = new MdmsCriteria();
        mdmsCriteria.setTenantId(tenantId.split("\\.")[0]);
        mdmsCriteria.setModuleDetails(moduleDetailList);

        MdmsCriteriaReq mdmsCriteriaReq = new MdmsCriteriaReq();
        mdmsCriteriaReq.setMdmsCriteria(mdmsCriteria);
        mdmsCriteriaReq.setRequestInfo(requestInfo);

        return mdmsCriteriaReq;
    }
}

Add the following properties in application.properties file -

#mdms urls
egov.mdms.host=https://dev.digit.org # REPLACE with your environment name
egov.mdms.search.endpoint=/egov-mdms-service/v1/_search

Add Workflow Configuration

Overview

Workflow configuration should be created based on the business requirements. More details on extracting workflow in the design guide.

Steps

For our guide, we will configure the following workflow which was the output of the design phase:

We will re-use the workflow service which is deployed in the development/sandbox environment.

This guide assumes you can call the development environment workflow service directly with a valid auth token.

A sample curl is posted below. Make sure to replace the server hostname and the username and password in the below statement:

curl --location --request POST 'https://yourserver.digit.org/user/oauth/token' \
--header 'Authorization: Basic ZWdvdi11c2VyLWNsaWVudDo=' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'scope=read' \
--data-urlencode 'username=<mobile_no_of user>' \
--data-urlencode 'password=<pwd_you_created>' \
--data-urlencode 'tenantId=<your_tenant_id>' \
--data-urlencode 'userType=CITIZEN'

In POSTMan, create a new POST request and paste the below content in the body. The URL for the request is http://yourserver.digit.org/egov-workflow-v2/egov-wf/businessservice/_create to create the workflow.

Make sure to replace the authToken field in the body with appropriate auth token in your environment. Login to the server as a CITIZEN or EMPLOYEE user (depending on which one you've created) and obtain the authToken from the response body.

In DIGIT, the API Gateway (Zuul) enriches user information based on the auth token for all requests that go via the gateway. Port forwarding by-passes the API gateway. In this case, when accessing a service directly, for a request to be valid, a user has to send the userInfo JSON inside the RequestInfo object. This is true not just for Workflow but for any service. Sample:

"userInfo": {

"id": 24226,

"uuid": "11b0e02b-0145-4de2-bc42-c97b96264807",

"userName": "sample_user",

"roles": [

{"name": "Citizen", "code": "CITIZEN"}

]

}

Note that UUID and roles can be dummy place-holder entities in this case for local testing.

Below is the URL and POST body for the business service creation request.

http://yourserver.digit.org/egov-workflow-v2/egov-wf/businessservice/_create

{
   "RequestInfo": {
       "apiId": "Rainmaker",
       "action": "",
       "did": 1,
       "key": "",
       "msgId": "20170310130900|en_IN",
       "requesterId": "",
       "ts": 1513579888683,
       "ver": ".01",
       "authToken": "{{devAuth}}"
   },
   "BusinessServices": [
       {
           "tenantId": "pb",
           "businessService": "BTR",
           "business": "birth-registration",
           "businessServiceSla": 432000000,
           "states": [
               {
                   "sla": null,
                   "state": null,
                   "applicationStatus": null,
                   "docUploadRequired": true,
                   "isStartState": true,
                   "isTerminateState": false,
                   "isStateUpdatable": true,
                   "actions": [
                       {
                           "action": "APPLY",
                           "nextState": "APPLIED",
                           "roles": [
                               "CITIZEN",
                               "EMPLOYEE"
                           ]
                       }
                   ]
               },
               {
                   "sla": null,
                   "state": "APPLIED",
                   "applicationStatus": "APPLIED",
                   "docUploadRequired": false,
                   "isStartState": false,
                   "isTerminateState": true,
                   "isStateUpdatable": false,
                   "actions": [
                       {
                           "action": "APPROVE",
                           "nextState": "APPROVED",
                           "roles": [
                               "EMPLOYEE"
                           ]
                       },
                       {
                           "action": "REJECT",
                           "nextState": "REJECTED",
                           "roles": [
                               "EMPLOYEE"
                           ]
                       }
                   ]
               },
               {
                   "sla": null,
                   "state": "APPROVED",
                   "applicationStatus": "APPROVED",
                   "docUploadRequired": false,
                   "isStartState": false,
                   "isTerminateState": false,
                   "isStateUpdatable": false,
                   "actions": [
                       {
                           "action": "PAY",
                           "nextState": "REGISTRATIONCOMPLETED",
                           "roles": [
                               "SYSTEM_PAYMENT",
                               "CITIZEN",
                               "EMPLOYEE"
                           ]
                       }
                   ]
               },
               {
                   "sla": null,
                   "state": "REJECTED",
                   "applicationStatus": "REJECTED",
                   "docUploadRequired": false,
                   "isStartState": false,
                   "isTerminateState": true,
                   "isStateUpdatable": false,
                   "actions": null
               },
               {
                   "sla": null,
                   "state": "REGISTRATIONCOMPLETED",
                   "applicationStatus": "REGISTRATIONCOMPLETED",
                   "docUploadRequired": false,
                   "isStartState": false,
                   "isTerminateState": true,
                   "isStateUpdatable": false,
                   "actions": null
               }
           ]
       }
   ]
}

Integrate URL Shortener Service

Overview

This page runs through the steps involved in integrating the URL shortening service.

Steps

Utility code to talk to the URL shortener resides here:

/utils/UrlShortenerUtil.java

Add the following properties in application.properties for integration -

#url shortner
egov.url.shortner.host=https://dev.digit.org
egov.url.shortner.endpoint=/egov-url-shortening/shortener

Section 4: Integrate Billing & Payment

TBD

Overview

What is a calculator?

In governments, each region, city or state has custom rates and charges for the same domain. For example, birth certificate registration rates may differ from city to city and the way it is computed can also vary. This cannot be generalised into a platform service. However, billing is a generic platform service.

The way DIGIT solves for this is to "unbundle" the problem and separate out billing from calculations. A customisable service called a "calculator" is used for custom calculation. The calculator then calls the billing service to generate the bill. Each service/module ships with a "default" calculator which can then be customised for that city.

Resources

Code of developer guide till this stage is available here. Postman collection of btr-calculator is available here. Postman collection of btr-services is availble here.

Custom Calculator Service

Information on creating a custom calculator service

Overview

This calculator service integrates with the core platform's billing service & generates a demand. A bill can be fetched based on the demand and presented to the user. This page provides details about creating a custom calculator service.

Details

Code for the custom calculator service is here. A separate API spec is written for the calculator service.

A calculator service typically has three APIs:

_calculate - This API returns the calculation for a given service application.
getbill or createbill - Creates and returns the bill associated with a particular application.
_search - to search for calculations.

The birth registration service calls the _calculate API to generate a demand for birth registration charges.

Integrate Calculator Service

Calculating costs for a service and raising demand for bill generation

Overview

Calculation

The calculation class will contain the calculation logic for the birth certificate registration charges. This can vary from city to city. Based on the application submitted, the calculator class will calculate the tax/charges and call the billing service to generate the demand.

What is a demand?

A demand is the official communication sent by a government authority to a citizen requesting them to pay for a service. A demand leads to a bill. When a bill is paid, a receipt is generated. A demand can be modified prior to bill generation.

Steps

For our guide, we are going to create a Calculation Service that will call the calculator to generate a demand. Follow the steps below -

Create a class under service folder by the name of CalculationService
Annotate this class with @Service annotation and add the following logic within it -

Payment Back Update

Status of payment

Overview

A demand leads to a bill which then leads to payment by a citizen. Once payment is done, the application status has to be updated. Since we have a microservices architecture, the two services can communicate with each other either through API calls or using events.

The collection service publishes an event on a Kafka topic when payment is collected for an application. Any microservice that wants to get notified when payments are done can subscribe to this topic. Once the service consumes the payment message, it will check if the payment is done for its service by checking the businessService code. The application status changes to PAID or triggers a workflow transition.

Steps

For our guide, we will follow the following steps to create payment back update consumer -

Create a consumer class by the name of PaymentBackUpdateConsumer. Annotate it with @Component annotation and add the following content to it -

Create a new class by the name of PaymentUpdateService in the service folder and annotate it with @Service. Put the following content in this class -

Create the following POJOs under models folder:

PaymentRequest.java

Payment.java

PaymentDetail.java

Bill.java

BillDetail.java

BillAccountDetail.java

Section 5: Other Advanced Integrations

Add Indexer Configuration

Overview

The indexer is designed to perform all the indexing tasks of the DIGIT platform. The service reads records posted on specific Kafka topics and picks the corresponding index configuration from the yaml file provided by the respective module configuration. Configurations are yaml based. A detailed guide to creating indexer configs is mentioned in the following document - .

Steps

Create a new file under egov-indexer in configs repo by the name of digit-developer-guide.yml and put the following content into it -

Note: Follow the steps below when the code is deployed to the DIGIT environment. These steps are not applicable for deployment in the local environment. You may choose to follow these when you build and deploy.

Indexer Configuration Deployment

Navigate to the forked DIGIT-DevOps repository. Under the deploy-as-code/helm/environments directory, find the deployment helm chart that was used to deploy DIGIT.
In the deployment helm chart (which was used to set up the DIGIT environment), find "egov-indexer". Find the "egov-indexer-yaml-repo-path" property and add the path to your new indexer file here. The code block is shown below for reference:

Raise a PR to the DevOps branch which was forked/used to create the deployment. Once that is merged, restart the indexer service and make sure the cluster configs are propagated.

Certificate Generation

Overview

The final step in this process is the creation of configurations to create a voter registration PDF for the citizens to download. For this, we will make use of DIGIT’s PDF service which uses PDFMake and Mustache libraries to generate PDF. Detailed documentation on generating PDFs using PDF service is

Steps

Follow the below steps to set up PDF service locally and generate PDF for our voter registration service -

Clone repo.

Clone repo.

Navigate to the DIGIT-Dev repo and open up a terminal. Checkout DIGIT_DEVELOPER_GUIDE branch.

Navigate to the configs folder and under pdf-service data config and format config folders, create file by the name of digit-developer-guide.json

Add the following content in this newly created data config file -

Create a file by the name of digit-developer-guide.json format-config folder and place the following content in it -

Open the PDF service (under core-services repository of DIGIT-Dev) on your IDE. Open Environment.js file and change the following properties to point to the local config files created. For example, in my local setup I have pointed these to the local files that I created -

Make sure that Kafka and Workflow services are running locally and port-forward the following services -
- egov-user to port 8284
- egov-localization to port 8286
- egov-filestore to 8288
- egov-mdms to 8082
PDF service is now ready to be started up. Execute the following commands to start it up

Once PDF service is up hit the following cURL to look at the created PDF -

Deploy PDF Service

Navigate to the forked DIGIT-DevOps repository.
Find the deployment helm chart that was used to deploy DIGIT within the deploy-as-code/helm/environments directory.
Find "pdf-service"in the deployment helm chart (which was used to set up the DIGIT environment).
Find the "data-config-urls" property.
Add the path to your new PDF config file here. For this module, we have added file:///work-dir/configs/pdf-service/data-config/digit-developer-guide.json to the end of the data-config-urls. The code block is shown below for reference:

Raise a PR for this to the appropriate branch of DevOps which was forked/used to create the deployment.
Restart the PDF service in the k8s cluster, once the PR is merged. It will pick up the latest config from the file above.

Section 6: Run Final Application

Run the deployed application in the local environment

Overview

It is time to test our completed application! Follow the steps on this page to test and run the deployed application.

Steps

Before testing our application, we need to run a couple of services locally -

To run the persister service locally -
- Run Kafka.
- Open egov-persister folder which should be present under core-services repository.
- Update the following properties in application.properties file -
To run indexer service locally (optional) -
- Run Kafka.
- Run ElasticSearch.
- Open egov-indexer folder which should be present under core-services repository.
- Update the following properties in application.properties file -

For example, the path to config files would be something like -

To run and test our sample application, follow the below steps -

Ensure that Kafka, Persister, Indexer and PDF services are running locally and run the code of voter-registration-service from DIGIT_DEVELOPER_GUIDE branch (for consistency).
Port-forward the following services -
- egov-user to port 8284 (for e.g. - kubectl port-forward egov-user-58d6dbf966-8r9gz 8284:8080)
- egov-localization to port 8286 (for e.g. kubectl port-forward egov-localization-d7d5ccd49-xz9s9 8286:8080)
- egov-filestore to 8288 (for e.g. kubectl port-forward egov-filestore-86c688bbd6-zzk72 8288:8080)
- egov-mdms to 8082 (for e.g. kubectl port-forward egov-mdms-service-c9d4877d7-kd4zp 8082:8080)
Run birth-registration-service that we just created.
Setup environment variables in Postman:
- hostWithPort - Eg. yourserver.digit.org:8080 or yourserver.digit.org if the service is running on port 80.
- applicationNumber - used in the search/update requests to search for that specific application number. Set it post the create birth registration application call.

In case no workflow has been configured, run the scripts to configure and search for workflow. Double-check ID gen by running the ID gen script.

a. Hit the _create API request to create a voter registration application.

b. Hit the _search API request to search for the created voter registration applications.

c. Hit _update API request to update your application or transition it further in the workflow by changing the actions.

Section 7: Build & Deploy Instructions

Overview

Follow the instructions on this page to build and deploy applications on DIGIT.

eGov recommends before developing on top of DIGIT. This ensures that new modules can be developed and deployed in a streamlined way. DIGIT ships with CI as code as part of the DevOps repository. Run the prior to developing on DIGIT.

Steps

Step 1: Add entry in build-config.yaml file in the master branch of the forked MDMS repository. This will set up the job pipeline in Jenkins. Make sure to also add the same config to the feature branch you are working on. Refer to this .

Step 2: Follow the instructions for the , and configuration.

Step 3: Go to the Jenkins build page, select "Job Builder" and click on "Build now". This will pull config from build_config.yaml and identify all modules that need to be built.

Step 4: Once the build is done, go to your Jenkins build page. The service will appear under the repository path in which it has been added, i.e. if the service is added under core-services, it will show up in the core-services section on the aforementioned page.

Step 5: Most likely, you will be working on a feature branch for this module and not on "master". Click on "Build with parameters" for the module and search for the branch name in the filter box. Select the feature branch you are working on and then click "Build". This will make sure that Jenkins builds the module pulling code from the branch you prefer.

Step 6: Click on "Console Output". If the build pipeline and docker registries have been set up properly as part of CD/CI setup, the docker image will be built and pushed to the registry. The console output will have the docker image ID for the module. Scroll down to the bottom and copy the following information -

Step 7: After copying the docker image ID, go to your Jenkins server home page, click on "Deployments" and scroll to find your deployment environment. Deployment environments have the template of deploy-to-<env name> and get created as part of CD/CI setup. If multiple environments have been configured, you will see multiple deploy-to-* entries.

Step 8: It is best practice to always test out any new module in the dev environment. Select the environment you would like to deploy to and click on the "Run" icon on the right-hand side of the page against the environment. In the Images text box, paste the copied docker image ID and click "Build". Refer to the screenshot below.

Jenkins will now take care of deploying the new image to the DIGIT environment.

Step 9: Test your new service by testing out the APIs via Postman.

FAQs

A list of FAQs for developers by developers

Q1. Which versions of tools are recommended for use with DIGIT?

A. Git - 2.38.1 (latest version)

Java - JDK 8

Intellij - 2022.2.3(latest version)

Kafka - 3.2.0(latest version)

Postman - v9.4(latest version)

Kubectl - 1.25.3(latest version)

Postgres - v10

Q2. Which IDE is recommended for DIGIT development?

A. Intellij, Eclipse, VS Code or any other preferred IDE can be used. Make sure that it supports Java development and install the Lombok plugin for the IDE.

Q3. What are the tools which make development effortless?

A. is a lightweight tool which is quite handy when it comes to working with sending postman requests and putting objects to Kafka topics. Another such site is which makes reading/designing APIs a lot easier and understandable. Use Postman to test APIs. Use k9s to work with your Kubernetes clusters.

Q4. Is it necessary to get a new auth token every few hours?

A. Auth tokens come with an expire period after which you will have to refresh it by hitting the oauth2 APIs.

If you don’t want to refresh auth now and again you can port-forward services and get the same result. There are a couple of things to keep in mind though.

As there are many services to port forward, you must not mix up port numbers while port forwarding.
Port forwarding by-passes the zuul api gateway, hence in this case, when accessing a service directly, for a request to be valid, a user has to send the userInfo JSON inside the RequestInfo object.

Sample

Q5. What are our options when we get a bad request response in postman?

Answer

You can disconnect the forwarded port and start port forwarding again with recheck on port numbers. If this doesn’t solve the error, in some cases like adding mdms configuration, you can restart the pod to get the desired output.

Q6. I deployed the service in a DIGIT environment. But I am getting a 400 or 500 from Zuul.

Check whether the service is up. Insert appropriate namespace in the commands below.

Check whether the pod is up.

kubectl describe pod -n egov

Check the ingress and make sure the context path is right.

kubectl describe ingress <service name> -n egov

Q7. My service, pod, ingress are all looking good. But I am still getting a ZuulRuntimeException. What do I do?

This could be a forwarding issue from Zuul to services. Restart Zuul and restart your service.

Q8. I made a change to the role action mapping in MDMS. But I am unable to access the APIs. Getting a 403 unauthorized error.

Restart MDMS so it picks up the role action mapping. Also restart the access control service. Access control caches the role action mappings for 15 minutes. Hence a hard restart will force the cache to reset.

Q9. My pod is in CrashLoopBackOff. What do I do?

Integrate Workflow Service

Overview

The birth registration module follows a simple workflow derived from the swimlane diagrams. Please check the design inputs section for correlation as well as the design guide for info on how the workflow configuration is derived.

Steps

Integration with workflow service requires the following steps -

Add a workflow object to BirthRegistrationApplication POJO (this may already exist. Do not add if it exists already).

@Valid
@JsonProperty("workflow")
private Workflow workflow = null;

2. Create POJOs to support workflow - Create the following POJOs under the digit.web.models package.

ProcessInstance.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;

import javax.validation.Valid;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;
import java.util.ArrayList;
import java.util.List;

@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@EqualsAndHashCode(of = { "id" })
@ToString
public class ProcessInstance {

    @Size(max = 64)
    @JsonProperty("id")
    private String id;

    @NotNull
    @Size(max = 128)
    @JsonProperty("tenantId")
    private String tenantId;

    @NotNull
    @Size(max = 128)
    @JsonProperty("businessService")
    private String businessService;

    @NotNull
    @Size(max = 128)
    @JsonProperty("businessId")
    private String businessId;

    @NotNull
    @Size(max = 128)
    @JsonProperty("action")
    private String action;

    @NotNull
    @Size(max = 64)
    @JsonProperty("moduleName")
    private String moduleName;

    @JsonProperty("state")
    private State state;

    @JsonProperty("comment")
    private String comment;

    @JsonProperty("documents")
    @Valid
    private List<Document> documents;

    @JsonProperty("assignes")
    private List<User> assignes;

    public ProcessInstance addDocumentsItem(Document documentsItem) {
        if (this.documents == null) {
            this.documents = new ArrayList<>();
        }
        if (!this.documents.contains(documentsItem))
            this.documents.add(documentsItem);

        return this;
    }

}

State.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;

import javax.validation.Valid;
import javax.validation.constraints.Size;
import java.util.ArrayList;
import java.util.List;

@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
@EqualsAndHashCode(of = {"tenantId","businessServiceId","state"})
public class State   {

    @Size(max=256)
    @JsonProperty("uuid")
    private String uuid;

    @Size(max=256)
    @JsonProperty("tenantId")
    private String tenantId;

    @Size(max=256)
    @JsonProperty("businessServiceId")
    private String businessServiceId;

    @JsonProperty("sla")
    private Long sla;

    @Size(max=256)
    @JsonProperty("state")
    private String state;

    @Size(max=256)
    @JsonProperty("applicationStatus")
    private String applicationStatus;

    @JsonProperty("docUploadRequired")
    private Boolean docUploadRequired;

    @JsonProperty("isStartState")
    private Boolean isStartState;

    @JsonProperty("isTerminateState")
    private Boolean isTerminateState;

    @JsonProperty("isStateUpdatable")
    private Boolean isStateUpdatable;

    @JsonProperty("actions")
    @Valid
    private List<Action> actions;

    private AuditDetails auditDetails;


    public State addActionsItem(Action actionsItem) {
        if (this.actions == null) {
            this.actions = new ArrayList<>();
        }
        this.actions.add(actionsItem);
        return this;
    }

}

Action.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;

import javax.validation.Valid;
import javax.validation.constraints.Size;
import java.util.ArrayList;
import java.util.List;

@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
@EqualsAndHashCode(of = {"tenantId","currentState","action"})
public class Action {

    @Size(max=256)
    @JsonProperty("uuid")
    private String uuid;

    @Size(max=256)
    @JsonProperty("tenantId")
    private String tenantId;

    @Size(max=256)
    @JsonProperty("currentState")
    private String currentState;

    @Size(max=256)
    @JsonProperty("action")
    private String action;

    @Size(max=256)
    @JsonProperty("nextState")
    private String nextState;

    @Size(max=1024)
    @JsonProperty("roles")
    @Valid
    private List<String> roles;

    private AuditDetails auditDetails;


    public Action addRolesItem(String rolesItem) {
        if (this.roles == null) {
            this.roles = new ArrayList<>();
        }
        this.roles.add(rolesItem);
        return this;
    }

}

ProcessInstanceRequest.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;
import org.egov.common.contract.request.RequestInfo;

import javax.validation.Valid;
import javax.validation.constraints.NotNull;
import java.util.ArrayList;
import java.util.List;


@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
public class ProcessInstanceRequest {
    @JsonProperty("RequestInfo")
    private RequestInfo requestInfo;

    @JsonProperty("ProcessInstances")
    @Valid
    @NotNull
    private List<ProcessInstance> processInstances;


    public ProcessInstanceRequest addProcessInstanceItem(ProcessInstance processInstanceItem) {
        if (this.processInstances == null) {
            this.processInstances = new ArrayList<>();
        }
        this.processInstances.add(processInstanceItem);
        return this;
    }

}

ProcessInstanceResponse.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;
import org.egov.common.contract.response.ResponseInfo;

import javax.validation.Valid;
import java.util.ArrayList;
import java.util.List;

@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class ProcessInstanceResponse {
    @JsonProperty("ResponseInfo")
    private ResponseInfo responseInfo;

    @JsonProperty("ProcessInstances")
    @Valid
    private List<ProcessInstance> processInstances;


    public ProcessInstanceResponse addProceInstanceItem(ProcessInstance proceInstanceItem) {
        if (this.processInstances == null) {
            this.processInstances = new ArrayList<>();
        }
        this.processInstances.add(proceInstanceItem);
        return this;
    }

}

BusinessService.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;

import javax.validation.Valid;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;
import java.util.ArrayList;
import java.util.List;


@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
@EqualsAndHashCode(of = {"tenantId","businessService"})
@JsonInclude(JsonInclude.Include.NON_NULL)
public class BusinessService   {

    @Size(max=256)
    @JsonProperty("tenantId")
    private String tenantId = null;

    @Size(max=256)
    @JsonProperty("uuid")
    private String uuid = null;

    @Size(max=256)
    @JsonProperty("businessService")
    private String businessService = null;

    @Size(max=256)
    @JsonProperty("business")
    private String business = null;

    @Size(max=1024)
    @JsonProperty("getUri")
    private String getUri = null;

    @Size(max=1024)
    @JsonProperty("postUri")
    private String postUri = null;

    @JsonProperty("businessServiceSla")
    private Long businessServiceSla = null;

    @NotNull
    @Valid
    @JsonProperty("states")
    private List<State> states = null;

    @JsonProperty("auditDetails")
    private AuditDetails auditDetails = null;



    public BusinessService addStatesItem(State statesItem) {
        if (this.states == null) {
            this.states = new ArrayList<>();
        }
        this.states.add(statesItem);
        return this;
    }


    /**
     * Returns the currentState with the given uuid if not present returns null
     * @param uuid the uuid of the currentState to be returned
     * @return
     */
    public State getStateFromUuid(String uuid) {
        State state = null;
        if(this.states!=null){
            for(State s : this.states){
                if(s.getUuid().equalsIgnoreCase(uuid)){
                    state = s;
                    break;
                }
            }
        }
        return state;
    }



}ava

BusinessServiceResponse.java

package digit.web.models;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;
import org.egov.common.contract.response.ResponseInfo;

import javax.validation.Valid;
import javax.validation.constraints.NotNull;
import java.util.ArrayList;
import java.util.List;



@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder
@ToString
public class BusinessServiceResponse {

    @JsonProperty("ResponseInfo")
    private ResponseInfo responseInfo;

    @JsonProperty("BusinessServices")
    @Valid
    @NotNull
    private List<BusinessService> businessServices;


    public BusinessServiceResponse addBusinessServiceItem(BusinessService businessServiceItem) {
        if (this.businessServices == null) {
            this.businessServices = new ArrayList<>();
        }
        this.businessServices.add(businessServiceItem);
        return this;
    }



}

Create Workflow service - Create a class to transition the workflow object across its states. For this, create a class by the name of WorkflowService.java under the service directory and annotate it with @Service annotation.
Add the below content to this class -

package digit.service;

import com.fasterxml.jackson.databind.ObjectMapper;
import digit.config.BTRConfiguration;
import digit.repository.ServiceRequestRepository;
import digit.web.models.*;
import lombok.extern.slf4j.Slf4j;
import org.egov.common.contract.request.RequestInfo;
import org.egov.tracer.model.CustomException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

@Component
@Slf4j
public class WorkflowService {

    @Autowired
    private ObjectMapper mapper;

    @Autowired
    private ServiceRequestRepository repository;

    @Autowired
    private BTRConfiguration config;

    public void updateWorkflowStatus(BirthRegistrationRequest birthRegistrationRequest) {
        birthRegistrationRequest.getBirthRegistrationApplications().forEach(application -> {
            ProcessInstance processInstance = getProcessInstanceForBTR(application, birthRegistrationRequest.getRequestInfo());
            ProcessInstanceRequest workflowRequest = new ProcessInstanceRequest(birthRegistrationRequest.getRequestInfo(), Collections.singletonList(processInstance));
            callWorkFlow(workflowRequest);
        });
    }

    public State callWorkFlow(ProcessInstanceRequest workflowReq) {

        ProcessInstanceResponse response = null;
        StringBuilder url = new StringBuilder(config.getWfHost().concat(config.getWfTransitionPath()));
        Object optional = repository.fetchResult(url, workflowReq);
        response = mapper.convertValue(optional, ProcessInstanceResponse.class);
        return response.getProcessInstances().get(0).getState();
    }

    private ProcessInstance getProcessInstanceForBTR(BirthRegistrationApplication application, RequestInfo requestInfo) {
        Workflow workflow = application.getWorkflow();

        ProcessInstance processInstance = new ProcessInstance();
        processInstance.setBusinessId(application.getApplicationNumber());
        processInstance.setAction(workflow.getAction());
        processInstance.setModuleName("birth-services");
        processInstance.setTenantId(application.getTenantId());
        processInstance.setBusinessService("BTR");
        processInstance.setDocuments(workflow.getDocuments());
        processInstance.setComment(workflow.getComments());

        if(!CollectionUtils.isEmpty(workflow.getAssignes())){
            List<User> users = new ArrayList<>();

            workflow.getAssignes().forEach(uuid -> {
                digit.web.models.User user = new digit.web.models.User();
                user.setUuid(uuid);
                users.add(user);
            });

            processInstance.setAssignes(users);
        }

        return processInstance;

    }
     
     public ProcessInstance getCurrentWorkflow(RequestInfo requestInfo, String tenantId, String businessId) {

        RequestInfoWrapper requestInfoWrapper = RequestInfoWrapper.builder().requestInfo(requestInfo).build();

        StringBuilder url = getSearchURLWithParams(tenantId, businessId);

        Object res = repository.fetchResult(url, requestInfoWrapper);
        ProcessInstanceResponse response = null;

        try{
            response = mapper.convertValue(res, ProcessInstanceResponse.class);
        }
        catch (Exception e){
            throw new CustomException("PARSING_ERROR","Failed to parse workflow search response");
        }

        if(response!=null && !CollectionUtils.isEmpty(response.getProcessInstances()) && response.getProcessInstances().get(0)!=null)
            return response.getProcessInstances().get(0);

        return null;
    }
    
    private BusinessService getBusinessService(BirthRegistrationApplication application, RequestInfo requestInfo) {
        String tenantId = application.getTenantId();
        StringBuilder url = getSearchURLWithParams(tenantId, "BTR");
        RequestInfoWrapper requestInfoWrapper = RequestInfoWrapper.builder().requestInfo(requestInfo).build();
        Object result = repository.fetchResult(url, requestInfoWrapper);
        BusinessServiceResponse response = null;
        try {
            response = mapper.convertValue(result, BusinessServiceResponse.class);
        } catch (IllegalArgumentException e) {
            throw new CustomException("PARSING ERROR", "Failed to parse response of workflow business service search");
        }

        if (CollectionUtils.isEmpty(response.getBusinessServices()))
            throw new CustomException("BUSINESSSERVICE_NOT_FOUND", "The businessService " + "BTR" + " is not found");

        return response.getBusinessServices().get(0);
    }

    private StringBuilder getSearchURLWithParams(String tenantId, String businessService) {

        StringBuilder url = new StringBuilder(config.getWfHost());
        url.append(config.getWfBusinessServiceSearchPath());
        url.append("?tenantId=");
        url.append(tenantId);
        url.append("&businessServices=");
        url.append(businessService);
        return url;
    }

    public ProcessInstanceRequest getProcessInstanceForBirthRegistrationPayment(BirthRegistrationRequest updateRequest) {

        BirthRegistrationApplication application = updateRequest.getBirthRegistrationApplications().get(0);

        ProcessInstance process = ProcessInstance.builder()
                .businessService("BTR")
                .businessId(application.getApplicationNumber())
                .comment("Payment for birth registration processed")
                .moduleName("birth-services")
                .tenantId(application.getTenantId())
                .action("PAY")
                .build();

        return ProcessInstanceRequest.builder()
                .requestInfo(updateRequest.getRequestInfo())
                .processInstances(Arrays.asList(process))
                .build();

    }
}

Add workflow to BirthRegistrationService.
Add the below field to BirthRegistrationService.java

 @Autowired
 private WorkflowService workflowService;

Transition the workflow - Modify the following methods in BirthRegistrationService.java as follows. Note that we are adding calls into the workflow service in each of these methods.

registerBtRequest

  public List<BirthRegistrationApplication> registerBtRequest(BirthRegistrationRequest birthRegistrationRequest) {
        birthApplicationValidator.validateBirthApplication(birthRegistrationRequest);
        birthApplicationEnrichment.enrichBirthApplication(birthRegistrationRequest);

        // Enrich/Upsert user in upon birth registration
        userService.callUserService(birthRegistrationRequest);

        // WORKFLOW INTEGRATION HERE: Initiate workflow for the new application
        workflowService.updateWorkflowStatus(birthRegistrationRequest);

        // Push the application to the topic for persister to listen and persist
        producer.push(configuration.getCreateTopic(), birthRegistrationRequest);

        // Return the response back to user
        return birthRegistrationRequest.getBirthRegistrationApplications();
    }Java

updateBtApplication

  public List<BirthRegistrationApplication> updateBtApplication(BirthRegistrationRequest birthRegistrationRequest) {
        // Validate whether the application that is being requested for update indeed exists
        List<BirthRegistrationApplication> existingApplication = birthApplicationValidator.validateApplicationUpdateRequest(birthRegistrationRequest);
        // Enrich application upon update
        birthApplicationEnrichment.enrichBirthApplicationUponUpdate(birthRegistrationRequest);
        //Workflow integration
        workflowService.updateWorkflowStatus(birthRegistrationRequest);
        // Just like create request, update request will be handled asynchronously by the persister
        producer.push(configuration.getUpdateTopic(), birthRegistrationRequest);

        return birthRegistrationRequest.getBirthRegistrationApplications();
    }

searchBtApplications

  public List<BirthRegistrationApplication> searchBtApplications(RequestInfo requestInfo, BirthApplicationSearchCriteria birthApplicationSearchCriteria) {
        // Fetch applications from database according to the given search criteria
        List<BirthRegistrationApplication> applications = birthRegistrationRepository.getApplications(birthApplicationSearchCriteria);

        // If no applications are found matching the given criteria, return an empty list
        if(CollectionUtils.isEmpty(applications))
            return new ArrayList<>();

        // Enrich mother and father of applicant objects
        applications.forEach(application -> {
            birthApplicationEnrichment.enrichFatherApplicantOnSearch(application);
            birthApplicationEnrichment.enrichMotherApplicantOnSearch(application);
        });
        
        //WORKFLOW INTEGRATION
        applications.forEach(application -> {
            application.setWorkflow(Workflow.builder().status(workflowService.getCurrentWorkflow(requestInfo, application.getTenantId(), application.getApplicationNumber()).getState().getState()).build());
        });

        // Otherwise, return the found applications
        return applications;
    }

Configure application.properties - Add the following properties to application.properties file of the birth registration module. Depending on whether you are port forwarding or using the service directly on the host, please update the host name.

#Workflow config
is.workflow.enabled=true
egov.workflow.host=http://localhost:8282
egov.workflow.transition.path=/egov-workflow-v2/egov-wf/process/_transition
egov.workflow.businessservice.search.path=/egov-workflow-v2/egov-wf/businessservice/_search
egov.workflow.processinstance.search.path=/egov-workflow-v2/egov-wf/process/_search

Run the workflow service locally - the application will call into it to create the necessary tables in the DB and effect the workflow transitions.

Backend Developer Guide

Section 0: Prep

Development Pre-requisites

Checklist

Design Inputs

High Level Design

API Specs

Process Workflow Diagram

Registries

Services

System Architecture Diagram

Low Level Design

DB Schema Diagram

Development Environment Setup

Overview

Steps

Section 1: Create Project

Generate Project Using API Specs

Overview

Steps

Build A Microservice

Generate API Skeleton

Create Database

Overview

Steps

Enable Flyway Migration

Configure Application Properties

Overview

Steps

Configure Application Properties

Configure Kafka

Kafka Topics For Module

Create & Update Kafka Topics For The Module

Import Core Models

Overview

Steps

Import Core Models

Implement Repository Layer

Overview

Steps

Implement Repository Layer

Create Validation & Enrichment Layers

Overview

Validation Layer

Steps

Enrichment Layer

Steps

Implement Service Layer

Overview

Steps

Build The Web Layer

Overview

Steps

Steps to setup the request handler in the controller layer

Section 2: Integrate Persister & Kafka

Add Kafka Configuration

Overview

Steps

Implement Kafka Producer & Consumer

Overview

Producer

Steps

Consumer

Steps

Add Persister Configuration

Overview

Steps

Add Persister Configuration

Run Persister Locally

Deploy Persister Configuration

Enable Signed Audit

Overview

Steps

Enable Signed Audit

Run Application

Overview

Steps

Run & Test Sample Application

Section 3: Integrate Microservices

Overview