Overview

This documents highlights the changes need to be done in a module to support the user privacy feature.

Pre-requisites

• Prior Knowledge of Java/J2EE.

• Prior Knowledge of Spring Boot.

• MDMS Service

• Encryption Service

• Prior Knowledge of React Js / Javascript

Backend service changes

Update the pom.xml file.

• Upgrade services-common library version.

<dependency>
    <groupId>org.egov.services</groupId>
    <artifactId>services-common</artifactId>
    <version>1.1.1-SNAPSHOT</version>
</dependency>

• Upgrade tracer library version.

<dependency>
    <groupId>org.egov.services</groupId>
    <artifactId>tracer</artifactId>
    <version>2.1.2-SNAPSHOT</version>
</dependency>

Demand generation changes:

At service level or calculator service level where demand generation process take place, there payer details value must be pass in plain form. And these user details must get through SYSTEM user.

Create a system user in the particular environment and make sure that system user has role INTERNAL_MICROSERVICE_ROLE. Use the curl mentioned below:

curl --location --request POST 'http://localhost:8081/user/users/_createnovalidate' \
--header 'Content-Type: application/json' \
--data-raw '{
    "RequestInfo": {
  "apiId": "Rainmaker",
  "authToken": "81a67bd9-3ff6-402c-ae59-d9209c9dfb3d",
  "userInfo": {
    "id": 29836,
    "uuid": "d8854322-a078-41f8-90b5-6f3a1aee9454",
    "userName": "qeroo",
    "name": "QARoo",
    "mobileNumber": "8339358368",
    "emailId": "",
    "locale": null,
    "type": "EMPLOYEE",
    "roles": [
      
      {
        "name": "Employee",
        "code": "EMPLOYEE",
        "tenantId": "pb.amritsar"
      },
      {
        "name": "Super User",
        "code": "SUPERUSER",
        "tenantId": "pb.amritsar"
      }
    ],
    "active": true,
    "tenantId": "pb.amritsar",
    "permanentCity": null
  },
  "msgId": "1653559822522|en_IN"
},
    "user": {
        "userName": "INTERNAL_MICROSERVICE_USER",
        "password": "eGov@123",
        "salutation": null,
        "name": "Internal Microservice User",
        "gender": null,
        "mobileNumber": "8888888880",
        "emailId": null,
        "altContactNumber": "",
        "pan": null,
        "aadhaarNumber": null,
        "permanentAddress": null,
        "permanentCity": null,
        "permanentPincode": null,
        "correspondenceCity": null,
        "correspondencePincode": null,
        "correspondenceAddress": null,
        "active": true,
        "dob": null,
        "pwdExpiryDate": null,
        "locale": "string",
        "type": "SYSTEM",
        "signature": null,
        "accountLocked": false,
        "fatherOrHusbandName": "null",
        "bloodGroup": null,
        "identificationMark": null,
        "photo": null,
        "otpReference": null,
        "tenantId": "pb",
        "roles": [
           {
                    "code": "INTERNAL_MICROSERVICE_ROLE",
                    "name": "Internal Microservice Role",
                    "tenantId": "pb"
            }
        ]
    }
}'

Mention the uuid of the system user in the environment file and in the application.properties file of service example:

Environment file:

egov-internal-microservice-user-uuid: b5b2ac70-d347-4339-98f0-5349ce25f99f

Application properties file:

egov.internal.microservice.user.uuid=b5b2ac70-d347-4339-98f0-5349ce25f99f

Create a method/ function which call the user search API to get the user details in plain form, in that call pass the userInfo of the user with INTERNAL_MICROSERVICE_ROLE

For reference follow the below code snippet

private User getPlainOwnerDetails(RequestInfo requestInfo, String uuid, String tenantId){
		User userInfoCopy = requestInfo.getUserInfo();
		StringBuilder uri = new StringBuilder();
		uri.append(configs.getUserHost()).append(configs.getUserSearchEndpoint()); // URL for user search call
		Map<String, Object> userSearchRequest = new HashMap<>();
		tenantId = tenantId.split("\\.")[0];

		//Creating role with INTERNAL_MICROSERVICE_ROLE
		Role role = Role.builder()
				.name("Internal Microservice Role").code("INTERNAL_MICROSERVICE_ROLE")
				.tenantId(tenantId).build();

        //Creating userinfo with uuid and role of internal micro service role
		User userInfo = User.builder()
				.uuid(configs.getEgovInternalMicroserviceUserUuid())
				.type("SYSTEM")
				.roles(Collections.singletonList(role)).id(0L).build();

		requestInfo.setUserInfo(userInfo);

        // Setting user search criteria
		userSearchRequest.put("RequestInfo", requestInfo);
		userSearchRequest.put("tenantId", tenantId);
		userSearchRequest.put("uuid",Collections.singletonList(uuid));
		User user = null;
		try {
			LinkedHashMap<String, Object> responseMap = (LinkedHashMap<String, Object>) serviceRequestRepository.fetchResult(uri, userSearchRequest);

			List<LinkedHashMap<String, Object>> users = (List<LinkedHashMap<String, Object>>) responseMap.get("user");
			String dobFormat = "yyyy-MM-dd";
			
			// parsing the user response and coverting object into User.class pojo
			notificationUtil.parseResponse(responseMap,dobFormat);
			user = 	mapper.convertValue(users.get(0), User.class);

		} catch (Exception e) {
			throw new CustomException("EG_USER_SEARCH_ERROR", "Service returned null while fetching user");
		}
		requestInfo.setUserInfo(userInfoCopy);
		return user;
	}

Report Configuration Changes:

For the report where PII data is present and decryption is required, such report definition must have UUID values from user table in the query result. And the Source column section of the report definition must have mention of the UUID column and for that entry, showColumn flag should be set as false.

 - name: uuid
    label: reports.{moduleName}.uuid
    type: string
    source: eg_user
    showColumn: false

And the base sql query in the report config should be written in such a way that in the query it must have the user uuid.

Example:

SELECT emp_data.id as id,
      emp_data.name as name,
      emp_data.uuid as uuid,
      desigmap_def.code as designation,
      deptmap_def.code as department,
      emp_data.employeestatus as status
    FROM emp_data ehe
    inner join eg_user eu ON ehe.uuid = eu.uuid

In the report config where PII data is present and decryption is required, add this new filed in the report config decryptionPathId. And in the Security Policy mdms file, must have the model configuration from the particular report. And the value of key model in Security Policy mdms file and decryptionPathId in a report must be same.

Example:-

{
  "model": "EmployeeReport",
  "uniqueIdentifier": {
    "name": "uuid",
    "jsonPath": "uuid"
  },
  "attributes": [
    {
      "name": "name",
      "jsonPath": "name",
      "patternId": "002",
      "defaultVisibility": "PLAIN"
    }
  ],
  "roleBasedDecryptionPolicy": [
    {
      "roles": [
        "CEMP"
      ],
      "attributeAccessList": [
        {
          "attribute": "name",
          "firstLevelVisibility": "MASKED",
          "secondLevelVisibility": "PLAIN"
        }
      ]
    }
  ]
}

Searcher Configuration changes:

For the searcher result where PII data is present and decryption is required, such searcher definition must have UUID values from user table in the query result. The base sql query in the searcher config should be written in such a way that in the query it must have the user uuid.

I searcher result, where data are coming from user table and to get those value in decrypted form, then in searcher config these following filed decryptionPathIdmust be present. And the Security Policy mdms file, must have the model configuration from the particular searcher config. And the value of key model in Security Policy mdms file and decryptionPathId in searcher config must be same.

For details information on Security Policy MDMS file refer this document.

Frontend Changes:

To Do

Overview

The enc-client library is a supplementary java library provided to support encryption-related functionalities so that every service does not need to pre-process the request before calling the encryption service.

Pre-requisites

• MDMS Service

• Encryption Service

• Kafka

Important Note

The MDMS configurations explained below are fetched by this library at boot time. So after you make changes in the MDMS repo and restart the MDMS service, you would also need to RESTART THE SERVICE which has imported the enc-client library. For example, the report service is using the enc-client library so after making configuration changes to Security Policy pertaining to any report, you will have to restart the report service.

Key Functionalities

• Encrypt a JSON Object - The encryptJson function of the library takes any Java Object as an input and returns an object which has encrypted values of the selected fields. The fields to be encrypted are selected based on an MDMS Configuration.

  • This function requires the following parameters:

    1. Java/JSON object - The oject whose fields will get encrypted.

    2. Model - It is used to identify the MDMS configuration to be used to select fields of the provided object.

    3. Tenant Id - The encryption key will be selected based on the passed tenantId.

• Encrypt a Value - The encryptValue function of the library can be used to encrypt single values. This method also required a tenantId parameter.

• Decrypt a JSON Object - The decryptJson function of the library takes any Java Object as an input and returns an object that has plain/masked or no values of the encrypted fields. The fields are identified based on the MDMS configuration. The returned value(plain/masked/null) of each of the attribute depends on the user’s role and if it is a PlainAccess request or a normal request. These configurations are part of the MDMS.

  • This function required following parameters:

    1. Java/JSON object - The object containing the encrypted values that are to be decrypted.

    2. Model - It is used to select a configuration from the list of all available MDMS configurations.

    3. Purpose - It is a string parameter that passes the reason of the decrypt request. It is used for Audit purposes.

    4. RequestInfo - The requestInfo parameter serves multiple purposes:

       1. User Role - A list of user roles are extracted from the requestInfo parameter.

       2. PlainAccess Request - If the request is an explicit plain access request, it is to be passed as a part of the requestInfo. It will contain the fields that user is requesting for decryption and the id of record.

  • While decrypting Java object, this method also audits the request.

Configurations

All the configurations related to enc-client library are stored in the MDMS. These master data are stored in DataSecurity module. It has two types of configurations:

1. Masking Patterns

2. Security Policy

Masking Patterns

{
    "patternId": "001",
    "pattern": ".(?=.{4})"
}

The masking patterns for different types of attributes(mobile number, name, etc.) are configurable in MDMS. It contains the following attributes:

1. patternId - It is the unique pattern identifier. This id is referred to in the SecurityPolicy MDMS.

2. pattern - This defines the actual pattern according to which the value will be masked.

Here is a link to a sample Masking Patterns master data.

Security Policy

The Security Policy master data contains the policy used to encrypt and decrypt JSON objects. Each of the Security Policy contains the following details:

1. model - This is the unique identifier of the policy.

2. uniqueIdentifier - The field defined here should uniquely identify records passed to the decryptJson function.

3. attributes - This defines a list of fields from the JSON object that needs to be secured.

4. roleBasedDecryptionPolicy - This defines attribute-level role-based policy. It will define visibility for each attribute.

Visibility

The visibility is an enum with the following options:

1. PLAIN - Show text in plain form.

2. MASKED - The returned text will contain masked data. The masking pattern will be applied as defined in the Masking Patterns master data.

3. NONE - The returned text will not contain any data. It would contain string like “Confidential Information”.

It defines what level of visibility should the decryptJson function return for each attribute.

Attribute

{
  "name": "mobileNumber",
  "jsonPath": "mobileNumber",
  "patternId": "001",
  "defaultVisibility": "MASKED"
}

The Attribute defines a list of attributes of the model that are to be secured. The attribute is defined by the following parameters:

1. name - This uniquely identifies the attribute out of the list of attributes for a given model.

2. jsonPath - It is the json path of the attribute from the root of the model. This jsonPath is NOT the same as Jayway JsonPath library. This uses / and * to define the json paths.

3. patternId - It refers to the pattern to be used for masking which is defined in the Masking Patterns master.

4. defaultVisibility - It is an enum configuring the default level of visibility of that attribute. If the visibility is not defined for a given role, then this defaultVisibility will apply.

Unique Identifier

This parameter is used to define the unique identifier of that model. It is used for the purpose of auditing the access logs. (This attribute’s jsonPath should be at the root level of the model.)

{
  "name": "uuid",
  "jsonPath": "uuid"
}

Role-Based Decryption Policy

{
  "roles": [ "PGR_LME", "GRO" ],
  "attributeAccessList": [
    {
      "attribute": "name",
      "firstLevelVisibility": "MASKED",
      "secondLevelVisibility": "PLAIN"
    },
    {
      "attribute": "mobileNumber",
      "firstLevelVisibility": "MASKED",
      "secondLevelVisibility": "PLAIN"
    }
  ]
}

It defines attribute-level access policies for a list of roles. It consists of the following parameters:

1. roles - It defines a list of role codes for which the policy will get applied. Please make sure not to duplicate role codes anywhere in the other policy. Otherwise, any one of the policies will get chosen for that role code.

2. attributeAccessList - It defines a list of attributes for which the visibility differs from the default for those roles.

   1. There are two levels of visibility:

      1. First level Visibility - It applies to normal search requests. The search response could have multiple records.

      2. Second level Visibility - It is applied only when a user explicitly requests for plain access of a single record with a list of fields required in plain.

Second level visibility can be requested by passing plainAccessRequest in the RequestInfo.

Plain Access Request

{
  "RequestInfo": {
    "plainAccessRequest": {
      "recordId": "d5ee3d45-13a1-4aa5-bd86-9b8dae34b900"
      "fields": [ "name", "mobileNumber" ]
    }
  }
}

Any user will be able to get plain access to the secured data(citizen’s PII) by requesting through the plainAccessRequest parameter. It takes the following parameters:

1. recordId - It is the unique identifier of the record that is requested for plain access.

2. fields - It defines a list of attributes that are requested for plain access.

Audit Service

Every decrypt request is audited. Based on the uniqueIdentifier defined as part of the Security Policy, it lists out the identifiers of the records that were decrypted as part of the request.

Each Audit Object contains the following attributes:

AuditObject {  
    private String id;      // Audit object's unique identifer

    private String userId;  // The user that has requested for decryption

    private Long timestamp; // The time at which the decryption was requested

    private String purpose; // The purpose of the decryption request

    private String model;   // The model that is requested to decrypt

    private List<String> entityIds; // A list of ids of the entity that were decrypted

    private PlainRequestAccess plainRequestAccess;  // The parameters that were passed as part of the decryption request

    private JsonNode additionalInfo;  // A space for storing any additional information
}

Overview

Encryption Service is used to secure sensitive data that is being stored in the database. The encryption services uses envelope encryption.

Pre-requisites

Before you proceed with the documentation, make sure the following pre-requisites are met -

• Java 17.

• Kafka server is up and running.

Key Functionalities

Encryption Service offers following features :

1. Encrypt - The service will encrypt the data based on given input parameters and data to be encrypted. The encrypted data will be mandatorily of type string.

2. Decrypt - The decryption will happen solely based on the input data (any extra parameters are not required). The encrypted data will have identity of the key used at the time of encryption, the same key will be used for decryption.

3. Sign - Encryption Service can hash and sign the data which can be used as unique identifier of the data. This can also be used for searching gicen value from a datastore.

4. Verify - Based on the input sign and the claim, it can verify if the the given sign is correct for the provided claim.

5. Rotate Key - Encryption Service supports changing the key used for encryption. The old key will still remain with the service which will be used to decrypt old data. All the new data will be encrypted by the new key.

Configuration Details

Following are the properties in application.properties file in egov-enc-service which are configurable.

Deployment Details

1. Deploy the latest version of Encryption Service

   Note: This video will give you an idea of how to deploy any Digit-service. Further you can find the latest builds for each service in out latest release document here.

2. Add Role-Action mapping for API’s.

Integration

Integration Scope

The Encryption service is used to encrypt sensitive data that needs to be stored in the database. For each tenant, a different data encryption key (DEK) is used. The DEK is encrypted using Key Encryption Keys (KEK). Currently, there are two implementations of encrypting the data encryption keys available. The first one is using AWS KMS service and the second one is using a master password. For any custom implementation, the MasterKeyProvider interface in the service should be extended. Based on the master.password.provider flag in appication.properties you can enable which implementation of MasterKeyProvider interface to use

Integration Benefits

• Can perform encryption without having to re-write encryption logic every time in every service.

Steps to Integration

1. To integrate, the host of encryption-services module should be overwritten in the helm chart.

2. /crypto/v1/_encrypt should be added as an endpoint for encrypting input data in the system

3. /crypto/v1/_decrypt should be added as the decryption endpoint.

4. /crypto/v1/_sign should be added as the endpoint for providing a signature for a given value.

5. /crypto/v1/_verify should be added as the endpoint for verifying whether the signature for the provided value is correct.

6. /crypto/v1/_rotatekey should be added as an endpoint to deactivate the keys and generate new keys for a given tenant.

Reference Docs

Doc Links

API List

a) POST /crypto/v1/_encrypt

Encrypts the given input value/s OR values of the object.

b) POST /crypto/v1/_decrypt

Decrypts the given input value/s OR values of the object.

c) /crypto/v1/_sign

Provide signature for a given value.

d) POST /crypto/v1/_verify

Check if the signature is correct for the provided value.

e) POST /crypto/v1/_rotatekey

Deactivate the keys for the given tenant and generate new keys. It will deactivate both symmetric and asymmetric keys for the provided tenant.

Play around with the APIs : DIGIT-Playground

User Service stores the PII data in the database in encrypted form. So whichever service is reading that data directly from the database will have to decrypt the data before responding to the user. As of now, to the best of our knowledge following services are reading the user data from the database:

1. User Service

2. Report Service

3. Searcher Service

Enc-Client Library is a supplementary library with features like masking, auditing, etc. The above services call the functions from the library to decrypt the data read from the database.

When any of these services reads the data from the database, it will be in encrypted form. Before responding to a request, they call the enc-client library to convert the data to plain/masked form. The data returned as part of the response should only be in plain/masked form. It should not contain any encrypted values. Detailed guidelines on how to write the necessary configurations are provided in Guidelines for supporting User Privacy in a module document.