Overview

The Fiscal Event Service maintains the fiscal event flow activities of multiple projects. The root level entity is a tenant which is basically the State Government. The tenant has several projects that are tagged to multiple attributes like department, scheme, mission, department, hierarchy, etc. All the entity information and COA (Chart of Account) are passed along with amount details which constitutes an array of fiscal event instances under the system.

Version

Current version : 2.0.0

Prerequisites

Before you proceed with the configuration, make sure the following pre-requisites are met.

1. Java 8

2. MongoDB instance

3. Required Service Dependencies.

   1. iFIX-Master-Data-Service

   2. iFIX-Fiscal Event Post Processor

   3. Apache Kafka Server

Features

This service provides two features - Push fiscal event data and search fiscal event data.

Push fiscal event data

• It is a secure endpoint and user info details are required to access it.

• It receives fiscal detail along with mandatory tenant and project info under attributes detail in publish requests, which triggers the whole fiscal event process and forwards it to the post-process service.

• Before encompassing all entities, it checks for entities validation via iFix core Master Data Service.

• While we create any fiscal event instance in the system, we log ingestion time and event occurrence timestamp which makes event flow clear on timestamp activities.

• After processing all these activities, it passes fiscal event information to other fiscal event post-processor services for further processing by publishing fiscal data into the Kafka topic "fiscal-event-request-validated" and “fiscal-event-mongodb-sink“.

• It also responds to snapshots of enriched fiscal event data which can be used by the source system for reference or any further processing.

Search fiscal Event Data

• It provides a search feature on the existing fiscal events which has been processed before by post-processor service and persisted into the MongoDB instance.

• We can mainly search on eventType, tenantId, referenceId and event time interval and in return we get enriched data (dereference with nested id details).

API List

Interaction Diagram

Environment

Note: Kafka topic needs to be configured with respect to the environment

Configurations and Setup

1. Update all the DB and URI configuration in the dev.yaml, qa.yaml, prod.yaml file.

2. Make sure the keycloak server is up and running And have been configured with the required client ID.

References and Notes

Overview

Master data service maintains information about Government and Chart of Accounts. We can create these details and search for the same details based on the given parameters/request data.

Version

Current version : 2.0.0

Prerequisites

Before we proceed with the configuration, make sure the following pre-requisites are met:

1. Java 8

2. MongoDB instance

Features

It creates secure endpoints for the master data service. The access token is required to create any master data.

The subsequent sections on this page discuss the service details maintained by the IFIX core master data service.

Government

This service provides the capabilities to maintain the Government details and allow users to Create and Search data. For creating the Government, we need a unique Id for the Government and a name for the same. Optionally, we can pass some additional details as part of the attribute. In the case of search, passing the unique ID(s) as search parameters can give you all the details of the required Government.

API List

Chart Of Account

This service provides the capabilities to maintain the Chart of Account (COA) details and support create and search of COA. The following information is passed while creating the Chart of Accounts - Government Id, majorHead, subMajorHead, minorHead, subHead, groupHead, objectHead and corresponding head names & types. A unique code named COACode is generated by combining (concatenating) majorHead, subMajorHead, minorHead, subHead, groupHead, objectHead with a hyphen ("-") and stored with the given request. Searching the details for COA is done based on the given search parameters like the Chart of Account IDs, COACodes, Government ID, majorHead, subMajorHead, minorHead, subHead, groupHead, objectHead.

API List

Interaction Diagram

Environment

No environment variables are required specific to the environment (migration).

Configurations and Setup

1. Update the DB and URI configurations in the dev.yaml, qa.yaml, prod.yaml file.

2. Make sure the keycloak server is up and running and has been configured with the required client ID.

References and Notes

Overview

Fiscal Event Post Processor is a streaming pipeline for validated fiscal event data. Apache Kafka has been used to stream the validated fiscal event data, process it for dereferencing, unbundle, flatten, and finally push these details to the Druid data store.

Version

Current version : 2.0.0

Prerequisites

Before you proceed with the configuration, make sure the following pre-requisites are met:

1. Java 8

2. Apache Kafka and Kafka-Connect server should be up and running

3. Druid DB should be up and running

4. Below dependent services are required: iFix Master data service iFix Fiscal Event service

Features

Fiscal Event post-processor consumes the fiscal event validated data from Kafka topic named “fiscal-event-request-validated” and processes it by following the below steps:

1. Fiscal event validated data gets dereferenced. For dereferencing, service ids like COA id, Tenant id etc. are passed to corresponding services - master service and fetch the corresponding object(s). Once the fiscal event data is dereferenced, push/send the same data to the dereferenced Topic.

2. Unbundle consumers pick up the dereferenced fiscal event data from the dereferencing topic. Dereference fiscal event data gets unbundled and then flattened. Once the flattening is complete, push/send the same data to the Druid Sink topic.

3. Flattened fiscal event data is pushed to Druid DB from a topic named: fiscal-event-druid-sink.

Kafka to Data Store Sink

MongoDB Sink

The Kafka-connect is used to push the data from a Kafka topic to MongoDB. Follow the steps below to start the connector.

1. Connect (port-forward) with the Kafka-connect server.

2. Create a new connector with a POST API call to localhost:8083/connectors.

4. Within that file, wherever ${---} replace it with the actual value based on the environment. Get ${mongo-db-authenticated-uri} from the configured secrets of the environment. (Optional) Verify and make changes to the topic names.

5. The connector is ready. You can check it using API call GET localhost:8083/connectors.

Druid Sink

The Druid console is used to start ingesting data from a Kafka topic to the Druid data store. Follow the steps below to start the Druid Supervisor.

1. Open the Druid console

2. Go to the Load Data section

3. Select Other

4. Click on Submit Supervisor

6. Verify the Kafka topic name and the Kafka bootstrap server address before submitting the config

7. Submit the config and the data ingestion should start into the fiscal-event data source

Interaction Diagram

Environment

Note: Kafka topic needs to be configured with respect to the environment

Configurations and Setup

Update the DB, Kafka producer & Consumer And URI configurations in the dev.yaml, qa.yaml, prod.yaml file.

References and Notes

Please follow the below steps to create iFIX Master Data. Have a look at the individual service documentation for details here.

1. Create the Government by providing valid government details. Once created, you’ll get an Id with provided details in the response. we are calling this id as a tenant id.

2. Create the Chart of Account (COA) by providing valid details. While creating, you need to pass the valid tenant Id in the COA request. Once created, you’ll get an id with all the provided details in the response. We are calling this id as COA Id and a code as COACode.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Key Domain Service Details

• iFIX Core Master Data Service

• iFIX Core Fiscal Event Service

• iFIX Core Fiscal Event Post-Processor Service

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Keycloak console is available at https://<host-name>/auth. The Ops team will provide the username and password secrets.

Import Realm

1. Open the Keycloak console

2. Near the top-left corner in the realm drop-down menu, select Add Realm

3. Select the ifix-realm.json file

4. After the realm gets created, select the ifix realm from the drop-down near the top-left corner

Create Clients

1. Remember to select the ifix realm from the Keycloak console before proceeding

2. From the Clients section of Keycloak Admin Console, create a client

3. Provide a unique username for the client

4. Go to the client's settings

5. Change Access Type to confidential

6. Turn on Service Account Enabled

7. In the Valid Redirect URIs field provide the root URL of the iFIX Instance (Not important for our purposes but need to set it because it is mandatory)

8. And Save these changes

9. In the Service Account Roles tab, assign the role "fiscal-event-producer"

10. In the Mappers tab, create a new mapper to associate the client with a tenantId

    1. Select Mapper Type to be "Hardcoded claim"

    2. In Token Claim Name, write "tenantId"

    3. In Claim value, write the under which the client is being created. (For example, "pb")

    4. Set Name same as Token Claim Name i.e. "tenantId"

    5. Select Claim Json Type to be "String"

Now you can get the credentials from the Credentials tab and configure them in the client's system.

Overview

Department Entity service manages the department and its hierarchies metadata. It deals with department entity and department hierarchy only. Department Hierarchy store only hierarchies definition like metadata for department level and department entity stores actual department data with its ancestry information.

Version History

Current Version: 2.0.0

Prerequisites

Before you proceed with the configuration, make sure the following pre-requisites are met.

1. Java 8

2. MongoDB instance

3. Required Service Dependencies - Adapter-Master-Data-Service

Features

Department Hierarchy

It defines the hierarchy definition for the department.

• department id: It is the ID of the department from the department master

• level: It defines the depth of hierarchy of department-level

• parent: It provides details about department hierarchy parent (UUID)

Level Value Evaluation

1. Root level department hierarchy should not contain any parent value and the level value will be zero

2. When parent ID is having any value then we search parent in the department hierarchy record for hierarchy level evaluation.

3. Get level value from the parent department hierarchy and increment the current department hierarchy level value by one.

Department Entity

It contains department entity information along with its hierarchy level and also attaches master department information (department id - UUID). It keeps all child level information lists at every department node (department record). Leaf level department does not have any children info. Child list contains department entity ID list, which makes the current department entity parent of all children list (department ID list). This is how it maintains the department entity level.

Hierarchy Level Approach

Define the hierarchy level using the top to bottom because it has the parent's references. For department entity, create using the bottom-to-top approach. The leaf department entity does not have any child reference. When the department entity goes higher (parent) only then does it defines child reference in its child list.

Department Hierarchy Request Action

1. Create Department Hierarchy: We pass the current hierarchy level and its parent details along with master department and tenant information. It stores data as meta-information about hierarchy level for department entity data processing, it works as a reference meta index which will tell about hierarchy level information.

2. Search Department Hierarchy: It just provides a preview of department hierarchies by providing request parameters - tenant ID, hierarchy level or department hierarchy ID.

API List

Department Entity Request Action

1. Create Department Entity: It passes tenant Id, master department id, hierarchy level, its children list with department entity name and code. Tenant, hierarchy level and the master department is root info about the department entity. If the department entity does not contain any child, that means it is a leaf department entity it can only seek for its parent.

2. Search Department Entity: It can make search requests based on any department entity attribute but can't skip tenant information, it returns the whole department entity details along with its child information. It finds all ancestry information of the current department entity.

API List

Interaction Diagram

Environment

There will not be any environment variables required specific to the environment (migration).

Configurations and Setup

1. Update the DB and URI configurations in the dev.yaml, qa.yaml, prod.yaml file.

References and Notes

Department Hierarchy API With Example

Department Hierarchy defines the definition(Meta- Data) for actual department entity creation.

In the above image, the second row contains the actual Department Entities. Department Entity Create API - follows Bottom to Up approach.

In the above example data, the very first Department Entity will be created for “BARUWAL” with code “7278”. Below are the Steps to create Department Entity :

• The first Department Entity will be created for “BARUWAL” (Since the Department Hierarchy has been defined from “Department” to “GPWSC”. Please refer Department-Hierarchy-API-With-Example.)

• The details that need to be passed in Department Entity Create API is -

  • tenantId: This will be from Ifix Master Data Service

  • departmentId: This will be from Adapter Master Data Service

  • code: For the first Department Entity, as per the above data image, it is “7278”

  • name: For the first Department Entity, as per the above data image, it is “BARUWAL”

  • hierarchyLevel: For the first Department Entity, as per the above data image and defined Department Hierarchy, It will be “6”.(Bottom to Up)

  • children: For the first Department Entity, as per the above data image, it will be an empty set

• For the next Department Entity Creation - repeat step 2 by making sure the subsequent request has - next Department Entity - Code, name, hierarchyLevel (this will decrement by one for every next Department entity), children (this will contain all or at least one previously created Department Entity Id(s)).

For reference, the Bottom to Up request & response of Department Entity Create API will look like :

Request :[For Department Entity at Bottom - “BARUWAL”]

 {
    "requestHeader": {
    "ts": 1627193067,
    "version": "V1",
    "msgId": "Unknown",
    "signature": "NON",
    "userInfo": {
        "uuid": "e4fd96e8-3b6b-4e36-9503-0f14a01af39d"
    }
  },      
  "departmentEntity": {
    "tenantId": "pb",
    "departmentId": "3d9ef18a-361a-40cf-b142-dd6f998e1ad2",
    "code": "7278",
    "name": "BARUWAL",
    "hierarchyLevel": 6,
    "children": []
  }
  }

Response:

{
    "responseHeader": {
        "ts": 1627193067,
        "correlationId": "4f3d17a1-9608-4471-8fb0-dda7d02e79e8",
        "msgId": "Unknown",
        "status": "successful",
        "signature": "NON",
        "version": "V1"
    },
    "departmentEntity": [
        {
            "id": "7bdf9514-e2e5-4563-bfea-f5aaa41b2137",
            "tenantId": "pb",
            "departmentId": "3d9ef18a-361a-40cf-b142-dd6f998e1ad2",
            "code": "7278",
            "name": "BARUWAL",
            "hierarchyLevel": 6,
            "auditDetails": {
                "createdBy": "4faa912d-d531-485b-ad9b-0a9878f792b0",
                "lastModifiedBy": "4faa912d-d531-485b-ad9b-0a9878f792b0",
                "createdTime": 1637563256539,
                "lastModifiedTime": 1637563256539
            },
            "children": []
        }
    ]
}

Subsequent Request: [For Department Entity - “Kiratpur Sahib”]

 {
    "requestHeader": {
    "ts": 1627193067,
    "version": "V1",
    "msgId": "Unknown",
    "signature": "NON",
    "userInfo": {
        "uuid": "e4fd96e8-3b6b-4e36-9503-0f14a01af39d"
    }
  },      
  "departmentEntity": {
    "tenantId": "pb",
    "departmentId": "3d9ef18a-361a-40cf-b142-dd6f998e1ad2",
    "code": "S557",
    "name": "Kiratpur Sahib",
    "hierarchyLevel": 5,
    "children": [
      "7bdf9514-e2e5-4563-bfea-f5aaa41b2137"  //This will be the previously created Department Entity Id(“BARUWAL”).
    ]
  }
  }

Response:

{
    "responseHeader": {
        "ts": 1627193067,
        "correlationId": "aba27b23-e5ab-4b64-a634-942553393f2a",
        "msgId": "Unknown",
        "status": "successful",
        "signature": "NON",
        "version": "V1"
    },
    "departmentEntity": [
        {
            "id": "7ae82e0e-5575-4fac-9b39-d12eead75c65",
            "tenantId": "pb",
            "departmentId": "3d9ef18a-361a-40cf-b142-dd6f998e1ad2",
            "code": "S557",
            "name": "Kiratpur Sahib",
            "hierarchyLevel": 5,
            "auditDetails": {
                "createdBy": "ba28ebe8-ed9c-4cdb-9328-b8e227eb0342",
                "lastModifiedBy": "ba28ebe8-ed9c-4cdb-9328-b8e227eb0342",
                "createdTime": 1631122344993,
                "lastModifiedTime": 1631122344993
            },
            "children": [
                "7bdf9514-e2e5-4563-bfea-f5aaa41b2137"
            ]
        }
    ]
}

And so on for the next Department Entity(ies) ...

Note: Below is the observation from the above example :

1. Tenant Id and Department Id will be created before creating the Department Entity hence before Department Hierarchy Create. Tenant Id and Department Id will be the same in all hierarchy levels for a Single Department Hierarchy and corresponding Department Entity(ies).

2. In the children attribute, we can pass a set of previous Department Entity Ids (or at least one or empty in case of Bottom level Department Entity).

Update Existing Children List

When we have to update the existing children list of Department Entity then update the existing children list using mongodb command like below

• Find the parent department entity where the new children need to be added. We should know which is the parent Department Entity beforehand. For the same, Do search by name and hierarchy level and find the corresponding parent department entity’s id (UUID).

db.departmentEntity.find({"name" : "<parent_department_entity's_name>","hierarchyLevel": <parent_department_entity_hierarchy_level>});

• Append the department entity id at the end of the parent department entity children's list that we got from step 1. And count the length of the parent department entity children’s list (it is 0 based indexing ) that we are calling as ‘n’ here (it could be any integer number as per the children’s list size ) and then set it as

  "children.n": "<picked_from_step_1_query_department_entity_id>"

  e.g.

 db.departmentEntity.update({"_id": "<new_child_department_entity_id>"},{$set:
      {"children.n": "<Resulted_department_entity_id_1>",
      "children.n+1": "<Resulted_department_entity_id_2>"}
      })

Overview

This page provides details on the steps involved in cleaning up the iFIX core data from various environments. Follow the instructions specific to select environment listed below.

• Druid Data Clean-Up

• Mongo DB Data Clean-Up

• Postgres Data Clean-Up process.

Druid Data Clean-up

• Open the druid console in the respective environment.

• Go to Ingestion → Supervisors. And select the particular supervisor (fiscal-event)

  1. In Action, click on Terminate. This terminates the supervisor. Wait for a minute.

• Go to DataSources and select the particular data source name (fiscal-event). And scroll down to Action.

  1. Click on the Mark as unused all segments → Mark as unused all segments.

• Click on Delete unused segments (issue kill task) and enable the permission to delete.

Once the clean-up process is completed, follow the instructions here - IFIX Fiscal Event Post Processor | Druid-Sink to run the supervisor.

MongoDB Data Clean-up

• Connect to the playground pod and run the command below to connect with mongo DB. mongo --host <mongo-db-host>:<mongo-db-port> -u <mongo-db-username> -p <mongo-db-password>

• Use the ifix core DB; use <db name>

• Run the below command to delete all the data from fiscal_event collection. db.fiscal_event.remove({});

• If you want to delete all fiscal event records of a particular Gram Panchayat then run the command below.

Here let us assume, we have to delete "LODHIPUR" GP (Gram Panchayat) details (that is hierarchy level 6) in the DWSS department. Run the command below.

db.fiscal_event.remove({"departmentEntity.ancestry.6.name":{$eq:"LODHIPUR"}});

• If you want to delete a fiscal event record based on some other attributes, you have to write a custom mongo delete query.

PostgresDB Data Clean-up

• Connect to the playground pod and run the below command to connect with postgresDB.

  psql -h <psql-host> -p <psql-port> -d <psql-database> -U <psql-username>

• It prompts for a password. Enter the password.

• Run the below query to delete all the data from fiscal_event_aggregated.

DELETE FROM fiscal_event_aggregated;

• If you want to delete the particular Gram Panchayat details from fiscal event aggregated record, run the query below.

Here let us assume, we want to delete "LODHIPUR" GP (Gram Panchayat) details (that is hierarchy level 6) in the DWSS department.

DELETE FROM fiscal_event_aggregated where departmentEntity_ancestry_6_name='LODHIPUR';

iFix uses Keycloak to manage clients. Click on this link for instructions to set up a Keycloak server and manage clients.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

The platform consists of the following core services.

Master Data Service

The master data service manages the following master data:

• Government

• Department

• Department Entity

• Chart of Account

• Expenditure

• Project

Business Services

• Fiscal Event Service

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.