Overview

iFIX Adapter enables existing or new source systems to integrate seamlessly with iFIX. iFIX Adapter has been developed as a reference implementation for developers of source systems who want to integrate their departmental system with iFIX.

iFix Adapter works as a mediator between iFIX and its clients. This system will receive requests from the client system and converts the data into the iFIX fiscal event or associated formats.

Pre-requisites

Before you proceed with the configuration, make sure the following pre-requisites are met -

• Java 8

• Kafka server is up and running

• PSQL server is running

• Redis

• Following services should be up and running:

  • Client Service Like mgramseva-ifix-adapter

  • Target service IFIX- fiscal-event-service

  • Target Service IFIX-keycloak

Key Functionalities

• iFIX client requests pushed to IFIX

• Auth token is fetched from keycloak and cached. Token will be re-fetched 5 minutes before expiry

• Every push to iFIX is recoded with http status

  • status series 200 series considered success

  • status 400 are marked client error and reported back to client

  • status 500 resubmitted by scheduler

Deployment Details

1. Update the keycloak credentials client-id and secrets in the environment file

2. Map the coa in HeadCodeToCoaMapping.yml

3. Map project in ProjectMapping.yml

4. Deploy the latest version of ifix-reference-adapter

Configuration Details

1. Update Key cloak credentials in dev.yaml, qa.yaml, prod.yaml according to environment

Interaction Diagram

Overview

Ifix-Adapter is a system that works as a mediator between iFIX and its clients. This system will receive requests from the client system and convert the data in the iFIX required format This document contains the details on how to set up the iFIX-adapter service and describes the functionalities it supports. It supports multiple events (Event Array) in a single request.

Pre-requisites

Before you proceed with the configuration, make sure the following pre-requisites are met -

• Java 8

• Kafka server is up and running

• PSQL server is running

• Redis

• Following services should be up and running:

  • Client Service Like mgramseva-ifix-adapter

  • Target service IFIX- fiscal-event-service

  • Target Service IFIX-keycloak

  • Adapter master data service

Key Functionalities

• IFIX client requests are pushed to IFIX.

• The authentication token is fetched from keycloak and cached. Token is re-fetched 5 minutes before expiry.

• project_id from request data is getting treated as Department Entity Code to fetch Department Entity.

• COA Code fetched from COA Mapping table by client code and cached it in Redis Server.

• Every push to IFIX is recorded in the table with HTTP status

  • status series 200 considered success

  • status 400 are marked client error

• It collects projectId form request data and treats it as department_entity_code and calls search API to Department Entity Service. It always expects it will receive only one Department Entity against a single department_entity_code, if it finds multiple raise an error message.

• One project can have multiple department entities but vice-versa cannot be true. In case of multiple projects for one department entity - the system will raise an error message.

Deployment Details

Deploy the latest version of the ifix-reference-adapter.

Configuration Details

2. Map clientcode, ifixcoacode, ifixid in ifix_adapter_coa_map table

   • “clientcode” is the tax head like “WATER_CHARGES” or ‘10011’ used in IFIX client like mgramseva

   • “ifixcoacode” is the 16 digit glcode in IFIX. 16 digit code is mapped then this can be ported to any environment like dev to qa ,or qa to uat or from uat to prod. Prefer mapping ifixcoacode

   • example is INSERT INTO public.ifix_adapter_coa_map(id, clientcode, ifixcoacode, ifixid, tenantid) VALUES (1,'10101', '0215-01-102-00-00-01', '6cbcb4a1-2431-4f78-89d7-b4f0565aba37', 'pb');

3. state.goverment.code set this value to the clients top level tenant_id

Interaction Diagram

Reference Docs

Doc Links

API List

Deployment of iFix Adapter

Post infra setup (Kubernetes Cluster), there starts the deployment process.

Pipeline as code is a practice of defining deployment pipelines through source code, such as Git. Pipeline as code is part of a larger “as code” movement that includes infrastructure as code. Teams can configure builds, tests, and deployment in code that is trackable and stored in a centralized source repository. Teams can use a declarative YAML approach or a vendor-specific programming language, such as Jenkins and Groovy, but the premise remains the same.

A pipeline as code file specifies the stages, jobs, and actions for a pipeline to perform. Because the file is versioned, changes in pipeline code can be tested in branches with the corresponding application release.

The pipeline as code model of creating continuous integration pipelines is an industry best practice, but deployment pipelines used to be created very differently.

The deployment process has got 2 stages and 2 modes. We can see the modes first and then the stages.‌

The 2 Modes of Deployment

Essentially, iFix adapter deployment means that we need to generate Kubernetes manifests for each individual service. We use the tool called helm, which is an easy, effective and customizable packaging and deployment solution. So depending on where and which env you initiate the deployment there are 2 modes that you can deploy.‌

1. From Local machine - whatever we are trying in this sample exercise so far.

2. Advanced: Setup CI/CD System like Jenkins - Depending on how you want to setup your CI/CD and the expertise the steps will vary, however here you can find how we eGov has set up a exemplar CI/CD on Jenkins and the pipelines are created automatically without any manual intervention.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Follow the steps below to create the Adapter Master Data. Individual Adapter Service documents can be accessed here.

1. Enter valid details along with Tenant ID to create the Department. Once the tenant ID is created, the user receives a response with an ID and related details. This ID is the Department ID.

2. Enter valid details along with Tenant ID to create the Expenditure. Once the tenant ID is created, the user receives a response with an ID and related details. This ID is the Expenditure ID.

3. Enter valid hierarchy details of the Master Department to create Department Hierarchy. Check this document for more information.

4. Provide valid details to create Department Entity. Check this document for more information.

5. On successful completion of steps 1 to 4, enter valid details to create a project. Check this document for more information.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Overview

Adapter master data service maintains information on Department, Expenditure and Projects. We can create these details and search for the same details based on the given parameters/request data.

Version

Current version : 1.0.0

Pre-requisites

Before we proceed with the configuration, make sure the following pre-requisites are met

1. Java 8

2. MongoDB instance

3. Required service dependency - Department entity service

Features

It creates secure endpoints for the master data service. The access token is required to create any master data. The subsequent sections on this page discuss the service details maintained by the master data service.

Department

Maintains the create and search department details. The following information is passed while creating the department - the Government ID, department code, department name, parent department if any. Searching the department details is on given parameters like IDs, Government ID, department code, department name.

API List

Expenditure

Maintains the expenditure details And provide create and search functionality. For creating the expenditure, the following details are required - the Government ID, the department ID, code, name, type (can be "SCHEME", "NON_SCHEME") details. While searching the expenditure details, pass the given parameters like IDs, Government IDs, names, code.

API List

Project

Maintains the project details and provide create and search functionality. The following details are required to create the project - Government, name, code, expenditure ID, the department entity ID(s), location IDs. While searching, pass the IDs, Government ID, name, code, expenditure ID, location ID.

API List

Interaction Diagram

Environment

No environment-specific variables are required for the environment (migration).

Configurations and Setup

Update the DB and URI configurations in the dev.yaml, qa.yaml, prod.yaml file.

References and Notes

Master Project API With Example

Project Create API creates the project when the Master data details (COA, Government, Expenditure, Department) and Department Entity have been created. COA And Government have to be created in iFIX core Master data service.

Project Create API takes the below attributes in request :

1. tenantId: This is the Id that will be defined while creating the Ifix core Master Government Service.

2. expenditureId: This is the Id that will be generated while creating the Adapter Master Expenditure Service.

3. code: This is the project code that needs to be created.

4. name: This is the project name that needs to be created.

5. departmentEntityIds: This is the Department Entity Ids. If we have to create a project at hierarchy level 1 then we need to pass the Department Entity Id of that corresponding level. It depends on the Department hierarchy level on which the project has to be created and hence the same level Department Entity Id. You can pass a list of departmentEntityIds and can create the same project.

For reference, Below is a Dummy project create example:

Request :

{
  "requestHeader": {
    "ts": 1627193067,
    "version": "2.0.0",
    "msgId": "Unknown",
    "signature": "NON",
    "userInfo": {
        "uuid": "e4fd96e8-3b6b-4e36-9503-0f14a01af39d"
    }
  },
  "project": {
    "tenantId": "pb",
    "code": "7330_S557_DIV23SD02",
    "name": "DOLOWAL UPPER",
    "expenditureId": "13ef1c53-702d-43b5-9f97-43fa03c145c5",
    "departmentEntityIds":  ["295180a0-4d60-4805-a77f-92143bd115b4","901f76a8-1911-4960-b389-57b62bd4dcdb"
    ]
  }
}

Response:

{
    "responseHeader": {
        "ts": 1627193067,
        "correlationId": null,
        "msgId": "Unknown",
        "status": "successful",
        "signature": "NON",
        "version": "2.0.0"
    },
    "project": [
        {
            "id": "1fb13f0a-588f-476d-b839-afddae7980ef",
            "tenantId": "pb",
            "code": "7330_S557_DIV23SD02",
            "name": "DOLOWAL UPPER",
            "expenditureId": "13ef1c53-702d-43b5-9f97-43fa03c145c5",
            "departmentEntityIds": [
                "295180a0-4d60-4805-a77f-92143bd115b4",
                "901f76a8-1911-4960-b389-57b62bd4dcdb"
            ],
            "locationIds": null,
            "auditDetails": {
                "createdBy": "e4fd96e8-3b6b-4e36-9503-0f14a01af39d",
                "lastModifiedBy": "e4fd96e8-3b6b-4e36-9503-0f14a01af39d",
                "createdTime": 1646726943845,
                "lastModifiedTime": 1646726943845
            }
        }
    ]
}

Overview

Ifix-Adapter is a system that works as a mediator between iFix and its clients. This system will receive requests from the client system and convert the data in the Ifix required format. This document contains the details about how to set up ifix-adapter service and describes the functionalities it provides.

Pre-requisites

Before you proceed with the configuration, make sure the following pre-requisites are met -

• Java 8

• Kafka server is up and running

• PSQL server is running

• Redis

• Following services should be up and running:

  • Client Service Like mgramseva-ifix-adapter

  • Target service IFIX- fiscal-event-service

  • Target Service IFIX-keycloak

  • IFIX master-data-service

Key Functionalities

• IFIX client requests are pushed to IFIX

• The authentication token is fetched from keycloak and cached. Token will be re-fetched 5 minutes before expiry

• project id is fetched from IFIX and cached

• COA id fetched from IFIX and cached

• Every push to IFIX is recorded in the table with HTTP status

  • status series 200 considered success

  • status 400 are marked client error and reported back to the client

  • status 500 resubmitted by the scheduler

Deployment Details

Deploy the latest version of ifix-reference-adapter

Configuration Details

1. Update Key cloak credentials in dev.yaml,qa.yaml,prod.yaml according to the environment The credentials are “keycloak.credentials.clientid and “keycloak.credentials.clientsecret” Example is given in here and here

2. Map clientcode, ifixcoacode, ifixid in ifix_adapter_coa_map table

   1. “clientcode” is the tax head like “WATER_CHARGES” or ‘10011’ used in IFIX client like mgramseva

   2. “ifixcoacode” is the 16 digit glcode in IFIX. 16 digit code is mapped then this can be ported to any environment like dev to qa ,or qa to uat or from uat to prod. Prefer mapping ifixcoacode

   3. Another way is to map the IFIX COA ID itself. Since these are generated ids you cant port to other environments. ID mapping has to be done for every environment.

   4. Preference is given to COA Code, if it is null ID will be used

   5. example is INSERT INTO public.ifix_adapter_coa_map( id, clientcode, ifixcoacode, ifixid, tenantid) VALUES (1,'10101', '0215-01-102-00-00-01', '6cbcb4a1-2431-4f78-89d7-b4f0565aba37', 'pb');

3. If client “project code” and IFIX project code are the same then no need for mapping. If it is different then map clientprojectcode, ifixprojectid in ifix_adapter_project_map table. Ideally, you should keep both codes the same for getting meaningful data on the dashboard. This way you don't have to do any mapping for project code for any environment. But if for any reason you have different project codes in IFIX and its client or has multiple projects having the same project code then only go for this mapping. The adapter will first check in the IFIX for the supplied “projectCode”, If found it will use it and caches it. If multiple projects or not found it will look into this table for mapping

   1. example is INSERT INTO public.ifix_adapter_project_map( id, clientprojectcode, ifixprojectid, tenantid) VALUES (1, '7374', 'e42db9bb-8427-40a6-9939-4f2189d032bf','pb');

4. state.goverment.code set this value to the clients top level tenantid

Interaction Diagram

Reference Docs

Doc Links

API List

The 2 Stages

Stage 1 : Jenkins Setup

Post infra setup (Kubernetes Cluster), We start with deploying the Jenkins and kaniko-cache-warmer.

Pre-requisites

• Sub Domain to expose CI/CD URL

• GitHub Oauth App

• GitHub User ssh key

• With GitHub user generate a personal read only access token

• Docker hub account details (username and password)

• SSL Certificate for the sub-domain

Prepare an <ci.yaml> master config file and <ci-secrets.yaml>, you can name this file as you wish which will have the following configurations.

• credentials, secrets (You need to encrypt using sops and create a ci-secret.yaml separately)

• Check and Update ci-secrets.yaml details (like github Oauth app clientId and clientSecret, GitHub user details gitReadSshPrivateKey and gitReadAccessToken etc..)

• To create Jenkins namespace mark this flag true

• Add your env's kubconfigs under kubConfigs like https://github.com/misdwss/iFix-DevOps/blob/mgramseva/deploy-as-code/helm/environments/ci-secrets.yaml#L19

• KubeConfig env's name and deploymentJobs name from ci.yaml should be the same

• Update the CIOps and DIGIT-DevOps repo name with your forked repo name and provide read-only access to github user to those repo's.

• SSL Certificate for the sub-domain

cd iFix-DevOps/deploy-as-code/egov-deployer

kubectl config use-context <your cluster name>
go run main.go deploy -c -e ci 'jenkins,kaniko-cache-warmer,nginx-ingress'

You have launched the Jenkins. You can access the same through your sub-domain which you configured in ci.yaml.

The Jenkins CI pipeline is configured and managed 'as code'.

​Example URL - https://<Jenkins_domain>​

Stage 2: Continuous Integration (CI)

Since there are many services and the development code is part of various git repos, you need to understand the concept of cicd-as-service which is open-sourced. This page also guides you through the process of creating a CI/CD pipeline.

As a developer - To integrate any new service/app to the CI/CD below is the starting point:

Once the desired service is ready for the integration: decide the service name, type of service, whether DB migration is required or not. While you commit the source code of the service to the git repository, the following file should be added with the relevant details which are mentioned below:

Build-config.yml –It is present under the build directory in each repository

https://github.com/misdwss/punjab-mgramseva/blob/master/build/build-config.yml

This file contains the below details which are used for creating the automated Jenkins pipeline job for your newly created service.

config:
  - name: < Name of the job, foo/bar would create job named bar inside folder foo>
    build:
    - work-dir: < Working directory of the app to be built >
      dockerfile: < Path to the dockerfile, optional, assumes dockerfile in working directory if not provided >
      image-name: < Docker image name  >

While integrating a new service/app, the above content needs to be added in the build-config.yml file of that app repository. For example: If we are onboarding a new service called egov-test, then the build-config.yml should be added as mentioned below.

config:   
  - name: core-services/egov-test     
    build:     - work-dir: egov-test       
    dockerfile: build/maven/Dockerfile       
    image-name: egov-test

If a job requires multiple images to be created (DB Migration) then it should be added as below,

config:   
  - name: core-services/egov-test     
    build:     
    - work-dir: egov-test       
      dockerfile: build/maven/Dockerfile       
      image-name: egov-test     
    - work-dir: egov-test/src/main/resources/db       
      dockerfile: build/maven/Dockerfile       
      image-name: egov-test-db

Note - If a new repository is created then the build-config.yml should be created under the build folder and then the config values are added to it.

The git repository URL is then added to the Job Builder parameters

When the Jenkins Job => job builder is executed the CI Pipeline gets created automatically based on the above details in build-config.yml. Eg: egov-test job will be created under the core-services folder in Jenkins because the “build-config was edited under core-services” And it should be the “master” branch only. Once the pipeline job is created, it can be executed for any feature branch with build parameters (Specifying which branch to be built – master or any feature branch).

As a result of the pipeline execution, the respective app/service docker image will be built and pushed to the Docker repository.

Job Builder – Job Builder is a Generic Jenkins job that creates the Jenkins pipeline automatically which are then used to build the application, create the docker image of it and push the image to the docker repository. The Job Builder job requires the git repository URL as a parameter. It clones the respective git repository and reads the build/build-config.yml file for each git repository and uses it to create the service build job.

‌Check git repository URL is available in ci.yaml​‌‌

If git repository URL is available build the Job-Builder Job

If the git repository URL is not available ask the Devops team to add it.

Continuous Deployment (CD)‌

The services deployed and managed on a Kubernetes cluster in cloud platforms like AWS, Azure, GCP, OpenStack, etc. Here, we use helm charts to manage and generate the Kubernetes manifest files and use them for further deployment to the respective Kubernetes cluster. Each service is created as charts which will have the below-mentioned files in them.

billing-service/   
# Directory – name of the service/appChart.yaml         
# A YAML file containing information about the chartLICENSE            
# OPTIONAL: A plain text file containing the license for the chartREADME.md          # OPTIONAL: A human-readable README filevalues.yaml        
# The default configuration values for this charttemplates/         
# A directory of templates that, when combined with values, will generate valid Kubernetes manifest files.

To deploy a new service, we need to create the helm chart for it. The chart should be created under the charts/helm directory in iFix-DevOps repository.

Github repository     https://github.com/misdwss/iFix-DevOps

We have an automatic helm chart generator utility that needs to be installed on the local machine, the utility prompts for user inputs about the newly developed service (app specifications) for creating the helm chart. The requested chart with the configuration values (created based on the inputs provided) will be created for the user.

‌ Name of the service? test-service Application Type? NA Kubernetes health checks to be enabled? Yes Flyway DB migration container necessary? No, Expose service to the internet? Yes, Route through API gateway [zuul] No Context path? hello‌

The generated chart will have the following files.

create Chart.yaml
create values.yaml
create templates/deployment.yaml
create templates/service.yaml
create templates/ingress.yaml

This chart can also be modified further based on user requirements.

The Deployment of manifests to the Kubernetes cluster is made very simple and easy. We have Jenkins Jobs for each state and are environment-specific. We need to provide the image name or the service name in the respective Jenkins deployment job.

Enter a caption for this image (optional)

Enter a caption for this image (optional)

‌The deployment Jenkins job internally performs the following operations,‌

• Reads the image name or the service name given and finds the chart that is specific to it.

• Generates the Kubernetes manifests files from the chart using the helm template engine.

• Execute the deployment manifest with the specified docker image(s) to the Kubernetes cluster.

The 2 Stages

Essentially, there are 2 stages that should allow you to use the full potential of DeploymentConfig and pipeline-as-code.

Stage 1: Clone the DevOps repo, choose your iFix product branch as iFix-adapter.‌

Prepare an <env.yaml> master config file, you can name this file as you wish which will have the following configurations, this env file need to be in line with your cluster name.‌

• each service global, local env variables

• credentials, secrets (You need to encrypt using sops and create a <env>-secret.yaml separately)

• Number of replicas/scale of individual services (Depending on whether dev or prod)

• mdms, config repos (Master Data, ULB, Tenant details, Users, etc)

• sms g/w, email g/w, payment g/w

• GMap key (In case you are using Google Map services in your PGR, PT, TL, etc)

• S3 Bucket for Filestore

• URL/DNS on which the DIGIT will be exposed

• SSL Certificate for the above URL

• End-points configs (Internal/external)

Stage 2: Run the iFix_Adapter_setup deployment script and simply answer the questions that it asks.

cd iFix-DevOps/deploy-as-code/egov-deployer

go run ifix_adapter_setup.go

#Be prepared for the following questions
1. Do you have the Kubernetes Setup?
2. Provide the path of the intented env kubeconfig file
3. Which version of the DIGIT that you want to install
4. What mGramSeva Modules that you choose to install (Choose PGR)
5. All, done, Now do you want to preview the deployment manifests 
6. Are you good to proceed with the DIGIT Installation

All Done.Essentially, DIGIT deployment means that we need to generate Kubernetes manifests for each individual service. We use the tool called helm, which is an easy, effective and customizable packaging and deployment solution. So depending on where and which env you initiate the deployment there are 2 modes that you can deploy.