Release Summary

iFIX Core v2.3 contains the following changes.

• The data stores have been shifted:

  1. From MongoDB to PostgreSQL

  2. From Druid to ElasticSearch

Enhancements

Document Resources and Links

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Download our ebook -

Mission

Enable governments to continuously improve their financial health, ensure a functioning and responsive service delivery system, and deploy available funds in ways that create an environment for businesses, communities, and individuals to prosper and stay healthy.

Overview

iFIX is an open-source fiscal information exchange platform. The platform enables connected applications to exchange standardised fiscal events e.g. Demand, Receipts, Bills, and Payments. The fiscal event consists of attributes explaining the details of why, who, what, where and when it happened.

Need For iFIX

As government service delivery requires multiple actors and interactions to come together across different levels, visibility of information is critical to bring down the cost of coordination. This includes real-time information on the financial health of a government agency/department- expenditure, revenue and availability of funds.

Public finance management faces significant challenges in terms of promoting accountability and transparency. The problem is more on account of siloed information structures that restrict the scope to get a broader view of the flow of funds or data across agencies and stakeholders.

The flow of information is both slow and limited, resulting in a number of gaps and breaks in the PFM processes. This is essentially attributed to the lack of information standards and exchange mechanisms that make it difficult for the seamless flow of data across agencies.

iFIX allows departments to share fiscal information from existing systems without having to invest in multiple integrations for different stakeholder requirements. The platform plays a pivotal role in driving efficient and performance-driven financial planning across all levels of governance. Real-time availability of financial information to stakeholders facilitates data-driven deployment of public funds and policy-making. The platform specification sets the base for the real-time exchange of fiscal information across funding and implementing agencies.

Platform Scope

The iFIX platform identifies and resolves Public Financial Management issues like delays in funds flow, floating of unutilised funds, problems of low data fidelity, the administrative burden of implementation and outcome-oriented funding using the platform approach and associated policy reforms.

The platform helps achieve overall fiscal discipline, allocation of resources to priority needs, efficient and effective allocation of public services. Public financial management includes all phases of the budget cycle, including the preparation of the budget, internal control and audit, procurement, monitoring and reporting arrangements, and external audit.

iFIX helps address the following challenges with respect to the information on government fiscal programmes

• Unlocking fiscal and operational data locked in silos

• Generating relevant information based on common data standards

• Ensuring the information generated is credible, reliable and verifiable

• Accelerating fund flows by providing trusted, usable data and reducing the interdepartmental coordination time

The present-day public finance system supports multiple types of information flows that are unique and accessed by multiple sources. iFIX simplifies the network and fiscal information flows through the use of standardised formats and protocols.

Benefits

No change to the source system: Enabling information exchange through iFIX does not necessitate any change to the source systems, therefore ensuring minimal disruption to current ways of working.

Scalability across governance levels and geographies: iFIX approach for standardisation can be applied to multiple entities at the same level and across different levels. This gives the platform infinite applicability within the PFM context to provide visibility on aggregated and disaggregated information.

Objectives

The standardised fiscal event and exchange mechanism enable various government agencies e.g. various departments, local governments, autonomous bodies, national government, and development agencies to -

• exchange fiscal data much like email systems exchange data with each other

• ease flow of fiscal information resulting in better planning, better execution, better accounting and better auditing thus transforming the entire PFM cycle

• promote transparency and improve accountability while ensuring real-time access to the financial health of the government stakeholders

Approach

iFIX offers a platform that standardises fiscal event data and the capabilities to exchange information across multiple sources. The platform is built as a digital public good and is open source with minimal specifications. The design incorporates the security and privacy of all data and users and ensures performance at scale.

Data standards streamline the flow of information paving the way for timely exchange and cost-effective means of managing fiscal events. The iFIX platform establishes the standards and specifications for fiscal event data that make it easier to exchange fiscal information between funding and implementing agencies. Event data is captured in real-time and at the micro level which ensures there is no intentional or unintentional data loss. Any transaction triggers a fiscal event that is accessible to integrated agencies for necessary approvals.

The standardized data exchange platform approach ensures that iFIX is not a replacement for the existing finance system. It is just a coordination and visibility layer between agencies within government and across governments.

Visit the Products pages to learn more about the iFIX platform capabilities.

Useful Links

Contact Us

Migration to DIGIT architecture primarily involves the following changes to iFIX-Core:

1. Transactional DB is switched from MongoDB to PostgreSQL

2. Analytics DB is switched from Druid to ElasticSearch

   1. Correspondingly, the dashboard is switched from Metabase to the DSS dashboard

This migration will require the following tasks to be performed:

iFIX Core

1. Copy data to ElasticSearch

   • Fiscal Events

2. Copy data to PostgreSQL

   • iFIX Master Data Service

     • Government

     • Chart of Accounts

   • Fiscal Events

iFIX Adapter

Apart from iFIX-Core, the other services relying on MongoDB were also migrated to PostgreSQL. So they will also need to be migrated.

1. Department Entity Service

   • Department Hierarchy Level

   • Department Entity

2. Adapter Master Data Service

   • Expenditure

   • Department

   • Project

Principles

The platform is built as a Digital Public Good and follows a key set of design principles listed below.

• Single Source of Truth - Data resides in multiple systems across departments and getting an integrated, consistent and disaggregated view of data is imperative.

• Federated - Central, State and Local Governments represent the federated structure of government. This federated structure must be taken into cognisance while designing systems that enable intergovernmental information exchange.

• Unbundled - Break down complex systems into smaller manageable and reusable parts that are evolvable and scale independently.

• Minimum - Minimum data attributes that have a well-defined purpose is mandatory; everything else is optional.

• Privacy and Security - Ensuring the privacy of citizens, employees and users while ensuring the system's security.

• Performance at Scale - The system is designed to perform at scale.

• Open Standards - The system is based on open standards so that it is easy to discover, comprehend, integrate, operate and evolve.

Approach

The platform applies the microservice architecture concept. Access the list of services supported by iFIX here.

Standards

The services are designed to comply with standards which consist of the Common Information Model and APIs. Click here to view the details.

Setup

To set up the platform, follow the installation steps listed here.

With the iFIX v2.3-alpha update, some of the DIGIT core services also need to be deployed. The builds of the same are also listed below. These have been picked from DIGIT-v2.7.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Release Summary

MGramSeva iFix Adapter v2.3 has the below changes.

• Data store moved from MongoDB to ElasticSearch

• Non-backward compatible minor change to the /_create and /_update of department-entity and project master.

All content on this page by is licensed under a .

Release Summary

IFIX 2.3-alpha release details and highlights are available in the below release notes.

All content on this page by is licensed under a .

1. Deploy ifix-migration-toolkit-db:develop-b7a2050-13 to ifix environment

2. Port forward to the ifix-migration-toolkit pod

Instruction

The following common request body is used to hit the ifix-migration-toolkit endpoints:

{
    "requestHeader": {},
    "tenantId": "<tenant-id>"
}

Migrate ifix-master-data-service

Migrate Government

Government master data is moved to MDMS. Please manually copy the government(tenant) data to the tenants master of tenant module. Here is a link to a sample tenants master.

Migrate Chart of Account

1. First, deploy the latest image of the ifix-master-data-service

   1. (So that the tables get created in PostgreSQL.)

2. Deploy plain search docker image of ifix-master-data:

   Copy

   ifix-master-data-service-db:mongodb-plainsearch-b12bf24-7

3. Ensure ifix-master-data-persister is configured in the egov-persister

4. Hit the Chart of Account migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/coa/v1/_migrate

Migrate fiscal-event-service: Fiscal Events

With this, we will be performing migration from MongoDB to both, PostgreSQL and ElasticSearch. Data will be read from MongoDB using fiscal-event-service's plain search api, and pushed to a kafka topic.

1. From there, the data will reach PostgreSQL using egov-persister.

2. From kafka-topic data will pass through ifix-es-pipeline. After that ifix-es-pipeline will post the processed data to the next kafka topic in the pipeline. From there, egov-indexer will read from the topic and index the data onto ElasticSearch.

Instructions:

1. First, deploy the latest image of the fiscal-event-service

   1. (So that the tables get created in PostgreSQL.)

2. Follow the instructions mentioned in the ifix-es-pipeline project's README to setup the ES mapping before performing the migration.

3. Deploy plain search docker image of fiscal-event-service:

   Copy

   egovio/fiscal-event-service-db:mongodb-plainsearch-1bca621-7

4. Deploy the latest image of ifix-es-pipeline

5. Ensure fiscal-events-persister is configured in egov-persister

6. Ensure ifix-fiscal-event-indexer is configured in egov-indexer

7. (Ensure ifix-migration-progress is configured in egov-persister to record the progress of the migration. In case of any failures during migration, with this, it will resume the migration from where it left off. )

8. Hit the Fiscal Event migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/fiscal_event/v1/_migrate

Information Model

1. Government

2. Department

3. Department Entity (For example, Ropar Water Department)

4. Chart of Account

5. Expenditure

6. Project

7. Fiscal Event

APIs (v 0.1)

• Business Services

• Fiscal Event

  • Post-Fiscal Event

  • Search Fiscal Event

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

With this update, the iFIX Adapter services now depend on some of the DIGIT Core services. egov-persister should be deployed and configured.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Overview

iFix is a fiscal data exchange platform that enables the exchange of standardized fiscal data between various agencies. iFix is designed to enable the exchange of fiscal data between various agencies and ensure the visibility of fiscal data. iFix makes it possible to chain the fiscal data with each other and establish a chain of custody for the entire lifecycle from budgeting to accounting.

From the iFIX perspective, there are two types of agencies

1. Fiscal Data Provider - posts the fiscal data into iFix using well-defined formats.

2. Fiscal Data Consumer - can query the fiscal data.

Both these roles are interchangeable.

Registration

Providers and consumers need to register on iFix before they can post or query fiscal data. To register the concerned person from the agency must be provided with the following information on the iFix portal - Name of the Agency, Contact Person, Name, Contact Person’s Phone Number, and Contact Person’s Official Email Address. The OTP sent to the email address for the person should be used to register.

System Registration & Access Key Generation

Registered users -

• logs in to the iFix portal using the official email address

• registers one or more systems as a provider or consumer or both

• provides the name of the system

• a unique ID is generated for each system (example: mgramseva@punjab.ifix.org)

• a secret API key is also generated for each system - use this key to post or query fiscal data

• The API key can be regenerated if required - only one API key is active at a given point in time

The portal provides the ability to generate new keys for each system.

Posting

Fiscal data providers post the fiscal data in two ways.

• A fiscal message - is directed to a specific consumer and is delivered to intended consumers. These messages are available for query by intended consumers only.

• A fiscal event - iFix stores the events for consumers to query

Fiscal Event consists of

• Header

  1. From

  2. To

  3. Date of Posting

• Body

  1. Fiscal Event Type e.g. Revenue, Expenditure, Debt

  2. Fiscal Event Subtype

     • Revenue - Estimate, Plan, Demand, Receipt, Credit

     • Expenditure - Estimate, Plan, Bill, Payment, Debit

     • Debt - in progress - will be provided later.

  3. Array of fiscal line items

     • Amount

     • CoA

     • Location Code - from Location Registry

     • Program Code - from Program Registry

     • Project Code - from Project Registry

     • Administrative Hierarchy Code from Administrative Hierarchy Registry

     • Start Date of Period

     • End Date of Period

     • ….

     • ….

  4. Attachment - Attachments consist of additional attributes like key-value pairs e.g. Account Number, Correlation ID or Documents

  5. Signature - Fiscal messages can be signed by multiple agencies and add the signature to the Signature Array that contains the below-mentioned values -

     • Array of Signature

       1. System

       2. eSign - Signed Value of the Fiscal Event/Message Body using the System Key

       3. Purpose - Acknowledgement or Approval or Rejection

       4. Comments

       5. Date of Signing

Reversal

Data providers can reverse a previous fiscal message or event. The data provider reverses the data by posting the same event with a negative amount in the line item(s). The data consumers should handle reversals appropriately.

Querying

Data consumers can query fiscal data. They can query Messages - the unread messages that have been delivered to them. When consumers read the unread messages, these messages are marked as read. Events - Consumers can also query fiscal events posted by other data providers.

Reference Data Management

1. Location

2. Administrative

3. Chart of Account …. … …

Government

Department

Chart of Account

Expenditure

Project

Amount

Fiscal Event

List Of Services

• Tenant Management

• Master Data Management (Chart of Accounts)

• Fiscal Project Management

• Immutable Fiscal events

  • Type of events: Bill, Demand, Payment, Receipt (budgetary, cash and accrual events in the budget cycle)

  • Deduplication Reversals, Adjustments - Planned

• Fiscal Messaging and Subscription (coordination between different entities) - Planned

• Fiscal reporting and analytics - Planned

• Reference Adaptor

• Reference Dashboard

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

iFIX Fiscal Event Service APIs

iFIX Master Data APIs

Introduction

iFIX/mGramSeva is a microservices-based distributed cloud-native application. The microservices streamline processes to meet outcomes at scale and speed. Each of the microservices is dockerized and deployed on Kubernetes infrastructure.

Pre-reads

It is essential to understand some of the key concepts, benefits and best practices of Kubernetes platform before we understand the deployment of the iFix/mGramSeva.

• Know the basics of Kubernetes: ​

• Know the commands

• Know Kubernetes manifests: ​

• Know how to manage env values, secrets of any service deployed in Kubernetes ​

• Know how to port forward to a pod running inside k8s cluster and work locally ​

• Know sops to secure your keys/creds: ​

1. Choose Installation Type

Choose the target infra type and follow the instructions to set up a Kubernetes cluster before moving on to the deployment.

2. Deployment

Before we begin the deployment, it is important to understand the deployment architecture that starts from the source code to the production-ready stage. Deploying and managing Kubernetes have emerged as a streamlined way to deploy containers in the cloud infrastructure. When running Kubernetes at scale, managing, operating, and scaling its infrastructure to maximize cluster utilization can be challenging. There are too many parameters the development team needs to manage and configure. This includes selecting the best instance type and size, determining when to scale up or down, and making sure all of the containers are scheduled and running on the best instances — and that is even before starting to think about cost resource optimization.

The simplest way to get started with the deployment process is to manage deployment configuration as code. Each service deployment configuration is defined as Helm charts and deployed into the Kubernetes cluster. We can collocate the deployment-as-code as source code, leveraging all the benefits of source control including change tracking and branching, and then package it. So below is the source code repo that contains all the deployment-as-code for iFIX.

1. mGramSeva Installation

2. iFIX-adapter Installation

3. iFIX Ref Dashboard Installation

3. Destroy the Setup

Clean up the cluster Setup using the command below, if required. This deletes the entire cluster and other cloud resources that were provisioned for the mGramSeva Infra Setup.

Conclusion

All done, the infra on local, cloud, and deployment of iFIX into the Kubernetes cluster is completed successfully.

1. Deploy ifix-migration-toolkit-db:develop-2557812-9 to mgramseva environment.

2. Port forward to the ifix-migration-toolkit pod

Instruction

The following common request body is used to hit the ifix-migration-toolkit endpoints:

{
    "requestHeader": {}
    "tenantId": "<tenant-id>"
}

Migrate ifix-department-entity-service

1. First, deploy the latest image of the ifix-department-entity-service

   1. (So that the tables get created in PostgreSQL.)

2. Deploy plain search docker image of ifix-department-entity-service:

   Copy

   egovio/ifix-department-entity-service-db:mongodb-plainsearch-b12bf24-4

3. Ensure department-entity-persister.yml is configured in the egov-persister.

4. (Ensure department-entity-migration-progress is configured in egov-persister to record the progress of the migration. In case of any failures during migration, with this, it will resume the migration from where it left off. )

5. Hit the Department Hierarchy migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/department_hierarchy/v1/_migrate

6. Hit the Department Entity migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/department_entity/v1/_migrate

Migrate adapter-master-data-service

1. First, deploy the latest image of the ifix-department-entity-service

   1. (So that the tables get created in PostgreSQL.)

2. Deploy plain search docker image of ifix-department-entity-service:

   Copy

   egovio/adapter-master-data-service-db:mongodb-plainsearch-b12bf24-5

3. Ensure adapter-master-data-service.yml is configured in egov-persister.

4. Hit the Department migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/department/v1/_migrate

5. Hit the Expenditure migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/expenditure/v1/_migrate

6. Hit the Project migrate endpoint of the migration-toolkit

   Copy

   /ifix-migration-toolkit/project/v1/_migrate

Quickstart Installation helps you jump-start with the mGramSeva basic installation with limited functionalities.‌

mGramSeva is a distributed microservice-based platform that comprises many services that are containerized. Depending upon the required features, the specific services can be run on any container-supported orchestration platform like docker-compose, Kubernetes, etc.‌

The Quickstart guide covers the installation steps for basic services to get the platform up. Before setting up mGramSeva, create a lightweight Kubernetes cluster called k3d on a local machine with specified H/W requirements. The H/W requirements are listed below to ensure before we proceed further.‌

1. Infra Setup

To provision a lightweight Kubernetes cluster, please follow the instructions below in context to your OS and install the k3d on your machine.‌

H/W or VM Size

• min 4 vCPUs (recommended 8)

• min 8GiB of RAM (recommended 16)

• min 30GiB of HDD (recommended 30+)

Tools

• Linux distribution running in a VM or bare metal

  • Ubuntu 18.04 or Debian 10 (VM or bare metal)

  • Install Docker​

  • ​Install kubectl on Linux

  • Open terminal and Install k3d on Linux using the below command

  Copy

  wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash

• OSX or Mac

  • ​Docker Desktop local Kubernetes cluster enabled

  • ​Install kubectl on Mac

  • Install k3d on Mac, on terminal use Homebrew (Homebrew is available for MacOS) using the below command

  Copy

  brew install k3d

• Windows 10 or above

  • ​Docker Desktop for windows need to be installed

  • ​Install kubectl on Windows

  • ​Install Chocolatey package manager for windows

  • Install GitBash as an alternative command prompt that allows most of the Linux commands on windows.

  • Open gitbash and Install k3d on Windows using the below command

  Copy

  choco install k3d

Infra (Kubernetes Cluster) Creation

Once the above prerequisites are met, run the following tasks depending upon your OS.‌

• login/ssh into the machine, go to terminal/command prompt and run the following commands as an admin user.

• Create /Kube directory and change permission. To use this directory for persistent data mount. This means data from all container logs will be stored here.

 mkdir /kube
 chmod 777 /kube

 #if you are unable to create the /kube folder in the root
 # you can create it your user directory and provide the absolue path below

• Create a cluster with a single master node and 2 agents (Worker Nodes) and mount the pre-created directory (for data persistence).

k3d cluster create --k3s-server-arg "--no-deploy=traefik" --agents 2 -v "/kube:/kube@agent[0,1]" -v "/kube:/kube@server[0]" --port "80:80@loadbalancer"

• ‌When cluster creation is successful, get the kubeconfig file, that allows you to connect to the cluster at any time.

k3d kubeconfig get k3s-default > k3dconfig
kubectl config use-context k3d-k3s-default --kubeconfig=k3dconfig

• Verify the Cluster creation by running the following commands from your local machine where the kubectl is installed. It gives you the sample output as below

root@ip:/# kubectl cluster-info

OutPut:
Kubernetes control plane is running at https://0.0.0.0:33931
CoreDNS is running at https://0.0.0.0:33931/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://0.0.0.0:33931/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy



root@ip:/# k3d cluster list
NAME          SERVERS   AGENTS   LOADBALANCER
k3s-default   1/1       2/2      true

• You can verify the workers' nodes created by using the following command.

kubectl get nodes

Output:
NAME                       STATUS   ROLES                  AGE     VERSION
k3d-k3s-default-agent-0    Ready    <none>                 3d18h   v1.21.1+k3s1
k3d-k3s-default-agent-1    Ready    <none>                 3d18h   v1.21.1+k3s1
k3d-k3s-default-server-0   Ready    control-plane,master   3d18h   v1.21.1+k3s1


kubectl top nodes

Output:
W0625 07:56:24.588781   12810 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k3d-k3s-default-agent-0    547m         6%     1505Mi          9%        
k3d-k3s-default-agent-1    40m          0%     2175Mi          13%       
k3d-k3s-default-server-0   59m          0%     2286Mi          14%

‌Once the above steps are completed successfully, your Cluster is now up and running ready to proceed with the DIGIT Deployment.‌

2. mGramSeva Setup

‌Now that we have the Infra setup to proceed with the DIGIT Deployment. Following are the tools that need to be installed on the machine before proceeding with the DIGIT Services deployment.‌

What we'll deploy in Quickstart:‌

• mGramSeva core platform services

Prerequisites

1. mGramSeva uses golang (required v1.13.3) automated scripts to deploy the builds on to Kubernetes - Linux or Windows or Mac​

3. ​kubectl is a CLI to connect to the kubernetes cluster from your machine

4. Install CURL for making API calls

5. ​Install Visualstudio IDE Code for better code/configuration editing capabilities

6. The mGramSeva services deployment configurations are in GitRepo which needs to install git and then git clone it to your local.

7. ​Install Postman to run some DIGIT bootstrap scripts

root@ip:/# git clone -b quickstart https://github.com/misdwss/iFix-DevOps

‌After cloning the repo CD into the folder iFix-DevOps and type the "code ." command that will open the visual editor and opens all the files from the repo iFix-DevOps

root@ip:/# cd iFix-DevOps
root@ip:DIGIT-DevOps# code .

Check the sample deployment config file that needs to be configured as per any specific values according to your needs. (For a quick start you can run as it is)

https://github.com/misdwss/iFix-DevOps/blob/quickstart/deploy-as-code/helm/environments/quickstart-config.yaml

Add the following entries in your host file /etc/hosts depending on your OS, instructions can be found here.

127.0.0.1 quickstart.local.digit

‌3. Deployment

‌Once the prerequisite setup is complete, go to the following repo, run the command and follow the instructions.

root@ip:# cd iFix-DevOps/deploy-as-code/egov-deployer

root@ip:# go run mgramseva_setup.go

#Be prepared for the following questions
1. Do you have the Kubernetes Setup?
2. Provide the path of the intented env kubeconfig file
3. Which version of the DIGIT that you want to install - Choose "Quickstart"
4. What mGremSeva Modules that you choose to install
5. All, done, Now do you want to preview the deployment manifests 
6. Are you good to proceed with the DIGIT Installation

All Done.

‌You can now test the DIGIT application status in the command prompt/terminal by using the below command.

curl -Is http://quickstart.local.digit/employee/login |  head -n 1HTTP/2 200

The Amazon Elastic Kubernetes Service (EKS) is one of the AWS services for deploying, managing, and scaling any distributed and containerized workloads, here we can provision the EKS cluster on AWS from ground up and using an automated way (infra-as-code) using terraform and then deploy the DIGIT-iFIX Services config-as-code using Helm.

Pre-reads

• Know about EKS: https://www.youtube.com/watch?v=SsUnPWp5ilc

• Know what is terraform: https://youtu.be/h970ZBgKINg

Pre-requisites

1. ​AWS account with the admin access to provision EKS Service, you can always subscribe to free AWS account to learn the basics and try, but there is a limit to what is offered as free, for this demo you need to have a commercial subscription to the EKS service.

2. Install kubectl on your local machine that helps you interact with the kubernetes cluster

3. Install Helm that helps you package the services along with the configurations, envs, secrets, etc into a kubernetes manifests

4. Install terraform version (0.14.10) for the Infra-as-code (IaC) to provision cloud resources as code and with desired resource graph and also it helps to destroy the cluster at one go.

5. ​Install AWS CLI on your local machine so that you can use aws cli commands to provision and manage the cloud resources on your account.

6. Install AWS IAM Authenticator that helps you authenticate your connection from your local machine so that you should be able to deploy DIGIT services.

7. Use the AWS IAM User credentials provided for the Terraform (Infra-as-code) to connect with your AWS account and provision the cloud resources.

   ​

   • You'll get a Secret Access Key and Access Key ID. Save them safely.

   • Open the terminal and run the following command. The AWS CLI is already installed and the credentials are saved. (Provide the credentials and you can leave the region and output format blank).

   Copy

   aws configure --profile mgramseva-infra-account 
   
   AWS Access Key ID []:<Your access key>
   AWS Secret Access Key []:<Your secret key>
   Default region name []: ap-south-1
   Default output format []: text

   The above will create the following file In your machine as /Users/.aws/credentials

   Copy

   [mgramseva-infra-account] 
   aws_access_key_id=*********** 
   aws_secret_access_key=****************************

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (EKS, Worker Nodes, Postgress DB, EBS Volumes, Load Balancer)

Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every environment, it'll have the same infra.

• EKS Control Plane (Kubernetes Master)

• Work node group (VMs with the estimated number of vCPUs, Memory)

• Node-pool's (mgramseva and ifix)

• EBS Volumes (persistent volumes)

• RDS (Postgresql)

• VPCs (private network)

• Users to access, deploy and read-only

Understand The Resource Graph In Terraform Script

• Ideally, one would write the terraform script from the scratch using this doc.

• Here we have already written the terraform script that provisions the production-grade DIGIT Infra and can be customized with the specified configuration.

• Let's clone the iFix-DevOps GitHub repo where the terraform script to provision EKS cluster is available and below is the structure of the files.

git clone https://github.com/misdwss/iFix-DevOps.git
cd iFix-DevOps/infra-as-code/terraform


└── modules
    ├── db
    │   └── aws
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── kubernetes
    │   └── aws
    │       ├── eks-cluster
    │       │   ├── main.tf
    │       │   ├── outputs.tf
    │       │   └── variables.tf
    │       ├── network
    │       │   ├── main.tf
    │       │   ├── outputs.tf
    │       │   └── variables.tf
    │       └── workers
    │           ├── main.tf
    │           ├── outputs.tf
    │           └── variables.tf
    └── storage
        └── aws
            ├── main.tf
            ├── outputs.tf
            └── variables.tf

Example:

• VPC Resources:

  • VPC

  • Subnets

  • Internet Gateway

  • Route Table

• EKS Cluster Resources:

  • IAM Role to allow EKS service to manage other AWS services

  • EC2 Security Group to allow networking traffic with EKS cluster

  • EKS Cluster

• EKS Worker Nodes Resources:

  • IAM role allowing Kubernetes actions to access other AWS services

  • EC2 Security Group to allow networking traffic

  • Data source to fetch the latest EKS worker AMI

  • AutoScaling Launch Configuration to configure worker instances

  • AutoScaling Group to launch worker instances

• Database

  • Configuration in this directory creates a set of RDS resources including DB instance, DB subnet group, and DB parameter group.

• Storage Module

  • Configuration in this directory creates EBS volume and attaches it together.​

The following main.tf with create s3 bucket to store all the state of the execution to keep track.

iFix-DevOps/Infra-as-code/terraform/sample-eks/remote-state

main.tf​

provider "aws" {
  region = "ap-south-1"
}

#This is a bucket name that you can name as you wish
resource "aws_s3_bucket" "terraform_state" {
  bucket = "try-workshop-yourname" 

  versioning {
    enabled = true
  }

  lifecycle {
    prevent_destroy = true
  }
}

#This is a bucket name that you can name as you wish
resource "aws_dynamodb_table" "terraform_state_lock" {
  name           = "try-workshop-yourname"
  read_capacity  = 1
  write_capacity = 1
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

2. The following main.tf contains the detailed resource definitions that need to be provisioned, please have a look at it.

Dir: iFix-DevOps/Infra-as-code/terraform/sample-eks

main.tf

terraform {
  backend "s3" {
    bucket = "try-workshop-yourname"
    key = "terraform"
    region = "ap-south-1"
  }
}

module "network" {
  source             = "../modules/kubernetes/aws/network"
  vpc_cidr_block     = "${var.vpc_cidr_block}"
  cluster_name       = "${var.cluster_name}"
  availability_zones = "${var.network_availability_zones}"
}


module "iam_user_deployer" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "${var.cluster_name}-kube-deployer"
  force_destroy = true  
  create_iam_user_login_profile = false
  create_iam_access_key         = true

  # User "egovterraform" has uploaded his public key here - https://keybase.io/egovterraform/pgp_keys.asc
  pgp_key = "${var.iam_keybase_user}"
}

module "iam_user_admin" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "${var.cluster_name}-kube-admin"
  force_destroy = true  
  create_iam_user_login_profile = false
  create_iam_access_key         = true

  # User "egovterraform" has uploaded his public key here - https://keybase.io/egovterraform/pgp_keys.asc
  pgp_key = "${var.iam_keybase_user}"
}

module "iam_user_user" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "${var.cluster_name}-kube-user"
  force_destroy = true  
  create_iam_user_login_profile = false
  create_iam_access_key         = true

  # User "test" has uploaded his public key here - https://keybase.io/test/pgp_keys.asc
  pgp_key = "${var.iam_keybase_user}"
}

data "aws_eks_cluster" "cluster" {
  name = "${module.eks.cluster_id}"
}

data "aws_eks_cluster_auth" "cluster" {
  name = "${module.eks.cluster_id}"
}
provider "kubernetes" {
  host                   = "${data.aws_eks_cluster.cluster.endpoint}"
  cluster_ca_certificate = "${base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)}"
  token                  = "${data.aws_eks_cluster_auth.cluster.token}"
  load_config_file       = false
  version                = "~> 1.11"
}

module "eks" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = "${var.cluster_name}"
  cluster_version = "${var.kubernetes_version}"
  subnets         = "${concat(module.network.private_subnets, module.network.public_subnets)}"

  tags = "${
    map(
      "kubernetes.io/cluster/${var.cluster_name}", "owned",
      "KubernetesCluster", "${var.cluster_name}"
    )
  }"

  vpc_id = "${module.network.vpc_id}"

  worker_groups_launch_template = [
    {
      name                    = "spot"
      subnets                 = "${concat(slice(module.network.private_subnets, 0, length(var.availability_zones)), slice(module.network.public_subnets, 0, length(var.availability_zones)))}"
      override_instance_types = "${var.override_instance_types}"
      asg_max_size            = 1
      asg_desired_capacity    = 1
      kubelet_extra_args      = "--node-labels=node.kubernetes.io/lifecycle=spot"
      spot_allocation_strategy= "capacity-optimized"
      spot_instance_pools     = null
    },
  ]
  
  map_users    = [
    {
      userarn  = "${module.iam_user_deployer.iam_user_arn}"
      username = "${module.iam_user_deployer.iam_user_name}"
      groups   = ["system:masters"]
    },
    {
      userarn  = "${module.iam_user_admin.iam_user_arn}"
      username = "${module.iam_user_admin.iam_user_name}"
      groups   = ["global-readonly", "digit-user"]
    },
    {
      userarn  = "${module.iam_user_user.iam_user_arn}"
      username = "${module.iam_user_user.iam_user_name}"
      groups   = ["global-readonly"]
    },    
  ]
}

module "db" {
  source                        = "../modules/db/aws"
  subnet_ids                    = "${module.network.private_subnets}"
  vpc_security_group_ids        = ["${module.network.rds_db_sg_id}"]
  availability_zone             = "${element(var.availability_zones, 0)}"
  instance_class                = "db.t3.medium"
  engine_version                = "11.5"
  storage_type                  = "gp2"
  storage_gb                    = "100"
  backup_retention_days         = "7"
  administrator_login           = "egovdev"
  administrator_login_password  = "${var.db_password}"
  db_name                       = "${var.cluster_name}-db"
  environment                   = "${var.cluster_name}"
}

module "es-master" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "es-master"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}
module "es-data-v1" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "es-data-v1"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "25"
  
}

module "zookeeper" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "zookeeper"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}

module "kafka" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "kafka"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "50"
  
}

module "zookeeper-ifix" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "zookeeper-ifix"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}

module "kafka-ifix" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "kafka-ifix"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "50"
  
}

module "node-group" {  
  for_each = toset(["ifix-dev", "mgramseva"])
  source = "../modules/node-pool/aws"

  cluster_name        = "${var.cluster_name}"
  node_group_name     = "${each.key}-ng"
  kubernetes_version  = "${var.kubernetes_version}"
  security_groups     =  ["${module.network.worker_nodes_sg_id}"]
  subnet              = "${concat(slice(module.network.private_subnets, 0, length(var.node_pool_zone)))}"
  node_group_max_size = 1
  node_group_desired_size = 1
}  

Custom Variables/Configurations

Define your configurations in variables.tf. Provide the environment-specific cloud requirements and use the same terraform template to customize the configurations.

├── sample-eks
│   ├── main.tf 
│   ├── outputs.tf
│   ├── providers.tf
│   ├── remote-state
│   │   └── main.tf
│   └── variables.tf

The values given below must be mentioned in the following files. The blank ones will be prompted for inputs while execution.

​variables.tf ​

## Add Cluster Name
variable "cluster_name" {
  default = "<Desired Cluster name>"  #eg: my-digit-eks
}

## Add vpc_cidr_block
variable "vpc_cidr_block" {
  default = "CIDR" 
}

# If you want prod grade N/W, you can define HA, DRS with multi zone
variable "network_availability_zones" {  
  default = ["ap-south-1b", "ap-south-1a"]
}

# Which zone, it matters
variable "availability_zones" {
  default = ["ap-south-1b"]
}

variable "node_pool_zone" {
 default = "ap-south-1a"  
}

variable "kubernetes_version" {
  default = "1.18"
}

# instance type for your worker nodes like r5a.large is 8 vCPU and 16GB RAM
variable "instance_type" {
  default = "r5a.large"
}

# spot instance configuration
variable "override_instance_types" {
  default = ["r5a.large", "r5ad.large", "r5d.large", "t3a.xlarge"] 
}

# number of machines as per estimate
variable "number_of_worker_nodes" {
  default = "1"
}

##Add ssh key in case you want to ssh to nodes
variable "ssh_key_name" {
  default = "ssh key name"
}

# terraform users ssh public key, you need to one for you, refer below to create yours
variable "iam_keybase_user" {
  default = "keybase:egovterraform"
}

# will be prompted to provide during the execution
variable "db_password" {}

​Use the URL https://keybase.io/ to create your own PGP key. This creates both public and private keys on your machine. Upload the public key into the keybase account that you have just created, give a name to it and ensure that you mention that in your terraform. This allows the encryption of all sensitive information.

• Example - the keybase user (in eGov case is "egovterraform") needs to be created and has to be uploaded the public key here - https://keybase.io/egovterraform/pgp_keys.asc​

• you can use this portal to decrypt your secret key. To decrypt PGP Message, upload the PGP Message, PGP Private Key and the Passphrase.

Run Terraform

Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.

Let's begin to run the Terraform scripts to provision infra required to Deploy DIGIT on AWS.

1. First CD into the following directory and run the following command 1-by-1 and watch the output closely.

cd iFix-DevOps/infra-as-code/terraform/sample-eks/remote-state 
terraform init
terraform plan
terraform apply


cd iFix-DevOps/infra-as-code/terraform/sample-eks 
terraform init
terraform plan
terraform apply

Upon Successful execution following resources get created which can be verified by the command "terraform output"

• s3 bucket: to store terraform state.

• Network: VPC, security groups.

• IAM users auth: using the key base to create admin, deployer and the user. Use this URL https://keybase.io/ to create your own PGP key, this will create both public and private keys in your machine. Upload the public key into the keybase account that you have just created, give a name to it and ensure that you mention that in your terraform. This allows for encrypting all sensitive information.

  • Example: keybase user (in eGov case is "egovterraform") needs to be created and has to be uploaded the public key here - https://keybase.io/egovterraform/pgp_keys.asc​

  • you can use this portal to decrypt your secret key. To decrypt PGP Message, Upload the PGP Message, PGP Private Key and Passphrase.

• EKS cluster: with master(s) & worker node(s).

• Storage(s): for es-master, es-data-v1, es-master-infra, es-data-infra-v1, zookeeper, kafka, kafka-infra.

2. Use this link to get the kubeconfig from EKS to get the kubeconfig file and be able to connect to the cluster from your local machine so that you should be able to deploy DIGIT services to the cluster.

aws sts get-caller-identity

# Run the below command and give the respective region-code and the cluster name
aws eks --region <region-code> update-kubeconfig --name <cluster_name>

3. Verify that you are able to connect to the cluster by running the following command

kubectl config use-context <your cluster name>

kubectl get nodes

NAME                                             STATUS AGE   VERSION               OS-Image           
ip-192-168-xx-1.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-2.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-3.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-4.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2 

Whola! All set and now you can go with Deploy Product..

Architecture Overview

The iFIX platform architecture and the interacting systems are divided into 3 parts:

1. iFIX Platform

2. External Agency Systems

3. Common Reference Data

iFIX Platform

1. Fiscal Event Service: It receives/sends fiscal events from/to the external systems.

2. Fiscal Event Store: After basic checks, the raw fiscal events are stored in the fiscal event store.

3. Fiscal Event Post Processor: Post Processor de-references the master data and flattens the object to store the event in the analytical data store.

4. Fiscal Event Analytical Store: Fiscal Events will be stored in a separate analytical store to run analytics queries.

5. Fiscal Event Analytical Service: All the analytics queries might not be able to run on the raw fiscal events, so a new fiscal event analytical service is developed to process the raw data and prepare it for the analytics queries.

6. Client Registry: All the registered clients of iFIX are maintained in this registry.

External Agency System

1. Fiscal Event Producer: They are the on-ground systems that generate new financial transactional information.

2. Producer Adapter: It helps the producer systems send the fiscal information to the iFIX Platform and link that transaction with some common reference data.

3. Fiscal Event Consumer: There are systems on the other end of the transaction that executes some process based on fiscal transaction information.

4. Consumer Adapter: It helps communicate fiscal events from the iFIX Platform to the consumer systems.

5. Reference Dashboard: Based on the consumed fiscal events, reference dashboards can be built with aggregated amounts by reference data.

Common Reference Data

We can link fiscal events with common reference data to derive more value from the fiscal events data. This allows us to run better analytics queries.

1. Administrative Registry: It will maintain the hierarchy of entities for a department integrated with iFIX.

2. Program Registry: It will maintain the master data for government missions/schemes.

3. Location Registry: It will maintain location reference data. This data is shared across all iFIX clients.

4. Chart of Account Registry: It will maintain the chart of account details.

The Azure Kubernetes Service (AKS) is one of the Azure services used for deploying, managing, and scaling any distributed and containerized workloads. Here we can provision the AKS cluster on Azure from the ground up and using an automated way (infra-as-code) using terraform and then deploy the DIGIT-iFIX Services config-as-code using Helm.

This quickstart assumes a basic understanding of Kubernetes concepts. For more information, see Kubernetes core concepts for Azure Kubernetes Service (AKS).

If you don't have an Azure subscription, create a free account before you begin.

Pre-requisites

• Use the Bash environment in Azure Cloud Shell.

• If you prefer, install the Azure CLI to run CLI reference commands.

  • If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For additional sign-in options, see Sign in with the Azure CLI.

  • When you're prompted, install Azure CLI extensions on first use. For more information about extensions, see Use extensions with the Azure CLI.

  • Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

• This article requires version 2.0.64 or greater of the Azure CLI. If using Azure Cloud Shell, the latest version is already installed.

• The identity you are using to create your cluster has the appropriate minimum permissions. For more details on access and identity for AKS, see Access and identity options for Azure Kubernetes Service (AKS).

• Install kubectl on your local machine that helps you interact with the kubernetes cluster

• Install Helm that helps you package the services along with the configurations, envs, secrets, etc into a kubernetes manifests

• Install terraform version (0.14.10) for the Infra-as-code (IaC) to provision cloud resources as code and with desired resource graph and also it helps to destroy the cluster at one go.

Note: Run the commands as administrator if you plan to run the commands in this quickstart locally instead of in Azure Cloud Shell.

AKS Architecture

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (AKS, Worker Nodes, Postgres DB, Volumes, Load Balancer)

AKS Architecture For iFIX Setup

Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every env, it'll have the same infra.

• AKS Azure (Kubernetes Service Master)

• Work node group (VMs with the estimated number of vCPUs, Memory

• Volumes (persistent volumes)

• PostGres Database

• Virtual Network

• Users to access, deploy and read-only

Understand the Resource Graph In Terraform Script

• Ideally, one would write the terraform script from the scratch using this doc.

• Here we have already written the terraform script that provisions the production-grade DIGIT Infra and can be customized with the specified configuration.

• Let's clone the iFix-DevOps GitHub repo where the terraform script to provision the AKS cluster is available and below is the structure of the files.

git clone --branch release https://github.com/misdwss/iFix-DevOps.git
cd iFix-DevOps/infra-as-code/terraform


└── modules
    ├── db
    │   └── azure
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── kubernetes
    │   └── azure
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── node-pool
    │   └── azure
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    └── storage
        └── azure
            ├── main.tf
            ├── outputs.tf
            └── variables.tf
    ​

The following main.tf contains the detailed resource definitions that need to be provisioned, please have a look at it.

Dir: iFix-DevOps/Infra-as-code/terraform/aks-ifix-dev

main.tf

provider "azurerm" {
  # whilst the `version` attribute is optional, we recommend pinning to a given version of the Provider
  version = "~>2.0"
  features {}
  subscription_id  = "71f67180-c7fb-43dd-988a-e9f9e3135adc"
  tenant_id        = "b36b0fbe-cea1-4178-8664-ba81a1e51765" 
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
}

resource "azurerm_resource_group" "resource_group" {
  name     = "${var.resource_group}"
  location = "${var.location}"
  tags = {
     environment = "${var.environment}"
  }
}

module "kubernetes" {
  source = "../modules/kubernetes/azure"
  environment = "${var.environment}"
  name = "${var.environment}"
  ssh_public_key = "~/.ssh/id_rsa.pub"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${azurerm_resource_group.resource_group.name}"
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
  nodes = "1"
  vm_size = "Standard_DS2_v2"
}

module "node-group" {  
  for_each = toset(["ifix", "mgramseva"])
  source = "../modules/node-pool/azure"
  node_group_name     = "${each.key}ng"
  cluster_id          = "${module.kubernetes.cluster_id}"
  vm_size             = "Standard_D4ds_v4"
  nodes          = 2
}

module "zookeeper" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "zookeeper"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "5"
  
}

module "kafka" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "kafka"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Standard_LRS"
  disk_size_gb = "100"
  
}
module "es-master" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "es-master"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "2"
  
}
module "es-data-v1" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "es-data-v1"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "100"
  
}

module "kafka-ifix" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "kafka-ifix"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Standard_LRS"
  disk_size_gb = "100"
  
}

module "zookeeper-ifix" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "zookeeper-ifix"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "5"
  
}

module "postgres-db" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "2"
  disk_prefix = "postgres-db"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "20"
  
}

Custom Variables/Configurations

You can define your configurations in variables.tf and provide the environment-specific cloud requirements so that using the same terraform template you can customize the configurations.

├── aks-iFix-dev
│   ├── main.tf 
│   ├── outputs.tf
│   ├── providers.tf
│   ├── remote-state
│   │   └── main.tf
│   └── variables.tf

Following are the values that you need to mention in the following files, the blank ones will be prompted for inputs while execution.

​variables.tf ​

## Add Cluster Name
variable "cluster_name" {
  default = "<Desired Cluster name>"  #eg: my-digit-aka
}
## Environment Name
variable "environment" {
    default = "<Desired Environment name>"  #eg: mgramsewa-qa
}
## Resource Group Name
variable "resource_group" {
    default = "<Desired Resource Group name>"  #eg: mgramsewa-qa
}
## Location Name
variable "location" {
    default = "<Desired Location name>"  #eg: southeastAsia
}

​Run Terraform

Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.

Let's begin to run the terraform scripts to provision infra required to Deploy DIGIT on AZ.

1. First CD into the following directory and run the following command 1-by-1 and watch the output closely.

cd DIGIT-DevOps/infra-as-code/terraform/aks-ifix-dev
terraform init
terraform plan
terraform apply

Upon Successful execution following resources gets created which can be verified by the command "terraform output"

• Network: Virtual Network.

• AKS cluster: with nodepool(s), master(s) & worker node(s).

• Storage(s): for es-master, es-data-v1, es-master-infra, es-data-infra-v1, zookeeper, kafka, kafka-infra.

Connect To The Cluster

To manage a Kubernetes cluster, use the Kubernetes command-line client, kubectl. kubectl is already installed if you use Azure Cloud Shell.

1. Install kubectl locally using the az aks install-cli command:

   Copy

   az aks install-cli

2. Configure kubectl to connect to your Kubernetes cluster using the az aks get-credentials command. The following command:

   • Downloads credentials and configures the Kubernetes CLI to use them.

   • Uses ~/.kube/config, the default location for the Kubernetes configuration file. Specify a different location for your Kubernetes configuration file using --file.

   Copy

   az aks get-credentials --resource-group myResourceGroup --name myAKSClusteVerify the connection to your cluster using the  command. This command returns a list of the cluster nodes.

3. Finally, Verify that you are able to connect to the cluster by running the following command

kubectl config use-context <your cluster name>

kubectl get nodes

NAME                       STATUS   ROLES   AGE     VERSION
aks-nodepool1-31718369-0   Ready    agent   6m44s   v1.12.8 

Whola! All set and now you can go with Deploy Product.

Introduction

iFIX/mGramSeva is a microservices-based distributed cloud-native application. Each of these context-specific microservices is dockerized and deployed on Kubernetes infrastructure.

Pre-reads

It is essential to understand some of the key concepts, benefits and best practices of the Kubernetes platform before we understand the deployment of the iFIX/mGramSeva.

• Know the basics of Kubernetes: https://www.youtube.com/watch?v=PH-2FfFD2PU&t=3s​

• Know the basics of kubectl commands

• Know Kubernetes manifests: https://www.youtube.com/watch?v=ohSUtEfDefc​

• Know how to manage environment values and secrets of any service deployed in Kubernetes https://www.youtube.com/watch?v=OW244LxB4oI​

• Know how to port forward to a pod running inside k8s cluster and work locally https://www.youtube.com/watch?v=TT3nd5n5Yus​

• Know sops to secure your keys/creds: https://www.youtube.com/watch?v=DWzJ87KbwxA​

1. Choose the Install Type

Choose the target infra type and follow the instructions to set up a Kubernetes cluster before moving on to the deployment.

2. Deployment

Before we begin the deployment, it is important to understand the deployment architecture that starts from the source code to the production-ready stage. Deploying and managing Kubernetes have emerged as a streamlined way to deploy containers in the cloud infrastructure. When running Kubernetes at scale, managing, operating, and scaling its infrastructure to maximize cluster utilization can be challenging. There are too many parameters the development team needs to manage and configure. This includes selecting the best instance type and size, determining when to scale up or down, and making sure all of the containers are scheduled and running on the best instances — and that is even before starting to think about cost resource optimization.

The simplest way to get started with the deployment process is to manage deployment configuration as code. Each service deployment configuration is defined as Helm charts and deployed into the Kubernetes cluster. We can collocate the deployment-as-code as source code, leveraging all the benefits of source control including change tracking and branching and then packaging it. The source code repo below contains the deployment-as-code details for iFIX.

1. mGramSeva Installation

2. iFIX-adapter Installation

3. iFIX Ref Dashboard Installation

3. Destroy the Setup

Use the command below to clean up the setup cluster. This deletes the entire cluster and other cloud resources that were provisioned for the mGramSeva Infra Setup.

cd iFix-DevOps/infra-as-code/terraform/my-iFix-eksterraformdestroy​

Conclusion

All done, the infra on local, cloud, and deployment of iFIX into the Kubernetes cluster is completed successfully.

1. Java 8 (https://www.oracle.com/java/technologies/java8.html)

2. Apache Kafka (https://kafka.apache.org/)

3. Keycloak (https://www.keycloak.org/)

4. Postgres(https://www.postgresql.org/)

5. Zuul (https://github.com/Netflix/zuul)

6. MongoDB (https://www.mongodb.com/)

7. Apache Druid (https://druid.apache.org/)

8. Metabase (https://www.metabase.com)

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Overview

This page provides the details of the fiscal event-based approach for building out iFIX as an information exchange platform. These details are used to define the technical specifications for iFIX and the functional specifications that are open for validation and inputs internally and from the ecosystem.

Scope

The specifications have been defined from the lens of a sub-national government (the state in the current context) and are limited to interactions from a financial information perspective. The interactions covered on this page include:

• State Finance Department (FD) and Central (National) Government

• State Finance Department (FD) and Line Department(s) at the state level

• State Line Department’s interactions with other line departments

• State Line Department’s interactions with government autonomous bodies including local government and/or non-government agencies

The functional specifications are built using publicly available information and inputs from the Department of Water Supply and Sanitation (DWSS), Punjab, and Finance Department, Punjab. Financial information-related interactions of the Central Government with other Central Line Departments are out of the scope of the current specs.

Fiscal Events Based Approach

The broad objective of iFIX is to enable the flow of reliable and verifiable fiscal information in a timely manner. Recognizing the multiplicity of unique types of information flows in the current PFM system, iFIX aims at simplifying this network of information flow. The key driver of this simplification will be the use of standardised formats for fiscal information exchange. To arrive at a crystallised set of formats and protocols covering all fiscal information exchanges following steps have been followed

• Step 1 - Define what is a fiscal event (and its types) or what is the scope of relevant fiscal information from an iFIX perspective (refer section)

• Step 2 - Document the current public finance management processes at a generic level, i.e. defined using actors, verbs, inputs and outputs to ensure they are representative of all minor variants of the process at the level of various sub-national governments in India. (refer to tables in , columns 1-6)

• Step 3 - Apply the definition of the fiscal event to the current processes to collapse or abstract the fiscal event essence of the whole process to a set of fiscal events. (refer to tables in , columns 7-9).

• Step 4 - Extract the current data attributes used for fiscal information exchange to define the format for fiscal information exchange. (refer to tables in , columns 10)

Definition of Fiscal Event

Events that trigger the generation of relevant fiscal information, at any stage in the budget cycle, are termed fiscal events. To be classified as a fiscal event, the event will need to meet one of the following criteria:

Transaction-Based Fiscal Events: Such fiscal events are triggered when there is fiscal information being generated due to an actual change of hands of a financial asset or in simple words due to a financial transaction.

Non-Transactional Fiscal Events: While there are numerous non-transactional events that occur throughout the budget cycle, these are termed Fiscal events only if they meet at least one of the following criteria:

• Minimum Degree of Finality: A non-transactional event will be considered a fiscal event only when the action resulting from or the document produced from the event has definitive implications for the budgetary cycle. Examples:

  1. A draft of the budget that has been prepared at the state line department level (DDO) and sent to the next competent authority (BCO) for approval will trigger a fiscal event. The reason for this is that it is assumed that the line department at its level has done the calculations for arriving at a final figure which then needs to be approved by the next competent authority.

  2. With respect to any payments to be made out of state treasury, any verbal/ email-based intra-Finance department go-ahead (between State Treasury and Cash Planning Department) to make the payment will not be considered a Fiscal Event. Only the generation of Payment Advice to the concerned bank will trigger a fiscal event.

• Change in ability to claim/ use/ dispose of a financial asset: A non-transactional event will be considered a fiscal event if it results in (de-)authorizing a certain individual or entity to claim/ utilize/ dispose of a financial asset. Examples:

  1. Allocation of the budget by the department to respective DDOs authorises the DDOs to utilize the funds as planned in the budget and will trigger a fiscal event.

Types of Fiscal Events

The fiscal information generated during each fiscal event needs to be recorded in a specific format for the purpose of exchange. Based on the current budgetary practices and guidelines followed at the sub-national and national levels in India, the fiscal events triggered during the course of the budget cycle can be classified into four major types:

1. Revenue Receipts: Revenue receipts comprise receipts that do not result in the creation of a liability on the government. The total revenue receipts include State’s Own Tax and Non-Tax revenues and Grants-in-Aid and Share in Central Taxes from the Government of India. The non-tax revenues consist mainly of interest and dividends on investments made by the Government, fees and other receipts for services rendered by the Government.

2. Capital Receipts: The capital receipts are loans raised by the Government from the public (these are termed as market loans), borrowings by the Government through the sale of Treasury Bills, the loans received from Central Government and bodies, disinvestment receipts and recoveries of any loans and advances given.

3. Revenue Expenditure: Revenue expenditure is for the normal running of different Government Departments and for the rendering of various services, making interest payments on debt, meeting subsidies, grants in aid, etc. Broadly, the expenditure which does not result in the creation of assets for the Government of India is treated as revenue expenditure. All grants given by the State are also treated as revenue expenditure even though some of the grants may be used for the creation of capital assets.

4. Capital Expenditure: Capital payments consist of capital expenditure on the acquisition of assets like land, buildings, machinery, and equipment, as also investments in shares, etc., and loans and advances made by the State Government to boards, corporations and other institutions.

Fiscal Event - Sub-Types

Further within each major type of fiscal event, there are varied types of events. To enable information exchange using an easily understandable and standardised format, subtypes of fiscal events are identified based on the similarity in nature of fiscal information generated due to these events.

Application of Fiscal Event Framework to As-Is-Processes

Budget Cycle - Overview

Budget Cycle comprises the following stages:

1. Budget Planning

2. Budget Preparation

3. Budget Approval

4. Budget Allocation

5. Budget Execution

6. Budget Accounting

7. Budget Auditing

Additionally, Budget Planning is an activity that sits outside of the Budget Cycle and takes place throughout the year based on need. Examples:

• A scheme/project announced by a state government official can happen at any point of the year, the planning for which begins right after the announcement. Thereafter, all the required approvals happen and estimates are prepared accordingly which then feed into the budget preparation phase of the budget cycle

• Planning for already approved projects and planning to get approval

Inferring Fiscal Events from As-is-Process Flows

Budget Planning

Process for Planning for the New Projects/ Schemes (for Revenue and Capital Expenditure)

Budget Preparation

Budget Preparation for Revenue Receipt

Budget Preparation for Revenue Expenditure

Budget Preparation for Capital Expenditure - Capital Outlay

Budget Approval

Budget Approval from Legislature (Same process for Revenue and Capital nature Receipts and Expenditures)

Budget Allocation

[5]Communication and Distribution of funds (Same process for Revenue and Capital nature Expenditure)

Budget Execution

Request for Release of Funds from State Treasury - Scheme-related Capital Expenditure

Release of funds to DDO

Work awarded to vendors

Work bill payment to vendors

₹Request for Release of Funds from State Treasury - Revenue Expenditure

Collection of Revenue Receipt into the State Treasury

Request for Supplementary Grants (Same process for Revenue and Capital Expenditure)

Surrender of Excess / Savings (Same process for Revenue and Capital expenditure)

Budget Accounting

Process for monthly budget accounting/reporting for Receipts (Same Process for Revenue and Capital Receipts)

Process for monthly budget accounting/reporting -Expenditure (Same Process for Revenue and Capital Expenditure)

Budget Auditing

Process for budget auditing (have to explore if there are variations for revenue and expenditure)

Definition Of Format Of Fiscal Event Sub-Types

Each fiscal event extracted above is defined in terms of

• Header

• Body

• Array of Fiscal Line Items

• Attachment

• Signature

Explore our iFIX (PFM) roadmap detailed view below.

The platform consists of the following core services.

Master Data Service

The master data service manages the following master data:

• Department

• Expenditure

• Project

Business Services

All content on this page by is licensed under a .

The 2 Stages

Essentially, there are 2 stages that should allow you to use the full potential of DeploymentConfig and pipeline-as-code.

Stage 1: Clone the DevOps , choose your iFix product branch as mGramSeva‌

Prepare an <> master config file, you can name this file as you wish which will have the following configurations, this env file needs to be in line with your cluster name.‌

• each service global, local env variables

• credentials, secrets (You need to encrypt using and create a <env>-secret.yaml separately)

• Number of replicas/scale of individual services (Depending on whether dev or prod)

• mdms, config repos (Master Data, ULB, Tenant details, Users, etc)

• sms g/w, email g/w, payment g/w

• GMap key (In case you are using Google Map services in your PGR, PT, TL, etc)

• S3 Bucket for Filestore

• URL/DNS on which the DIGIT will be exposed

• SSL Certificate for the above URL

• End-points configs (Internal/external)

Stage 2: Run the mGramSeva_setup deployment script and simply answer the questions that it asks

Deploy mGramSeva

Post infra setup (Kubernetes Cluster), there starts the deployment process.

Pipeline as code is a practice of defining deployment pipelines through source code, such as Git. Pipeline as code is part of a larger “as code” movement that includes infrastructure as code. Teams can configure builds, tests, and deployment in code that is trackable and stored in a centralized source repository. Teams can use a declarative approach or a vendor-specific programming language, such as Jenkins and Groovy, but the premise remains the same.

A pipeline as code file specifies the stages, jobs, and actions for a pipeline to perform. Because the file is versioned, changes in pipeline code can be tested in branches with the corresponding application release.

The pipeline as code model of creating continuous integration pipelines is an industry best practice, but deployment pipelines used to be created very differently.

The deployment process has got 2 stages and 2 modes. We can see the modes first and then the stages.‌

The 2 Modes of Deployment

Essentially, mGramSeva deployment means that we need to generate Kubernetes manifests for each individual service. We use the tool called the helm, which is an easy, effective and customizable packaging and deployment solution. So depending on where and which env you initiate the deployment there are 2 modes that you can deploy.‌

1. From - whatever we are trying in this sample exercise so far.

2. Advanced: like Jenkins - Depending on how you want to set up your CI/CD and the expertise the steps will vary, however here you can find how we eGov has set up an exemplar CI/CD on Jenkins and the pipelines are created automatically without any manual intervention.

All content on this page by is licensed under a .

ADMIN ONLY

Only the administrator can generate the API key and secret. The steps for doing this is as follows.

Keycloak console will be available at https://<host-name>/auth. The admin will log in using the username and password secret.

Step 1: Import Realm (If not already done)

1. Open the Keycloak console

2. Near the top-left corner in the realm drop down menu, select Add Realm.

3. Select the file.

4. After the realm gets created, select the ifix realm from the drop-down near the top-left corner.

Creating Clients

1. Remember to select the ifix realm from the Keycloak console before proceeding.

2. From the Clients section of Keycloak Admin Console, create a client.

3. Provide a unique username for the client.

4. Go to the client's settings

5. Change Access Type to confidential

6. Turn on Service Account Enabled

7. In the Valid Redirect URIs field provide the root URL of the iFIX Instance. (Not important for our purposes but need to set it because it is mandatory)

8. And Save these changes

9. In the Service Account Roles tab, assign the role "fiscal-event-producer"

10. In the Mappers tab, create a new mapper to associate the client with a tenantId

    1. Select Mapper Type to be "Hardcoded claim"

    2. In Token Claim Name, write "tenantId"

    3. In Claim value, write the under which the client is being created. (For example, "pb")

    4. Set Name same as Token Claim Name i.e. "tenantId"

    5. Select Claim Json Type to be "String"

Now you can get the credentials from the Credentials tab and configure them in the client's system.

The 2 Stages

Stage 1 : Jenkins Setup

Post infra setup (Kubernetes Cluster), We start with deploying the Jenkins and kaniko-cache-warmer.

Pre-requisites

• Sub Domain to expose CI/CD URL

• GitHub

• With

• (username and password)

• SSL Certificate for the sub-domain

Prepare an <> master config file and <>, you can name this file as you wish which will have the following configurations.

• credentials, secrets (You need to encrypt using and create a ci-secret.yaml separately)

• Check and Update details (like github Oauth app clientId and clientSecret, GitHub user details gitReadSshPrivateKey and gitReadAccessToken etc..)

• To create Jenkins namespace mark this true

• Add your env's kubconfigs under kubConfigs like

• KubeConfig env's name and deploymentJobs name from ci.yaml should be the same

• Update the and repo name with your forked repo name and provide read-only access to github user to those repo's.

• SSL Certificate for the sub-domain

You have launched the Jenkins. You can access the same through your sub-domain which you configured in ci.yaml.

The Jenkins CI pipeline is configured and managed 'as code'.

​Example URL - https://<Jenkins_domain>​

Stage 2: Continuous Integration (CI)

Since there are many services and the development code is part of various git repos, you need to understand the concept of cicd-as-service which is open-sourced. This page also guides you through the process of creating a CI/CD pipeline.

As a developer - To integrate any new service/app to the CI/CD below is the starting point:

Once the desired service is ready for the integration: decide the service name, type of service, whether DB migration is required or not. While you commit the source code of the service to the git repository, the following file should be added with the relevant details which are mentioned below:

Build-config.yml –It is present under the build directory in each repository

This file contains the below details which are used for creating the automated Jenkins pipeline job for your newly created service.

While integrating a new service/app, the above content needs to be added in the build-config.yml file of that app repository. For example: If we are onboarding a new service called egov-test, then the build-config.yml should be added as mentioned below.

If a job requires multiple images to be created (DB Migration) then it should be added as below,

Note - If a new repository is created then the build-config.yml should be created under the build folder and then the config values are added to it.

The git repository URL is then added to the Job Builder parameters

When the Jenkins Job => job builder is executed the CI Pipeline gets created automatically based on the above details in build-config.yml. Eg: egov-test job will be created under the core-services folder in Jenkins because the “build-config was edited under core-services” And it should be the “master” branch only. Once the pipeline job is created, it can be executed for any feature branch with build parameters (Specifying which branch to be built – master or any feature branch).

As a result of the pipeline execution, the respective app/service docker image will be built and pushed to the Docker repository.

If git repository URL is available build the Job-Builder Job

If the git repository URL is not available ask the Devops team to add it.

Continuous Deployment (CD)‌

The services deployed and managed on a Kubernetes cluster in cloud platforms like AWS, Azure, GCP, OpenStack, etc. Here, we use helm charts to manage and generate the Kubernetes manifest files and use them for further deployment to the respective Kubernetes cluster. Each service is created as charts which will have the below-mentioned files in them.

To deploy a new service, we need to create the helm chart for it. The chart should be created under the charts/helm directory in iFix-DevOps repository.

We have an automatic helm chart generator utility that needs to be installed on the local machine, the utility prompts for user inputs about the newly developed service (app specifications) for creating the helm chart. The requested chart with the configuration values (created based on the inputs provided) will be created for the user.

‌ Name of the service? test-service Application Type? NA Kubernetes health checks to be enabled? Yes Flyway DB migration container necessary? No, Expose service to the internet? Yes, Route through API gateway [zuul] No Context path? hello‌

The generated chart will have the following files.

This chart can also be modified further based on user requirements.

The Deployment of manifests to the Kubernetes cluster is made very simple and easy. We have Jenkins Jobs for each state and are environment-specific. We need to provide the image name or the service name in the respective Jenkins deployment job.

Enter a caption for this image (optional)

Enter a caption for this image (optional)

‌The deployment Jenkins job internally performs the following operations,‌

• Reads the image name or the service name given and finds the chart that is specific to it.

• Generates the Kubernetes manifests files from the chart using the helm template engine.

• Execute the deployment manifest with the specified docker image(s) to the Kubernetes cluster.