The 2 Stages

Stage 1 : Jenkins Setup

Post infra setup (Kubernetes Cluster), We start with deploying the Jenkins and kaniko-cache-warmer.

Pre-requisites

• Sub Domain to expose CI/CD URL

• GitHub Oauth App

• GitHub User ssh key

• With GitHub user generate a personal read only access token

• Docker hub account details (username and password)

• SSL Certificate for the sub-domain

Prepare an <ci.yaml> master config file and <ci-secrets.yaml>, you can name this file as you wish which will have the following configurations.

• credentials, secrets (You need to encrypt using sops and create a ci-secret.yaml separately)

• Check and Update ci-secrets.yaml details (like github Oauth app clientId and clientSecret, GitHub user details gitReadSshPrivateKey and gitReadAccessToken etc..)

• To create Jenkins namespace mark this flag true

• Add your env's kubconfigs under kubConfigs like https://github.com/misdwss/iFix-DevOps/blob/mgramseva/deploy-as-code/helm/environments/ci-secrets.yaml#L19

• KubeConfig env's name and deploymentJobs name from ci.yaml should be the same

• Update the CIOps and DIGIT-DevOps repo name with your forked repo name and provide read-only access to github user to those repo's.

• SSL Certificate for the sub-domain

cd iFix-DevOps/deploy-as-code/egov-deployer

kubectl config use-context <your cluster name>
go run main.go deploy -c -e ci 'jenkins,kaniko-cache-warmer,nginx-ingress'

You have launched the Jenkins. You can access the same through your sub-domain which you configured in ci.yaml.

The Jenkins CI pipeline is configured and managed 'as code'.

​Example URL - https://<Jenkins_domain>​

Stage 2: Continuous Integration (CI)

Since there are many services and the development code is part of various git repos, you need to understand the concept of cicd-as-service which is open-sourced. This page also guides you through the process of creating a CI/CD pipeline.

As a developer - To integrate any new service/app to the CI/CD below is the starting point:

Once the desired service is ready for the integration: decide the service name, type of service, whether DB migration is required or not. While you commit the source code of the service to the git repository, the following file should be added with the relevant details which are mentioned below:

Build-config.yml –It is present under the build directory in each repository

https://github.com/misdwss/punjab-mgramseva/blob/master/build/build-config.yml

This file contains the below details which are used for creating the automated Jenkins pipeline job for your newly created service.

config:
  - name: < Name of the job, foo/bar would create job named bar inside folder foo>
    build:
    - work-dir: < Working directory of the app to be built >
      dockerfile: < Path to the dockerfile, optional, assumes dockerfile in working directory if not provided >
      image-name: < Docker image name  >

While integrating a new service/app, the above content needs to be added in the build-config.yml file of that app repository. For example: If we are onboarding a new service called egov-test, then the build-config.yml should be added as mentioned below.

config:   
  - name: core-services/egov-test     
    build:     - work-dir: egov-test       
    dockerfile: build/maven/Dockerfile       
    image-name: egov-test

If a job requires multiple images to be created (DB Migration) then it should be added as below,

config:   
  - name: core-services/egov-test     
    build:     
    - work-dir: egov-test       
      dockerfile: build/maven/Dockerfile       
      image-name: egov-test     
    - work-dir: egov-test/src/main/resources/db       
      dockerfile: build/maven/Dockerfile       
      image-name: egov-test-db

Note - If a new repository is created then the build-config.yml should be created under the build folder and then the config values are added to it.

The git repository URL is then added to the Job Builder parameters

When the Jenkins Job => job builder is executed the CI Pipeline gets created automatically based on the above details in build-config.yml. Eg: egov-test job will be created under the core-services folder in Jenkins because the “build-config was edited under core-services” And it should be the “master” branch only. Once the pipeline job is created, it can be executed for any feature branch with build parameters (Specifying which branch to be built – master or any feature branch).

As a result of the pipeline execution, the respective app/service docker image will be built and pushed to the Docker repository.

Job Builder – Job Builder is a Generic Jenkins job that creates the Jenkins pipeline automatically which are then used to build the application, create the docker image of it and push the image to the docker repository. The Job Builder job requires the git repository URL as a parameter. It clones the respective git repository and reads the build/build-config.yml file for each git repository and uses it to create the service build job.

‌Check git repository URL is available in ci.yaml​‌‌

If git repository URL is available build the Job-Builder Job

If the git repository URL is not available ask the Devops team to add it.

Continuous Deployment (CD)‌

The services deployed and managed on a Kubernetes cluster in cloud platforms like AWS, Azure, GCP, OpenStack, etc. Here, we use helm charts to manage and generate the Kubernetes manifest files and use them for further deployment to the respective Kubernetes cluster. Each service is created as charts which will have the below-mentioned files in them.

billing-service/   
# Directory – name of the service/appChart.yaml         
# A YAML file containing information about the chartLICENSE            
# OPTIONAL: A plain text file containing the license for the chartREADME.md          # OPTIONAL: A human-readable README filevalues.yaml        
# The default configuration values for this charttemplates/         
# A directory of templates that, when combined with values, will generate valid Kubernetes manifest files.

To deploy a new service, we need to create the helm chart for it. The chart should be created under the charts/helm directory in iFix-DevOps repository.

Github repository     https://github.com/misdwss/iFix-DevOps

We have an automatic helm chart generator utility that needs to be installed on the local machine, the utility prompts for user inputs about the newly developed service (app specifications) for creating the helm chart. The requested chart with the configuration values (created based on the inputs provided) will be created for the user.

‌ Name of the service? test-service Application Type? NA Kubernetes health checks to be enabled? Yes Flyway DB migration container necessary? No, Expose service to the internet? Yes, Route through API gateway [zuul] No Context path? hello‌

The generated chart will have the following files.

create Chart.yaml
create values.yaml
create templates/deployment.yaml
create templates/service.yaml
create templates/ingress.yaml

This chart can also be modified further based on user requirements.

The Deployment of manifests to the Kubernetes cluster is made very simple and easy. We have Jenkins Jobs for each state and are environment-specific. We need to provide the image name or the service name in the respective Jenkins deployment job.

Enter a caption for this image (optional)

Enter a caption for this image (optional)

‌The deployment Jenkins job internally performs the following operations,‌

• Reads the image name or the service name given and finds the chart that is specific to it.

• Generates the Kubernetes manifests files from the chart using the helm template engine.

• Execute the deployment manifest with the specified docker image(s) to the Kubernetes cluster.

Overview

Fiscal Event Aggregator is a java standalone application, which runs as Cron Job to aggregate the fiscal event data from the Druid data store to Postgres DB.

Version History

Current Version : 2.0.0

Prerequisites

Before you proceed with the configuration, make sure the following pre-requisites are met

1. Java 8

2. Druid DB & Postgres DB should be up and running

Features

Fiscal-Event-Aggregator computes the aggregate of data over a selected time period. Aggregator will apply the time range filter according to the following approach :

Follow the steps below to aggregate the final fiscal event data as per the fiscal time periods.

1. Group the sum of the amount based on department entity ancestry[6] id (attributes.departmentEntity.ancestry[6].id) that is GP (Gram Panchayat), COA (chart of account) id, and event type.

2. Difference of the sum of amount of "DEMAND" and "RECEIPT" event type with respect to distinct department entity ancestry[6] id (attributes.departmentEntity.ancestry[6].id) that is GP (Gram Panchayat).

3. Difference of the sum of amount of "BILL" and "PAYMENT" event type with respect to distinct department entity ancestry[6] id (attributes.departmentEntity.ancestry[6].id) that is GP(Gram Panchayat).

Upsert the final aggregated fiscal event data into the Postgres DB.

Environment

Note: Below environment variables need to be configured with respect to the environment

Interaction Diagram

Configuration

Update the configurations in the dev.yaml, qa.yml, prod.yaml file.

Filters on the Dashboard

Date/Time Filter

The date & time filter on the dashboard defaults to the current financial year since most of the calculations and visualizations makes sense when viewed from a fiscal year perspective rather than an individual week/month view. However, users can change to respective dates and most charts will be filtered to the selected time range.

Department Hierarchy Filter

Currently, there are 6 levels of hierarchy as per administration set up by the Department of Water supply and Sanitation Punjab. These are State, Zone, Circle, Division, Sub Division, Section, Gram Panchayat.

All these filters are independent and work on the dashboard irrespective of whether other filters in the hierarchy are selected or not.

Visualization

Surplus/Deficit

This number shows whether the selected Administrative entity is financially surplus or not. It also compares with the previous year using the “trend” visualization in Metabase.

Pending Collections

For selected Administrative boundary what is the pending collections through water charges is represented in this card. If the pending amount is null. Then this card should display zero.

Outstanding Electricity Bills

Since electricity bills create a major component of the expenditure for all projects, it is important to show how much electricity bills are pending at each administrative entity. The total amount of pending bills filtered by electricity under COA gives us pending electricity bills

GPWSCs at Risk

Risky GPWSCs are divided into 3 types

High Risk: Demand is less than Bill. Medium Risk: Demand is more than bill but pending collections is less than pending bills.

Low Risk: Demand is more than bill and also pending collections is more than pending bills.

It is important to identify the risky GPWSCs and keep the officials of DWSS informed, for them to take the right actions before it’s too late.

Collections & Expenditure Time Series Graphs

These charts represent the Demand, Net Collections, Bills and Net Payments across time(month-on-month) so that officials get a fair idea of how the amount that is getting collected is being spent.

Any abnormalities in the graphs (Low collections or excessive spending) is something that needs to be paid attention to.

Expenditure by Chart of Accounts

Expenditure is currently divided into 4 broad categories - Electricity Bills, Salaries, Operations and Maintenance and Miscellaneous. How these 4 categories accumulate to total expenditure for the selected entity over time is presented on the chart. Usually, 75% of the expenditure should be of type electricity

Department Hierarchy Table

Here, we represent Total Demand, Receipt, Bills and Payments by all levels of the hierarchy so that officials at any point, instead of just viewing the charts, trends from the visualizations can also see the tables and compare best performing entities.

The 2 Stages:

Essentially, there are 2 stages that should allow you to use the full potential of DeploymentConfig and pipeline-as-code.

Stage 1: Clone the DevOps repo, choose your iFix product branch as iFix-adapter.‌

Prepare an <env.yaml> master config file, you can name this file as you wish which will have the following configurations, this env file need to be in line with your cluster name.‌

• each service global, local env variables

• credentials, secrets (You need to encrypt using sops and create a <env>-secret.yaml separately)

• Number of replicas/scale of individual services (Depending on whether dev or prod)

• mdms, config repos (Master Data, ULB, Tenant details, Users, etc)

• sms g/w, email g/w, payment g/w

• GMap key (In case you are using Google Map services in your PGR, PT, TL, etc)

• S3 Bucket for Filestore

• URL/DNS on which the DIGIT will be exposed

• SSL Certificate for the above URL

• End-points configs (Internal/external)

Stage 2: Run the iFix_Dashboard_setup deployment script and simply answer the questions that it asks.

cd iFix-DevOps/deploy-as-code/egov-deployer

go run ifix_dashboard_setup.go

#Be prepared for the following questions
1. Do you have the Kubernetes Setup?
2. Provide the path of the intented env kubeconfig file
3. Which version of the DIGIT that you want to install
4. What mGramSeva Modules that you choose to install (Choose PGR)
5. All, done, Now do you want to preview the deployment manifests 
6. Are you good to proceed with the DIGIT Installation

All Done.Essentially, DIGIT deployment means that we need to generate Kubernetes manifests for each individual service. We use the tool called helm, which is an easy, effective and customizable packaging and deployment solution. So depending on where and which env you initiate the deployment there are 2 modes that you can deploy.

iFIX Dashboard is built on Metabase and can be easily configured to develop various dashboards.

The below dashboard provides information about the fiscal position of various projects. The dashboard can be filtered for various date ranges and departmental hierarchies.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

iFix dashboard is developed using the opensource tools that included the complete OLAP system that streams the data from various sources, transform and process data and visualize the data using dashboards

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Fiscal Event services flatten each fiscal event line item and post them into Druid via the Kafka Druid connector. The raw events are stored in the fiscal-events dataset in Druid. Metabase is used for visualisations.

The flattened fiscal event consists of the following attributes

Reference Dashboard

This section contains details and information about the iFIX Reference Dashboard. Click on the links below to learn more.

• iFIX Fiscal Event Aggregator

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

Deployment of iFix Dashboard

Post infra setup (Kubernetes Cluster), the deployment has got 2 stages and 2 modes. We can see the stages first and then the modes.‌

The 2 Modes of Deployment

Essentially, iFix dashboard deployment means that we need to generate Kubernetes manifests for each individual service of the required OLAP components like a druid, metabase. We use the tool called helm, which is an easy, effective and customizable packaging and deployment solution. So depending on where and which env you initiate the deployment there are 2 modes that you can deploy.‌

1. From Local machine - whatever we are trying in this sample exercise so far.

2. Advanced: Setup CI/CD System like Jenkins - Depending on how you want to set up your CI/CD and the expertise the steps will vary, however here you can find how we eGov has set up an exemplar CI/CD on Jenkins and the pipelines are created automatically without any manual intervention.

You can choose the infra type and the env to either single fat server or distributed setup on Docker compose or Kubernetes.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.