The Azure Kubernetes Service (AKS) is one of the Azure services used for deploying, managing, and scaling any distributed and containerized workloads. Here we can provision the AKS cluster on Azure from the ground up and using an automated way (infra-as-code) using terraform and then deploy the DIGIT-iFIX Services config-as-code using Helm.

This quickstart assumes a basic understanding of Kubernetes concepts. For more information, see Kubernetes core concepts for Azure Kubernetes Service (AKS).

If you don't have an Azure subscription, create a free account before you begin.

Pre-requisites

• Use the Bash environment in Azure Cloud Shell.

• If you prefer, install the Azure CLI to run CLI reference commands.

  • If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For additional sign-in options, see Sign in with the Azure CLI.

  • When you're prompted, install Azure CLI extensions on first use. For more information about extensions, see Use extensions with the Azure CLI.

  • Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

• This article requires version 2.0.64 or greater of the Azure CLI. If using Azure Cloud Shell, the latest version is already installed.

• The identity you are using to create your cluster has the appropriate minimum permissions. For more details on access and identity for AKS, see Access and identity options for Azure Kubernetes Service (AKS).

• Install kubectl on your local machine that helps you interact with the kubernetes cluster

• Install Helm that helps you package the services along with the configurations, envs, secrets, etc into a kubernetes manifests

• Install terraform version (0.14.10) for the Infra-as-code (IaC) to provision cloud resources as code and with desired resource graph and also it helps to destroy the cluster at one go.

Note: Run the commands as administrator if you plan to run the commands in this quickstart locally instead of in Azure Cloud Shell.

AKS Architecture

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (AKS, Worker Nodes, Postgres DB, Volumes, Load Balancer)

AKS Architecture For iFIX Setup

Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every env, it'll have the same infra.

• AKS Azure (Kubernetes Service Master)

• Work node group (VMs with the estimated number of vCPUs, Memory

• Volumes (persistent volumes)

• PostGres Database

• Virtual Network

• Users to access, deploy and read-only

Understand the Resource Graph In Terraform Script

• Ideally, one would write the terraform script from the scratch using this doc.

• Here we have already written the terraform script that provisions the production-grade DIGIT Infra and can be customized with the specified configuration.

• Let's clone the iFix-DevOps GitHub repo where the terraform script to provision the AKS cluster is available and below is the structure of the files.

git clone --branch release https://github.com/misdwss/iFix-DevOps.git
cd iFix-DevOps/infra-as-code/terraform


└── modules
    ├── db
    │   └── azure
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── kubernetes
    │   └── azure
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── node-pool
    │   └── azure
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    └── storage
        └── azure
            ├── main.tf
            ├── outputs.tf
            └── variables.tf
    ​

The following main.tf contains the detailed resource definitions that need to be provisioned, please have a look at it.

Dir: iFix-DevOps/Infra-as-code/terraform/aks-ifix-dev

main.tf

provider "azurerm" {
  # whilst the `version` attribute is optional, we recommend pinning to a given version of the Provider
  version = "~>2.0"
  features {}
  subscription_id  = "71f67180-c7fb-43dd-988a-e9f9e3135adc"
  tenant_id        = "b36b0fbe-cea1-4178-8664-ba81a1e51765" 
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
}

resource "azurerm_resource_group" "resource_group" {
  name     = "${var.resource_group}"
  location = "${var.location}"
  tags = {
     environment = "${var.environment}"
  }
}

module "kubernetes" {
  source = "../modules/kubernetes/azure"
  environment = "${var.environment}"
  name = "${var.environment}"
  ssh_public_key = "~/.ssh/id_rsa.pub"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${azurerm_resource_group.resource_group.name}"
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
  nodes = "1"
  vm_size = "Standard_DS2_v2"
}

module "node-group" {  
  for_each = toset(["ifix", "mgramseva"])
  source = "../modules/node-pool/azure"
  node_group_name     = "${each.key}ng"
  cluster_id          = "${module.kubernetes.cluster_id}"
  vm_size             = "Standard_D4ds_v4"
  nodes          = 2
}

module "zookeeper" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "zookeeper"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "5"
  
}

module "kafka" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "kafka"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Standard_LRS"
  disk_size_gb = "100"
  
}
module "es-master" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "es-master"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "2"
  
}
module "es-data-v1" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "es-data-v1"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "100"
  
}

module "kafka-ifix" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "kafka-ifix"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Standard_LRS"
  disk_size_gb = "100"
  
}

module "zookeeper-ifix" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "3"
  disk_prefix = "zookeeper-ifix"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "5"
  
}

module "postgres-db" {
  source = "../modules/storage/azure"
  environment = "${var.environment}"
  itemCount = "2"
  disk_prefix = "postgres-db"
  location = "${azurerm_resource_group.resource_group.location}"
  resource_group = "${module.kubernetes.node_resource_group}"
  storage_sku = "Premium_LRS"
  disk_size_gb = "20"
  
}

Custom Variables/Configurations

You can define your configurations in variables.tf and provide the environment-specific cloud requirements so that using the same terraform template you can customize the configurations.

├── aks-iFix-dev
│   ├── main.tf 
│   ├── outputs.tf
│   ├── providers.tf
│   ├── remote-state
│   │   └── main.tf
│   └── variables.tf

Following are the values that you need to mention in the following files, the blank ones will be prompted for inputs while execution.

​variables.tf ​

## Add Cluster Name
variable "cluster_name" {
  default = "<Desired Cluster name>"  #eg: my-digit-aka
}
## Environment Name
variable "environment" {
    default = "<Desired Environment name>"  #eg: mgramsewa-qa
}
## Resource Group Name
variable "resource_group" {
    default = "<Desired Resource Group name>"  #eg: mgramsewa-qa
}
## Location Name
variable "location" {
    default = "<Desired Location name>"  #eg: southeastAsia
}

​Run Terraform

Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.

Let's begin to run the terraform scripts to provision infra required to Deploy DIGIT on AZ.

1. First CD into the following directory and run the following command 1-by-1 and watch the output closely.

cd DIGIT-DevOps/infra-as-code/terraform/aks-ifix-dev
terraform init
terraform plan
terraform apply

Upon Successful execution following resources gets created which can be verified by the command "terraform output"

• Network: Virtual Network.

• AKS cluster: with nodepool(s), master(s) & worker node(s).

• Storage(s): for es-master, es-data-v1, es-master-infra, es-data-infra-v1, zookeeper, kafka, kafka-infra.

Connect To The Cluster

To manage a Kubernetes cluster, use the Kubernetes command-line client, kubectl. kubectl is already installed if you use Azure Cloud Shell.

1. Install kubectl locally using the az aks install-cli command:

   Copy

   az aks install-cli

2. Configure kubectl to connect to your Kubernetes cluster using the az aks get-credentials command. The following command:

   • Downloads credentials and configures the Kubernetes CLI to use them.

   • Uses ~/.kube/config, the default location for the Kubernetes configuration file. Specify a different location for your Kubernetes configuration file using --file.

   Copy

   az aks get-credentials --resource-group myResourceGroup --name myAKSClusteVerify the connection to your cluster using the  command. This command returns a list of the cluster nodes.

3. Finally, Verify that you are able to connect to the cluster by running the following command

kubectl config use-context <your cluster name>

kubectl get nodes

NAME                       STATUS   ROLES   AGE     VERSION
aks-nodepool1-31718369-0   Ready    agent   6m44s   v1.12.8 

Whola! All set and now you can go with Deploy Product.

Quickstart Installation helps you jump-start with the mGramSeva basic installation with limited functionalities.‌

mGramSeva is a distributed microservice-based platform that comprises many services that are containerized. Depending upon the required features, the specific services can be run on any container-supported orchestration platform like docker-compose, Kubernetes, etc.‌

The Quickstart guide covers the installation steps for basic services to get the platform up. Before setting up mGramSeva, create a lightweight Kubernetes cluster called k3d on a local machine with specified H/W requirements. The H/W requirements are listed below to ensure before we proceed further.‌

1. Infra Setup

To provision a lightweight Kubernetes cluster, please follow the instructions below in context to your OS and install the k3d on your machine.‌

H/W or VM Size

• min 4 vCPUs (recommended 8)

• min 8GiB of RAM (recommended 16)

• min 30GiB of HDD (recommended 30+)

Tools

• Linux distribution running in a VM or bare metal

  • Ubuntu 18.04 or Debian 10 (VM or bare metal)

  • Install Docker​

  • ​Install kubectl on Linux

  • Open terminal and Install k3d on Linux using the below command

  Copy

  wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash

• OSX or Mac

  • ​Docker Desktop local Kubernetes cluster enabled

  • ​Install kubectl on Mac

  • Install k3d on Mac, on terminal use Homebrew (Homebrew is available for MacOS) using the below command

  Copy

  brew install k3d

• Windows 10 or above

  • ​Docker Desktop for windows need to be installed

  • ​Install kubectl on Windows

  • ​Install Chocolatey package manager for windows

  • Install GitBash as an alternative command prompt that allows most of the Linux commands on windows.

  • Open gitbash and Install k3d on Windows using the below command

  Copy

  choco install k3d

Infra (Kubernetes Cluster) Creation

Once the above prerequisites are met, run the following tasks depending upon your OS.‌

• login/ssh into the machine, go to terminal/command prompt and run the following commands as an admin user.

• Create /Kube directory and change permission. To use this directory for persistent data mount. This means data from all container logs will be stored here.

 mkdir /kube
 chmod 777 /kube

 #if you are unable to create the /kube folder in the root
 # you can create it your user directory and provide the absolue path below

• Create a cluster with a single master node and 2 agents (Worker Nodes) and mount the pre-created directory (for data persistence).

k3d cluster create --k3s-server-arg "--no-deploy=traefik" --agents 2 -v "/kube:/kube@agent[0,1]" -v "/kube:/kube@server[0]" --port "80:80@loadbalancer"

• ‌When cluster creation is successful, get the kubeconfig file, that allows you to connect to the cluster at any time.

k3d kubeconfig get k3s-default > k3dconfig
kubectl config use-context k3d-k3s-default --kubeconfig=k3dconfig

• Verify the Cluster creation by running the following commands from your local machine where the kubectl is installed. It gives you the sample output as below

root@ip:/# kubectl cluster-info

OutPut:
Kubernetes control plane is running at https://0.0.0.0:33931
CoreDNS is running at https://0.0.0.0:33931/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://0.0.0.0:33931/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy



root@ip:/# k3d cluster list
NAME          SERVERS   AGENTS   LOADBALANCER
k3s-default   1/1       2/2      true

• You can verify the workers' nodes created by using the following command.

kubectl get nodes

Output:
NAME                       STATUS   ROLES                  AGE     VERSION
k3d-k3s-default-agent-0    Ready    <none>                 3d18h   v1.21.1+k3s1
k3d-k3s-default-agent-1    Ready    <none>                 3d18h   v1.21.1+k3s1
k3d-k3s-default-server-0   Ready    control-plane,master   3d18h   v1.21.1+k3s1


kubectl top nodes

Output:
W0625 07:56:24.588781   12810 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k3d-k3s-default-agent-0    547m         6%     1505Mi          9%        
k3d-k3s-default-agent-1    40m          0%     2175Mi          13%       
k3d-k3s-default-server-0   59m          0%     2286Mi          14%

‌Once the above steps are completed successfully, your Cluster is now up and running ready to proceed with the DIGIT Deployment.‌

2. mGramSeva Setup

‌Now that we have the Infra setup to proceed with the DIGIT Deployment. Following are the tools that need to be installed on the machine before proceeding with the DIGIT Services deployment.‌

What we'll deploy in Quickstart:‌

• mGramSeva core platform services

Prerequisites

1. mGramSeva uses golang (required v1.13.3) automated scripts to deploy the builds on to Kubernetes - Linux or Windows or Mac​

3. ​kubectl is a CLI to connect to the kubernetes cluster from your machine

4. Install CURL for making API calls

5. ​Install Visualstudio IDE Code for better code/configuration editing capabilities

6. The mGramSeva services deployment configurations are in GitRepo which needs to install git and then git clone it to your local.

7. ​Install Postman to run some DIGIT bootstrap scripts

root@ip:/# git clone -b quickstart https://github.com/misdwss/iFix-DevOps

‌After cloning the repo CD into the folder iFix-DevOps and type the "code ." command that will open the visual editor and opens all the files from the repo iFix-DevOps

root@ip:/# cd iFix-DevOps
root@ip:DIGIT-DevOps# code .

Check the sample deployment config file that needs to be configured as per any specific values according to your needs. (For a quick start you can run as it is)

https://github.com/misdwss/iFix-DevOps/blob/quickstart/deploy-as-code/helm/environments/quickstart-config.yaml

Add the following entries in your host file /etc/hosts depending on your OS, instructions can be found here.

127.0.0.1 quickstart.local.digit

‌3. Deployment

‌Once the prerequisite setup is complete, go to the following repo, run the command and follow the instructions.

root@ip:# cd iFix-DevOps/deploy-as-code/egov-deployer

root@ip:# go run mgramseva_setup.go

#Be prepared for the following questions
1. Do you have the Kubernetes Setup?
2. Provide the path of the intented env kubeconfig file
3. Which version of the DIGIT that you want to install - Choose "Quickstart"
4. What mGremSeva Modules that you choose to install
5. All, done, Now do you want to preview the deployment manifests 
6. Are you good to proceed with the DIGIT Installation

All Done.

‌You can now test the DIGIT application status in the command prompt/terminal by using the below command.

curl -Is http://quickstart.local.digit/employee/login |  head -n 1HTTP/2 200

The Amazon Elastic Kubernetes Service (EKS) is one of the AWS services for deploying, managing, and scaling any distributed and containerized workloads, here we can provision the EKS cluster on AWS from ground up and using an automated way (infra-as-code) using terraform and then deploy the DIGIT-iFIX Services config-as-code using Helm.

Pre-reads

• Know about EKS: https://www.youtube.com/watch?v=SsUnPWp5ilc

• Know what is terraform: https://youtu.be/h970ZBgKINg

Pre-requisites

1. ​AWS account with the admin access to provision EKS Service, you can always subscribe to free AWS account to learn the basics and try, but there is a limit to what is offered as free, for this demo you need to have a commercial subscription to the EKS service.

2. Install kubectl on your local machine that helps you interact with the kubernetes cluster

3. Install Helm that helps you package the services along with the configurations, envs, secrets, etc into a kubernetes manifests

4. Install terraform version (0.14.10) for the Infra-as-code (IaC) to provision cloud resources as code and with desired resource graph and also it helps to destroy the cluster at one go.

5. ​Install AWS CLI on your local machine so that you can use aws cli commands to provision and manage the cloud resources on your account.

6. Install AWS IAM Authenticator that helps you authenticate your connection from your local machine so that you should be able to deploy DIGIT services.

7. Use the AWS IAM User credentials provided for the Terraform (Infra-as-code) to connect with your AWS account and provision the cloud resources.

   ​

   • You'll get a Secret Access Key and Access Key ID. Save them safely.

   • Open the terminal and run the following command. The AWS CLI is already installed and the credentials are saved. (Provide the credentials and you can leave the region and output format blank).

   Copy

   aws configure --profile mgramseva-infra-account 
   
   AWS Access Key ID []:<Your access key>
   AWS Secret Access Key []:<Your secret key>
   Default region name []: ap-south-1
   Default output format []: text

   The above will create the following file In your machine as /Users/.aws/credentials

   Copy

   [mgramseva-infra-account] 
   aws_access_key_id=*********** 
   aws_secret_access_key=****************************

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (EKS, Worker Nodes, Postgress DB, EBS Volumes, Load Balancer)

Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every environment, it'll have the same infra.

• EKS Control Plane (Kubernetes Master)

• Work node group (VMs with the estimated number of vCPUs, Memory)

• Node-pool's (mgramseva and ifix)

• EBS Volumes (persistent volumes)

• RDS (Postgresql)

• VPCs (private network)

• Users to access, deploy and read-only

Understand The Resource Graph In Terraform Script

• Ideally, one would write the terraform script from the scratch using this doc.

• Here we have already written the terraform script that provisions the production-grade DIGIT Infra and can be customized with the specified configuration.

• Let's clone the iFix-DevOps GitHub repo where the terraform script to provision EKS cluster is available and below is the structure of the files.

git clone https://github.com/misdwss/iFix-DevOps.git
cd iFix-DevOps/infra-as-code/terraform


└── modules
    ├── db
    │   └── aws
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── kubernetes
    │   └── aws
    │       ├── eks-cluster
    │       │   ├── main.tf
    │       │   ├── outputs.tf
    │       │   └── variables.tf
    │       ├── network
    │       │   ├── main.tf
    │       │   ├── outputs.tf
    │       │   └── variables.tf
    │       └── workers
    │           ├── main.tf
    │           ├── outputs.tf
    │           └── variables.tf
    └── storage
        └── aws
            ├── main.tf
            ├── outputs.tf
            └── variables.tf

Example:

• VPC Resources:

  • VPC

  • Subnets

  • Internet Gateway

  • Route Table

• EKS Cluster Resources:

  • IAM Role to allow EKS service to manage other AWS services

  • EC2 Security Group to allow networking traffic with EKS cluster

  • EKS Cluster

• EKS Worker Nodes Resources:

  • IAM role allowing Kubernetes actions to access other AWS services

  • EC2 Security Group to allow networking traffic

  • Data source to fetch the latest EKS worker AMI

  • AutoScaling Launch Configuration to configure worker instances

  • AutoScaling Group to launch worker instances

• Database

  • Configuration in this directory creates a set of RDS resources including DB instance, DB subnet group, and DB parameter group.

• Storage Module

  • Configuration in this directory creates EBS volume and attaches it together.​

The following main.tf with create s3 bucket to store all the state of the execution to keep track.

iFix-DevOps/Infra-as-code/terraform/sample-eks/remote-state

main.tf​

provider "aws" {
  region = "ap-south-1"
}

#This is a bucket name that you can name as you wish
resource "aws_s3_bucket" "terraform_state" {
  bucket = "try-workshop-yourname" 

  versioning {
    enabled = true
  }

  lifecycle {
    prevent_destroy = true
  }
}

#This is a bucket name that you can name as you wish
resource "aws_dynamodb_table" "terraform_state_lock" {
  name           = "try-workshop-yourname"
  read_capacity  = 1
  write_capacity = 1
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

2. The following main.tf contains the detailed resource definitions that need to be provisioned, please have a look at it.

Dir: iFix-DevOps/Infra-as-code/terraform/sample-eks

main.tf

terraform {
  backend "s3" {
    bucket = "try-workshop-yourname"
    key = "terraform"
    region = "ap-south-1"
  }
}

module "network" {
  source             = "../modules/kubernetes/aws/network"
  vpc_cidr_block     = "${var.vpc_cidr_block}"
  cluster_name       = "${var.cluster_name}"
  availability_zones = "${var.network_availability_zones}"
}


module "iam_user_deployer" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "${var.cluster_name}-kube-deployer"
  force_destroy = true  
  create_iam_user_login_profile = false
  create_iam_access_key         = true

  # User "egovterraform" has uploaded his public key here - https://keybase.io/egovterraform/pgp_keys.asc
  pgp_key = "${var.iam_keybase_user}"
}

module "iam_user_admin" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "${var.cluster_name}-kube-admin"
  force_destroy = true  
  create_iam_user_login_profile = false
  create_iam_access_key         = true

  # User "egovterraform" has uploaded his public key here - https://keybase.io/egovterraform/pgp_keys.asc
  pgp_key = "${var.iam_keybase_user}"
}

module "iam_user_user" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-user"

  name          = "${var.cluster_name}-kube-user"
  force_destroy = true  
  create_iam_user_login_profile = false
  create_iam_access_key         = true

  # User "test" has uploaded his public key here - https://keybase.io/test/pgp_keys.asc
  pgp_key = "${var.iam_keybase_user}"
}

data "aws_eks_cluster" "cluster" {
  name = "${module.eks.cluster_id}"
}

data "aws_eks_cluster_auth" "cluster" {
  name = "${module.eks.cluster_id}"
}
provider "kubernetes" {
  host                   = "${data.aws_eks_cluster.cluster.endpoint}"
  cluster_ca_certificate = "${base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)}"
  token                  = "${data.aws_eks_cluster_auth.cluster.token}"
  load_config_file       = false
  version                = "~> 1.11"
}

module "eks" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = "${var.cluster_name}"
  cluster_version = "${var.kubernetes_version}"
  subnets         = "${concat(module.network.private_subnets, module.network.public_subnets)}"

  tags = "${
    map(
      "kubernetes.io/cluster/${var.cluster_name}", "owned",
      "KubernetesCluster", "${var.cluster_name}"
    )
  }"

  vpc_id = "${module.network.vpc_id}"

  worker_groups_launch_template = [
    {
      name                    = "spot"
      subnets                 = "${concat(slice(module.network.private_subnets, 0, length(var.availability_zones)), slice(module.network.public_subnets, 0, length(var.availability_zones)))}"
      override_instance_types = "${var.override_instance_types}"
      asg_max_size            = 1
      asg_desired_capacity    = 1
      kubelet_extra_args      = "--node-labels=node.kubernetes.io/lifecycle=spot"
      spot_allocation_strategy= "capacity-optimized"
      spot_instance_pools     = null
    },
  ]
  
  map_users    = [
    {
      userarn  = "${module.iam_user_deployer.iam_user_arn}"
      username = "${module.iam_user_deployer.iam_user_name}"
      groups   = ["system:masters"]
    },
    {
      userarn  = "${module.iam_user_admin.iam_user_arn}"
      username = "${module.iam_user_admin.iam_user_name}"
      groups   = ["global-readonly", "digit-user"]
    },
    {
      userarn  = "${module.iam_user_user.iam_user_arn}"
      username = "${module.iam_user_user.iam_user_name}"
      groups   = ["global-readonly"]
    },    
  ]
}

module "db" {
  source                        = "../modules/db/aws"
  subnet_ids                    = "${module.network.private_subnets}"
  vpc_security_group_ids        = ["${module.network.rds_db_sg_id}"]
  availability_zone             = "${element(var.availability_zones, 0)}"
  instance_class                = "db.t3.medium"
  engine_version                = "11.5"
  storage_type                  = "gp2"
  storage_gb                    = "100"
  backup_retention_days         = "7"
  administrator_login           = "egovdev"
  administrator_login_password  = "${var.db_password}"
  db_name                       = "${var.cluster_name}-db"
  environment                   = "${var.cluster_name}"
}

module "es-master" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "es-master"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}
module "es-data-v1" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "es-data-v1"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "25"
  
}

module "zookeeper" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "zookeeper"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}

module "kafka" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "kafka"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "50"
  
}

module "zookeeper-ifix" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "zookeeper-ifix"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}

module "kafka-ifix" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "kafka-ifix"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "50"
  
}

module "node-group" {  
  for_each = toset(["ifix-dev", "mgramseva"])
  source = "../modules/node-pool/aws"

  cluster_name        = "${var.cluster_name}"
  node_group_name     = "${each.key}-ng"
  kubernetes_version  = "${var.kubernetes_version}"
  security_groups     =  ["${module.network.worker_nodes_sg_id}"]
  subnet              = "${concat(slice(module.network.private_subnets, 0, length(var.node_pool_zone)))}"
  node_group_max_size = 1
  node_group_desired_size = 1
}  

Custom Variables/Configurations

Define your configurations in variables.tf. Provide the environment-specific cloud requirements and use the same terraform template to customize the configurations.

├── sample-eks
│   ├── main.tf 
│   ├── outputs.tf
│   ├── providers.tf
│   ├── remote-state
│   │   └── main.tf
│   └── variables.tf

The values given below must be mentioned in the following files. The blank ones will be prompted for inputs while execution.

​variables.tf ​

## Add Cluster Name
variable "cluster_name" {
  default = "<Desired Cluster name>"  #eg: my-digit-eks
}

## Add vpc_cidr_block
variable "vpc_cidr_block" {
  default = "CIDR" 
}

# If you want prod grade N/W, you can define HA, DRS with multi zone
variable "network_availability_zones" {  
  default = ["ap-south-1b", "ap-south-1a"]
}

# Which zone, it matters
variable "availability_zones" {
  default = ["ap-south-1b"]
}

variable "node_pool_zone" {
 default = "ap-south-1a"  
}

variable "kubernetes_version" {
  default = "1.18"
}

# instance type for your worker nodes like r5a.large is 8 vCPU and 16GB RAM
variable "instance_type" {
  default = "r5a.large"
}

# spot instance configuration
variable "override_instance_types" {
  default = ["r5a.large", "r5ad.large", "r5d.large", "t3a.xlarge"] 
}

# number of machines as per estimate
variable "number_of_worker_nodes" {
  default = "1"
}

##Add ssh key in case you want to ssh to nodes
variable "ssh_key_name" {
  default = "ssh key name"
}

# terraform users ssh public key, you need to one for you, refer below to create yours
variable "iam_keybase_user" {
  default = "keybase:egovterraform"
}

# will be prompted to provide during the execution
variable "db_password" {}

​Use the URL https://keybase.io/ to create your own PGP key. This creates both public and private keys on your machine. Upload the public key into the keybase account that you have just created, give a name to it and ensure that you mention that in your terraform. This allows the encryption of all sensitive information.

• Example - the keybase user (in eGov case is "egovterraform") needs to be created and has to be uploaded the public key here - https://keybase.io/egovterraform/pgp_keys.asc​

• you can use this portal to decrypt your secret key. To decrypt PGP Message, upload the PGP Message, PGP Private Key and the Passphrase.

Run Terraform

Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.

Let's begin to run the Terraform scripts to provision infra required to Deploy DIGIT on AWS.

1. First CD into the following directory and run the following command 1-by-1 and watch the output closely.

cd iFix-DevOps/infra-as-code/terraform/sample-eks/remote-state 
terraform init
terraform plan
terraform apply


cd iFix-DevOps/infra-as-code/terraform/sample-eks 
terraform init
terraform plan
terraform apply

Upon Successful execution following resources get created which can be verified by the command "terraform output"

• s3 bucket: to store terraform state.

• Network: VPC, security groups.

• IAM users auth: using the key base to create admin, deployer and the user. Use this URL https://keybase.io/ to create your own PGP key, this will create both public and private keys in your machine. Upload the public key into the keybase account that you have just created, give a name to it and ensure that you mention that in your terraform. This allows for encrypting all sensitive information.

  • Example: keybase user (in eGov case is "egovterraform") needs to be created and has to be uploaded the public key here - https://keybase.io/egovterraform/pgp_keys.asc​

  • you can use this portal to decrypt your secret key. To decrypt PGP Message, Upload the PGP Message, PGP Private Key and Passphrase.

• EKS cluster: with master(s) & worker node(s).

• Storage(s): for es-master, es-data-v1, es-master-infra, es-data-infra-v1, zookeeper, kafka, kafka-infra.

2. Use this link to get the kubeconfig from EKS to get the kubeconfig file and be able to connect to the cluster from your local machine so that you should be able to deploy DIGIT services to the cluster.

aws sts get-caller-identity

# Run the below command and give the respective region-code and the cluster name
aws eks --region <region-code> update-kubeconfig --name <cluster_name>

3. Verify that you are able to connect to the cluster by running the following command

kubectl config use-context <your cluster name>

kubectl get nodes

NAME                                             STATUS AGE   VERSION               OS-Image           
ip-192-168-xx-1.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-2.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-3.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-4.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2 

Whola! All set and now you can go with Deploy Product..

Introduction

iFIX/mGramSeva is a microservices-based distributed cloud-native application. Each of these context-specific microservices is dockerized and deployed on Kubernetes infrastructure.

Pre-reads

It is essential to understand some of the key concepts, benefits and best practices of the Kubernetes platform before we understand the deployment of the iFIX/mGramSeva.

• Know the basics of Kubernetes: https://www.youtube.com/watch?v=PH-2FfFD2PU&t=3s​

• Know the basics of kubectl commands

• Know Kubernetes manifests: https://www.youtube.com/watch?v=ohSUtEfDefc​

• Know how to manage environment values and secrets of any service deployed in Kubernetes https://www.youtube.com/watch?v=OW244LxB4oI​

• Know how to port forward to a pod running inside k8s cluster and work locally https://www.youtube.com/watch?v=TT3nd5n5Yus​

• Know sops to secure your keys/creds: https://www.youtube.com/watch?v=DWzJ87KbwxA​

1. Choose the Install Type

Choose the target infra type and follow the instructions to set up a Kubernetes cluster before moving on to the deployment.

2. Deployment

Before we begin the deployment, it is important to understand the deployment architecture that starts from the source code to the production-ready stage. Deploying and managing Kubernetes have emerged as a streamlined way to deploy containers in the cloud infrastructure. When running Kubernetes at scale, managing, operating, and scaling its infrastructure to maximize cluster utilization can be challenging. There are too many parameters the development team needs to manage and configure. This includes selecting the best instance type and size, determining when to scale up or down, and making sure all of the containers are scheduled and running on the best instances — and that is even before starting to think about cost resource optimization.

The simplest way to get started with the deployment process is to manage deployment configuration as code. Each service deployment configuration is defined as Helm charts and deployed into the Kubernetes cluster. We can collocate the deployment-as-code as source code, leveraging all the benefits of source control including change tracking and branching and then packaging it. The source code repo below contains the deployment-as-code details for iFIX.

1. mGramSeva Installation

2. iFIX-adapter Installation

3. iFIX Ref Dashboard Installation

3. Destroy the Setup

Use the command below to clean up the setup cluster. This deletes the entire cluster and other cloud resources that were provisioned for the mGramSeva Infra Setup.

cd iFix-DevOps/infra-as-code/terraform/my-iFix-eksterraformdestroy​

Conclusion

All done, the infra on local, cloud, and deployment of iFIX into the Kubernetes cluster is completed successfully.