Excel Ingestion Configuration

Advanced Configurations

Overview

The Excel Ingestion service is a core component of the HCM (Health Campaign Management) platform. Its primary responsibility is to:

  • Generate Excel templates with validations and localisation

  • Ingest uploaded Excel files

  • Validate, transform, and process bulk data

  • Interact asynchronously with downstream services

The service supports ingestion for campaigns, boundaries, facilities, individuals, users, and related domain data.

Dependencies

The Excel Ingestion service depends on the following external services and infrastructure components:

  • PostgreSQL – persistent storage and migration tracking

  • Kafka – event-based asynchronous processing

  • MDMS – master data lookup

  • File Store – upload and download of Excel files

  • Boundary, Facility, Individual, Campaign services – domain validations

  • LocalisationLocalization service – multilingual support

Configuration is managed using Helm charts and deployed on Kubernetes via a values.yaml file.

Steps

Helm Configuration

The Excel Ingestion service depends on Kafka and Postgres. You need to set environment variables in the values.yaml file or Helm templates.

Excel Ingestion Helm

Common Labels

  • Used for Kubernetes resource identification and grouping.

Namespace Configuration

  • Deploys the service in the health namespace.

  • Database schema and DB URL selection depend on this value.

Ingress Configuration

  • Exposes the service through Zuul API Gateway

  • Base context path: /excel-ingestion

Init Containers – Database Migration

  • Executes Flyway database migrations before the application starts

  • Tracks migration history in excel_ingestion_flyway_history

Application Container Configuration

  • Runs a single replica by default

  • Can be scaled based on ingestion load

Health Checks

  • Ensures Kubernetes can detect unhealthy pods

  • Uses Spring Boot actuator-style health endpoint

Application Runtime Configuration

  • Java Spring Boot application

  • Tracing enabled for distributed request tracking

  • Heap size tuned to avoid OOM during large Excel processing

Debug Configuration

  • Enables remote debugging when required

  • Disabled by default in production

Server Configuration

  • Application runs on port 8080

  • Base context path: /excel-ingestion

Localisation Configuration

  • Default locale used for:

    • Template generation

    • Validation messages

    • Error responses

External Service Host Configuration

All service hosts are resolved from the egov-service-host ConfigMap.

Environment Variable
ConfigMap Key

EGOV_BOUNDARY_HOST

boundary-service

EGOV_FILESTORE_HOST

egov-filestore

EGOV_CAMPAIGN_HOST

project-factory

EGOV_HEALTH_INDIVIDUAL_HOST

health-individual

EGOV_FACILITY_HOST

facility

EGOV_LOCALIZATION_HOST

egov-localization

EGOV_MDMS_HOST

egov-mdms-service

Service Endpoints

Purpose
Endpoint

Campaign Search

project-factory/v1/data/campaign/_search

Campaign Bulk Decrypt

project-factory/v1/crypto/_bulkDecrypt

Individual Search

health-individual/v1/_search

Hierarchy Search

boundary-service/boundary-hierarchy-definition/_search

Boundary Relationship Search

boundary-service/boundary-relationships/_search

Facility Search

facility/v1/_search

File Upload

filestore/v1/files

File URL Fetch

filestore/v1/files/url

Localization Search

localization/messages/v1/_search

MDMS Search

egov-mdms-service/v2/_search

Database Configuration

Database connection values are dynamically selected based on the namespace.

Database Connection Details

Parameter
Source
Description

SPRING_DATASOURCE_URL

health-db-url or db-url

Namespace-based DB URL

DB_HOST

egov-config

Database host

DB_PORT

5432

PostgreSQL default port

DB_NAME

egov-config

Database name

DB_SCHEMA

Namespace or public

Schema selection

DB_USER

db secret

Database username

DB_PASSWORD

db secret

Database password

JPA / Hibernate Configuration

  • Automatically updates schema

  • Uses PostgreSQL dialect

Flyway Migration Configuration

Parameter
Source

FLYWAY_USER

db secret

FLYWAY_PASSWORD

db secret

FLYWAY_LOCATIONS

egov-config

SCHEMA_TABLE

excel_ingestion_flyway_history

Kafka Producer Configuration

  • Enables event publishing for async processing

Redis Health Check Configuration

  • Redis health check is disabled as Redis is not mandatory for Excel Ingestion

circle-info

Notes

  • Secrets vs ConfigMaps

    • Sensitive data (DB credentials, Flyway credentials) is stored in Kubernetes Secrets

    • Non-sensitive values are stored in ConfigMaps

  • Namespace-Aware Behaviour

    • Database URL and schema automatically change based on the namespace

    • Ensures isolation between environments

  • Performance Considerations

    • Heap and memory limits are tuned for large Excel uploads

    • Increase replicas if the concurrent ingestion volume increases

Last updated

Was this helpful?