Manage Data APIs

This section of the documentation details APIs for creating, searching, generating, and downloading data, including request/response structures, flows, and error handling. Campaign Creation V2 Flow – Documentation

Overview

The Campaign Creation V2 flow improves upon the earlier V1 implementation by introducing a type-aware, config-driven, and extensible architecture for handling Excel-based data uploads.

This change simplifies how Excel sheets are parsed, validated, transformed, and annotated, making the flow scalable and easier to maintain.

V1 vs V2 – Key Differences

Feature

V1 (/v1/data/_create)

V2 (/v2/data/_process)

API Endpoint

/v1/data/_create

/v2/data/_process

Sheet Type Support

Embedded in sheet

Explicit in API payload (type)

Validation Support

Manual/Inline

Automated via MDMS config

Flexibility

Rigid, fixed structure

Modular, type-based flow

Extensibility

Limited

Easily extendable by adding type-specific logic

🛠️ API: POST /v2/data/_process

🔹 Purpose

Processes uploaded Excel files for validation-type sheets. This API is exposed to the client UI through role-action mapping.

🔹 Supported Types

Category

Types

Data Entry

user, facility, boundary

Validation

userValidation, facilityValidation, boundaryValidation

🔹 Sample Payload

{
  "ResourceDetails": {
    "tenantId": "dev",
    "type": "userValidation",
    "fileStoreId": "{{fileId}}",
    "hierarchyType": "NEWTEST00222",
    "additionalDetails": {},
    "campaignId": "5c845eb8-f8b2-43d0-a301-8e24692499f1"
  },
  "RequestInfo": {
    "apiId": "Rainmaker",
    "authToken": "{{Auth}}",
    "userInfo": {
      "id": 1052,
      "uuid": "8b110055-330f-4e7b-b4c0-f618f29b9d47"
    }
  }
}

🔹 Processing Behavior

  • Sheet Parsing: Extracts data from each sheet using configured schema.

  • Data Transformation: Applies field-level or row-level transformations (e.g., date parsing, ID lookup).

  • Validation: Enforces rules defined in MDMS.

  • Error Annotation: Errors are annotated row-wise and can be returned in response or in the processed Excel file.

🔄 API: POST /v2/data/_process-any

🔹 Purpose

Processes any sheet type (both validation and data entry).

Internal Use Only – this API is not exposed via role-action mapping. It is used internally by services like project-factory-service.

🔹 Differences from /v2/data/_process

Feature

/v2/data/_process

/v2/data/_process-any

Scope

Validation-only

All types (data + validation)

Use Case

UI pre-submission checks

Backend ingestion, internal flows

Config Handling

Strict type filtering

Dynamic resolution based on type

Role Access

✅ Yes

❌ No (internal only)

Intended Caller

Client (UI)

Internal services

🔹 Sample Payload

Same structure as /v2/data/_process, e.g.,

{
  "ResourceDetails": {
    "tenantId": "dev",
    "type": "facility",
    "fileStoreId": "{{fileId}}",
    "hierarchyType": "NEWTEST00222",
    "additionalDetails": {},
    "campaignId": "5c845eb8-f8b2-43d0-a301-8e24692499f1"
  },
  "RequestInfo": {
    "apiId": "Rainmaker",
    "authToken": "{{Auth}}",
    "userInfo": {
      "id": 1052,
      "uuid": "8b110055-330f-4e7b-b4c0-f618f29b9d47"
    }
  }

✅ When to Use Which API?

Scenario

Recommended API

Validate uploaded sheet via UI

/v2/data/_process

Internal data ingestion (backend)

/v2/data/_process-any

🔐 Role Access Mapping

API Endpoint

Role Mapping

Intended Caller

/v2/data/_process

✅ Yes

UI / client-facing

/v2/data/_process-any

❌ No

Internal service use

🧠 Internal Flow – How Processing Works

The V2 system uses a dynamic, type-based processing mechanism to ensure clean separation of concerns and better maintainability.

🔧 Step 1: Type-Based Class Resolution

The system uses the type field in the request to determine the processing class:

const className = `${ResourceDetails?.type}-processClass`;

This maps to a file like

processFlowClasses/userValidation-processClass.ts

processFlowClasses/facility-processClass.ts

🔄 Step 2: Dynamic Class Import and Execution

const { TemplateClass } = await import(classFilePath);

const sheetMap = await TemplateClass.process(ResourceDetails, wholeSheetData, localizationMap);

Each class implements a static process() method that:

  • Accepts the parsed sheet data

  • Performs transformation + validation

  • Returns a SheetMap object with data, errors, and annotations

🧩 Inside Each Type Class

Each class (e.g., userValidation-processClass) encapsulates all logic for that type:

  • Data cleanup or enrichment

  • Business rule validation

  • Conditional field logic

  • Return of structured, annotated sheet data

🌟 Why This Approach?

Feature

Benefit

🔌 Low Coupling

Each type is self-contained – changes don't affect others

🔧 Easily Extendable

Adding a new type is as simple as creating a new class

🧼 Clean Architecture

Sheet logic lives in its own file, not the main service

⚙️ Config-Driven

Sheet schema, headers, validation rules all come from MDMS

🧑‍💻 Developer Friendly

Developers only need to work on their type-specific class

📄 Campaign Template Generation V2 Flow – Documentation

🧾 Overview

The Template Generation V2 flow is an upgrade over the earlier /v1/data/_generate implementation. It enables a loosely coupled, type-driven, and config-based Excel template generation system, ensuring better maintainability and extensibility.

The legacy endpoint:

http

POST /v1/data/_generate

has now been replaced with:

http

POST /v2/data/_generate

🔁 V1 vs V2 – Key Differences

Feature

V1 (/v1/data/_generate)

V2 (/v2/data/_generate)

API Endpoint

/v1/data/_generate

/v2/data/_generate

Sheet Type Support

Embedded logic

Passed via type query param

Logic Coupling

Tightly coupled

Type-based, dynamic classes

Extensibility

Hard to extend

Add new types via config + class

⚙️ Earlier most things were hardcoded. Now, ~80% is config-driven via MDMS or file-based config, and ~20% uses dynamic functions for specialized transformation logic.

🛠️ API: POST /v2/data/_generate

🔹 Purpose

Generates Excel templates for campaign resource types such as user, facility, boundary, etc., using MDMS-driven config and dynamic logic. UI integrates with this endpoint to export downloadable templates for upload flows.

🔹 Required Query Parameters

Parameter

Type

Description

tenantId

string

Tenant under which the campaign runs

type

string

Type of template to generate (user, facility, etc.)

hierarchyType

string

Hierarchy code associated with the campaign

campaignId

string

ID of the campaign

🔹 Sample cURL Request


curl --location 'http://localhost:8080/project-factory/v2/data/_generate?tenantId=dev&type=user&hierarchyType=NEWTEST00222&campaignId=5c845eb8-f8b2-43d0-a301-8e24692499f1' \
--header 'Content-Type: application/json;charset=UTF-8' \
--data '{
  "RequestInfo": {
    "apiId": "Rainmaker",
    "authToken": "your-auth-token",
    "userInfo": {
      "id": 1052,
      "uuid": "8b110055-330f-4e7b-b4c0-f618f29b9d47",
      "userName": "UATMZ",
      "name": "UATMZ",
      "mobileNumber": "8998988112",
      "type": "EMPLOYEE",
      "roles": [
        {
          "code": "CAMPAIGN_MANAGER",
          "tenantId": "mz"
        }
      ],
      "active": true,
      "tenantId": "dev"
    }
  }
}'

🔧 Internal Flow – How Generation Works

🔹 Step 1: Type-Based Class Resolution

const className = `${responseToSend?.type}-generateClass`;

let classFilePath = path.join(__dirname, '..', 'generateFlowClasses', `${className}.js`);

if (!fs.existsSync(classFilePath)) {

classFilePath = path.join(__dirname, '..', 'generateFlowClasses', `${className}.ts`);

}

Expected files (per type):

  • user-generateClass.ts

  • facility-generateClass.ts

  • boundary-generateClass.ts

🔄 Step 2: Dynamic Import and Execution

const { TemplateClass } = await import(classFilePath);

const sheetMap = await TemplateClass.generate(ResourceDetails, localizationMap);

Returns an object with:

  • ✅ Sheet Name(s)

  • ✅ Header Row(s)

  • ✅ Pre-filled Data (if any)

  • ✅ Column metadata (dropdowns, styles, widths)

🔃 Transform Configuration (transformConfigs)

This configuration defines how API response data maps to Excel sheets and how sheet uploads map back to request payloads.

🔹 Driven By

  • MDMS config

  • File-based transformConfigs

  • Optional dynamic transformation functions (transformBulkX, transformX)

🧩 Sample: employeeHrms

HRMS Config
employeeHrms: {
  metadata: { tenantId: "dev", hierarchy: "MICROPLAN" },
  fields: [
    { path: "$.user.name", source: { header: "HCM_ADMIN_CONSOLE_USER_NAME" } },
    { path: "$.user.mobileNumber", source: { header: "HCM_ADMIN_CONSOLE_USER_PHONE_NUMBER" } },
    { path: "$.user.roles[*].name", source: { header: "HCM_ADMIN_CONSOLE_USER_ROLE", splitBy: "," } },
    { path: "$.user.roles[*].code", source: { header: "HCM_ADMIN_CONSOLE_USER_ROLE", splitBy: "," } },
    { path: "$.user.roles[*].tenantId", value: "${metadata.tenantId}" },
    { path: "$.user.userName", source: { header: "UserName" } },
    { path: "$.user.password", source: { header: "Password" } },
    { path: "$.user.tenantId", value: "${metadata.tenantId}" },
    { path: "$.user.dob", value: 0 },
    {
      path: "$.employeeType",
      source: {
        header: "HCM_ADMIN_CONSOLE_USER_EMPLOYMENT_TYPE",
        transform: {
          mapping: {
            Permanent: "PERMANENT",
            Temporary: "TEMPORARY",
            "%default%": "TEMPORARY"
          }
        }
      }
    },
    {
      path: "$.jurisdictions[*].boundary",
      source: { header: "HCM_ADMIN_CONSOLE_BOUNDARY_CODE_MANDATORY", splitBy: "," }
    },
    { path: "$.jurisdictions[*].tenantId", value: "${metadata.tenantId}" },
    { path: "$.jurisdictions[*].hierarchy", value: "${metadata.hierarchy}" },
    { path: "$.jurisdictions[*].boundaryType", value: "${metadata.hierarchy}" },
    { path: "$.tenantId", value: "${metadata.tenantId}" },
    { path: "$.code", source: { header: "UserName" } }
  ],
  transFormSingle: "transformEmployee",
  transFormBulk: "transformBulkEmployee"
}

🧩 Sample: Facility

Sample Payload
```
   "CampaignDetails":  {
            "id": "2b68a3aa-fca2-462e-8d59-8821c9a56132",
            "tenantId": "mz",
            "status": "drafted",
            "action": "create",
            "campaignNumber": "CMP-2024-10-15-004031",
            "isActive": true,
            "parentId": "43503755-db1d-4cef-9851-72360ee6eaa1",
            "campaignName": "LLIN-OCT15a",
            "projectType": "LLIN-mz",
            "hierarchyType": "HIERARCHYTEST",
            "boundaryCode": "",
            "projectId": null,
            "startDate": 1727202600000,
            "endDate": 1727720999000,
            "additionalDetails": {
                "key": 10,
                "beneficiaryType": "HOUSEHOLD"
            },
            "resources": [
            {
                "type": "facility",
                "filename": "update-facility.xlsx",
                "resourceId": "42c04e62-4b7b-4697-b2bd-aa90a212164c",
                "filestoreId": "20420b29-d913-4412-b3bd-34b850e2677b"
            },
            {
                "type": "boundaryWithTarget",
                "filename": "updated-boundary.xlsx",
                "resourceId": "8567db53-450e-465d-a14a-46e2690cf728",
                "filestoreId": "30ae3fd6-b77c-42e7-9a33-e88e29369764"
            },
            {
                "type": "user",
                "filename": "updated-user.xlsx",
                "resourceId": "fcc4a423-6ac5-4ee4-80c2-0f915a95253f",
                "filestoreId": "3c531af7-d923-47cc-b58a-c501a73e749c"
            }
        ],
            "boundaries": [],
            "deliveryRules": [
                {
                    "active": true,
                    "cycleIndex": "1",
                    "deliveries": [
                        {
                            "active": true,
                            "deliveryIndex": "1",
                            "deliveryRules": [
                                {
                                    "ruleKey": 1,
                                    "delivery": {},
                                    "products": [
                                        {
                                            "key": 1,
                                            "count": 1,
                                            "value": "PVAR-2024-05-09-000333"
                                        }
                                    ],
                                    "attributes": [
                                        {
                                            "key": 1,
                                            "value": "1",
                                            "operator": {
                                                "code": "LESS_THAN_EQUAL_TO"
                                            },
                                            "attribute": {
                                                "code": "CAMPAIGN_BEDNET_INDIVIDUAL_LABEL"
                                            }
                                        },
                                        {
                                            "key": 2,
                                            "value": "3",
                                            "operator": {
                                                "code": "LESS_THAN_EQUAL_TO"
                                            },
                                            "attribute": {
                                                "code": "CAMPAIGN_BEDNET_HOUSEHOLD_LABEL"
                                            }
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            ],
            "auditDetails": {
                "createdBy": "63a21269-d40d-4c26-878f-4f4486b1f44b",
                "lastModifiedBy": "63a21269-d40d-4c26-878f-4f4486b1f44b",
                "createdTime": 1728983362986,
                "lastModifiedTime": 1728983362995
            }
        },
```

🔹 Field Properties

Property

Description

path

JSON path in request/response

source.header

Column name in Excel sheet

value

Static value or dynamic reference like ${metadata.tenantId}

splitBy

Delimiter for multi-valued fields

transform

Mapping to normalize values (e.g., from "Permanent" to boolean true)

🌟 Why This Approach?

Feature

Benefit

🔌 Low Coupling

Per-type logic and config separation

⚙️ Config-Driven

Most headers, styles, mappings are externalized

🔧 Easily Extendable

Drop in a class + config entry for new types

🧼 Clean Architecture

Controller remains generic; logic lives in type handlers

🧩 Reusable Logic

Same config powers both generation and upload parsing

🔁 Retry Mechanism During Campaign Creation

🧾 Overview

The retry mechanism allows safe re-triggering of campaign creation only if the campaign is in a drafted or failed state. It uses the existing /v1/project-type/update API with action: "create" inside CampaignDetails. There is no dedicated retry API.

⚠️ Not fully idempotent 🔁 Re-uses creation API ✅ Works only for failed/drafted campaigns

🚫 When Retry Does NOT Work

  • If the campaign is already created and active, and a retry (action: "create") is attempted, it will throw an error.

  • Only campaigns in the following statuses are allowed to be re-processed:

["drafted", "failed"]

🚀 Retry API (Same as Create)

http

CopyEdit

POST /project-factory/v1/project-type/update

🔹 Required Query Params

Parameter

Type

Required

Description

tenantId

string

✅

Tenant ID

campaignId

string

✅

Campaign UUID (external)

📦 Sample Request Body

{

"RequestInfo": {

"apiId": "Rainmaker",

"authToken": "your-auth-token",

"userInfo": {

"uuid": "user-uuid",

"roles": [{ "code": "CAMPAIGN_MANAGER", "tenantId": "mz" }],

"tenantId": "dev"

}

},

"CampaignDetails": {

"campaignId": "5c845eb8-f8b2-43d0-a301-8e24692499f1",

"campaignNumber": "HCMP-CAM-00001",

"action": "create",

...

}

}

📊 Internal Tables Used

📁 eg_cm_campaign_data — Sheet Row Status

Tracks each data row uploaded from sheet.

Column

Description

campaignNumber

Campaign ref

type

Template type (user, facility)

uniqueIdentifier

Unique row key

data

Original sheet data (JSONB)

uniqueIdAfterProcess

Generated ID (after success)

status

✅ pending, completed, failed

export const dataRowStatuses = {

failed: "failed",

completed: "completed",

pending: "pending"

}

🔗 eg_cm_campaign_mapping_data — Mapping Status

Stores mapping status per row and boundary.

Column

Description

type

Mapping type (userMapping, etc.)

uniqueIdentifierForData

FK to campaign data

boundaryCode

Boundary associated

mappingId

Generated ID post-mapping

status

✅ toBeMapped, mapped, toBeDeMapped, deMapped

export const mappingStatuses = {

toBeMapped: "toBeMapped",

mapped: "mapped",

toBeDeMapped: "toBeDeMapped",

deMapped: "deMapped"

}

🧩 eg_cm_campaign_process_data — Process Flow Status

Tracks which steps of campaign creation are done/pending/failed.

Column

Description

processName

Each campaign step name

status

✅ pending, completed, failed

ts

CopyEdit

export const processStatuses = {

failed: "failed",

pending: "pending",

completed: "completed"

}

👇 Process Names

export const allProcesses = {

facilityCreation: "CAMPAIGN_FACILITY_CREATION_PROCESS",

userCreation: "CAMPAIGN_USER_CREATION_PROCESS",

projectCreation: "CAMPAIGN_PROJECT_CREATION_PROCESS",

facilityMapping: "CAMPAIGN_FACILITY_MAPPING_PROCESS",

userMapping: "CAMPAIGN_USER_MAPPING_PROCESS",

resourceMapping: "CAMPAIGN_RESOURCE_MAPPING_PROCESS",

userCredGeneration: "CAMPAIGN_USER_CRED_GENERATION_PROCESS"

}

🔁 Retry Logic Summary

Step

Logic

✅ Allowed Status

Retry only works if campaign is in drafted or failed

❌ Disallowed

Retry fails if campaign is already active or completed

Sheet Rows

Retries only rows with status pending or failed

Mapping Rows

Retries only rows with status toBeMapped or toBeDeMapped

Process Steps

Retries only steps with status pending or failed

Completed Work

Skipped entirely

✅ Final Notes

  • This approach provides partial re-execution based on current statuses.

  • Helps in recovery from partial failure (e.g., during bulk upload).

  • Prevents overwriting already created resources.

  • Make sure to validate campaign status before retrying.

📘 Document: App Config & Localisation Setup During Campaign Creation

🧩 Overview

When a new campaign is created using the /project-factory/v1/project-type/create API, the backend asynchronously triggers:

  • Creation of campaign-specific MDMS module configs

  • Copying of base localisations to campaign-localised keys

This happens in the background and does not block the campaign creation response.

⚙️ Trigger Point

API Endpoint

bash

CopyEdit

POST /project-factory/v1/project-type/create

Trigger Logic

ts

CopyEdit

createAppConfig(tenantId, campaignNumber, campaignType); // No await

  • createAppConfig(...) runs without await

  • The API returns immediately while config generation runs in parallel

🔧 Internal Flow: createAppConfig(...)

1. Fetch Enabled Template Modules

From schema: HCM-ADMIN-CONSOLE.FormConfigTemplate

ts

CopyEdit

const templateModules = (templateResponse?.mdms || [])

.map(item => item?.data || [])

.flat()

.filter(module => !module?.disabled);

✔️ Only modules where disabled !== true are processed.

2. Localisation Copy

For each module:

  • Base Key: hcm-base-{moduleName}-{campaignType}

  • Target Key: hcm-{moduleName}-{campaignNumber}

For each locale:

  • Fetch localisation entries using the base key

  • Replace module with the new target key

  • Upload entries in chunks of 100

await localisation.createLocalisation(chunk, tenantId);

🧩 RequestInfo is not passed 💡 Failures are logged but do not interrupt other locale uploads

3. Insert MDMS Module Config

Target schema: HCM-ADMIN-CONSOLE.FormConfig

Each module:

  • Is enriched with:

  • project: campaignNumber

  • isSelected: true

  • Is posted to MDMS v2 API:

ts

CopyEdit

await httpRequest(mdms_create_or_update_url, requestBody);

🧱 Module configs are associated with the campaign being created.

✅ Summary Table

Step

Description

Trigger API

POST /project-factory/v1/project-type/create

Async Execution

createAppConfig(...) runs in background, no delay to API response

Template Schema

HCM-ADMIN-CONSOLE.FormConfigTemplate

Target Schema

HCM-ADMIN-CONSOLE.FormConfig

Localisation Source Key

hcm-base-{moduleName}-{campaignType}

Localisation Target Key

hcm-{moduleName}-{campaignNumber}

Upload Method

In chunks of 100, per locale

Error Handling

Errors logged per module or locale, but execution continues gracefully

📝 Additional Notes

  • Ensures every campaign has its own set of configuration modules and localised messages

  • Localisation is cloned and renamed from base templates for each campaign

  • This system supports high performance and reliability through async background execution

PreviousProject FactoryNextV1 Deprecated Data Apis

Last updated

Was this helpful?