Retry Campaign Creation

Campaign Creation and Retry Mechanism Documentation

Overview

This documentation outlines the Campaign Creation process, including the sequence of steps, status tracking, and a retry mechanism that allows the process to resume from the point of failure without restarting the entire workflow. The process is divided into multiple stages, with a system to handle failures and retries. Each step in the process involves a request and campaign object, which are passed to the main function for processing.

Key Components

1. Hard Configuration (Process Flow Configuration)

2. Campaign Creation and Retry Mechanism Design

3. Full Mechanism Workflow

4. Main Campaign Creation Function

5. Retry Logic identifiers

---

1. Hard Configuration (Process Flow Configuration)

The Process Flow Configuration defines the sequence of processes that occur during the campaign creation. This configuration serves as the blueprint for the campaign workflow, ensuring that each process is executed in the correct order.

Process Flow Configuration (JSON format):

{

"Project Creation": "Facility Creation",

"Facility Creation": "User Creation",

"User Creation": "Facility Mapping",

"Facility Mapping": "User Mapping",

"User Mapping": "Product Mapping",

"Product Mapping": "End"

}

Description of the Flow:

• Project Creation: Initialization of the campaign project.

• Facility Creation: Generation of facilities linked to the project.

• User Creation: Creation of users after facility setup.

• Facility Mapping: Mapping of facilities to the respective projects.

• User Mapping: Linking users to the respective projects.

• Product Mapping: Mapping products to the respective projects.

• End: Final step marking the completion of the campaign creation.

---

2. Campaign Creation and Retry Mechanism Design

2.1 Status Tracking Table

The Status Tracking Table monitors the progress of each process step for a specific campaign, capturing whether a process has started, completed, or failed. This table helps retry the failed steps.

Structure:

CampaignId	ProcessName	Status
2a7c9d91-e88b-4f54-8d88-4e70c08f85c1	Project Creation	Started
2a7c9d91-e88b-4f54-8d88-4e70c08f85c1	Facility Creation	Completed
2a7c9d91-e88b-4f54-8d88-4e70c08f85c1	Update Project Dates	Failed
2a7c9d91-e88b-4f54-8d88-4e70c08f85c1	Facility Mapping	Started

Columns:

• id : primary key

• CampaignId: Unique identifier for the campaign.

• ProcessName: The name of the current process being tracked.

• Status: The current status of the process (Started, Completed, Failed).

---

3. Full Mechanism Workflow

3.1 Starting Campaign Creation

• CampaignId Creation: A new CampaignId is generated for the campaign. The system checks the Status Tracking Table to ensure no existing entry for the campaign.

• Process Initialization: If the campaign does not exist, the first process step (Project Creation) begins.

3.2 Process Execution

The campaign creation proceeds through the following steps:

• Project Creation

• Facility Creation

• User Creation

• Facility Mapping

• User Mapping

• Product Mapping

• End: Marks the final step, indicating the campaign has been fully processed.

After each step, the Status Tracking Table is updated to reflect the process status and associated information .

3.3 Handling Failures

1. Failure Detection

• If a process fails during campaign creation, the Status Tracking Table is updated to reflect the Failed status for the affected process.

2. State Persistence

• At the time of failure, the current state of the CampaignObject is serialized and persisted into the campaign details table. This ensures that the system retains the latest state of the campaign, which can be used to resume the process during a retry.

3. Retry Trigger

• When the client triggers a retry, the following steps occur:

• 1. The system queries the Status Tracking Table to identify the failed process.

  2. The previously persisted CampaignObject state is retrieved.

  3. The system resumes the campaign creation process starting from the failed step, using the stored state to ensure consistency.

Retry Logic:

• The campaignId is used to get latest campaignObject and retry with proper param.

• The campaign resumes from the failed step, and once successful, the status is updated.

3.4 Completion

Once all steps are successfully completed, the End process is marked as "Completed." The CampaignId is marked as fully processed, completing the campaign creation process.

---

4. Main Campaign Creation Function

Function: createCampaign

Function Responsibility:

The createCampaign function is the central component responsible for orchestrating the entire campaign creation process. It:

• Accepts three parameters:

  • currentProcessName: Identifies the specific step in the campaign creation flow.

  • campaignId: The unique identifier for the campaign being created.

  • request: Represents the incoming request data required for the process.

• Uses a switch-case structure to determine and execute the corresponding action based on the provided currentProcessName.

---

Process Execution Flow

1. Validation Phase:

• Before invoking the createCampaign function, all required validations (e.g., input data checks, resource availability) must be successfully completed.

• This ensures that the campaign creation process is initiated only with valid and consistent data.

2. Execution Phase:

• After validation, the createCampaign function is invoked with the following parameters: currentProcessName, campaignId, and request.

• Based on the value of currentProcessName, the function performs the corresponding action (e.g., project creation, facility creation).

• It updates the Status Tracking Table with the progress or result of the process.

3. Failure Handling:

• If a failure occurs during any step:

  • The state of the campaign (identified by campaignId) is persisted in the Status Tracking Table.

  • The failure status is recorded.

  • The function exits gracefully, allowing the retry mechanism to resume from the failed step.

---

Function Implementation

public void createCampaign(String currentProcessName, String campaignId, Request request) {
    switch (currentProcessName) {
        case "Project Creation":
            // Handle Project Creation
            createProject(campaignId, request);
            break;

        case "Facility Creation":
            // Handle Facility Creation
            createFacility(campaignId, request);
            break;

        case "User Creation":
            // Handle User Creation
            createUser(campaignId, request);
            break;

        case "Facility Mapping":
            // Handle Facility Mapping
            mapFacility(campaignId, request);
            break;

        case "User Mapping":
            // Handle User Mapping
            mapUser(campaignId, request);
            break;

        case "Product Mapping":
            // Handle Product Mapping
            mapProduct(campaignId, request);
            break;

        case "End":
            // Final step, mark as completed
            completeCampaign(campaignId, request);
            break;

        default:
            throw new IllegalArgumentException("Unknown process: " + currentProcessName);
    }
}

---

Parameters:

• currentProcessName: The name of the current process being executed (e.g., "Project Creation").

• campaignId: The unique identifier of the campaign being processed.

• request: The request object that carries additional parameters or data necessary for the processing.

The createCampaign function dynamically handles each process in the campaign creation sequence and can resume from the point of failure when necessary.

---

Campaign Creation and Update Flow

1. Fetching Entity Data Based on Campaign ID:

• New Campaign Creation:

  • If the campaign is newly created, fetch the entity data using only the current campaignId.

  • This is determined by checking if the parentCampaignId is null. If it’s null, it indicates a new campaign creation, and only the current campaignId will be used to fetch the related data.

• Campaign Update (with Parent Campaign):

  • If the campaign is being updated (i.e., the parentCampaignId is a valid value), fetch entity data using both the current campaignId and the parentCampaignId.

  • This allows for fetching both the updated campaign's data and the parent campaign's data to apply any changes or updates properly.

2. Handling Campaign Status (Active/Inactive):

• Campaign Activation/Deactivation:

  • Before starting the createCampaign process, check if the campaign needs to be activated or deactivated.

  • If the campaign is being deactivated, update its status accordingly and ensure that no new actions are performed for the campaign.

  • If the campaign is being activated, ensure all dependencies and related data are in the correct state for an active campaign.

  • Activation or deactivation will happen before the main logic of campaign creation starts or at the "End" stage, depending on the campaign's status and any changes made.

3. Handling Other Info Persistence:

• Any additional campaign-related information, such as configurations, metadata, or settings, will be handled either at the start of the campaign creation process or at the "End" switch case.

• This ensures that all necessary data is persisted and that updates to campaign details are completed before finalizing the campaign.

---

Logic for createProject

About Entity table

The campaignProject table is used to store information about the projects related to a specific campaign. It keeps track of the project’s status, its target data, and the associated boundary.

Table Columns:

1. id: A unique identifier for each record (Primary Key).

2. projectId: The ID of the project linked to the campaign.

3. campaignNumber: The campaign number associated with the project.

4. boundaryCode: A code representing the geographical boundary for the project.

5. isActive: A flag indicating whether the project is active (true) or inactive (false).

Steps

1. Delete Projects

1. Fetch all active projects (isActive = true) associated with the parentCampaignId(if present) from the campaignProject table.

2. Compare the boundaryCode of each project in the database with the boundaries listed in the target sheet.

3. For any project in the database whose boundaryCode is not present in the target sheet:

• Fetch the project details from the Project Service.

• Update the project in the Project Service by setting its parent to null, and inactive, effectively removing its association.

• Mark the corresponding row in the campaignProject table as isActive = false. And then add new rows associated with current campaignId and projects.

---

2. Update Projects

• Compare Target Data:

  • Compare the target data from the target sheet with the targets JSON stored in the database.

• Identify Differences:

  • If there is a discrepancy between the target data in the target sheet and the stored targets JSON, proceed to the next steps.

• Fetch Project Details:

  • Retrieve the project details from the Project Service using the relevant project identifier.

• Update Project Data:

  • Update the project in the Project Service with the new target data.

• Update CampaignProject Data:

  • Update the targets JSON column in the campaignProject table for the current campaignNumber, ensuring the new targets data is reflected correctly.

---

3. Create Projects

• Check if Project Exists:

  • For each row in the target sheet with a boundaryCode, check if a project with the same boundaryCode exists in the campaignProject table for the given parentCampaignNumber, regardless of whether isActive is true or false.

• Project Exists with isActive = false:

  • If a project with the same boundaryCode exists but has isActive = false:

    1. Compare the target data in the target sheet with the targets JSON stored in the database for that project.

    2. If there is a difference between the target data:

       • Fetch the project details from the Project Service.

       • Update the project in the Project Service with the new target data using the Project Update API.

       • Update the targets JSON in the campaignProject table to reflect the new target data.

       • Set isActive = true for the project in the campaignProject table to mark it as active.

       • Additionally, create a new row in the campaignProject table with the current campaignId and the newly activated project.

• Project Does Not Exist:

  • If no project with the specified boundaryCode exists in the campaignProject table for the given parentCampaignId:

    1. Use the Project Create API to create a new project with the data from the target sheet.

    2. Insert a new row into the campaignProject table with the following details:

       • campaignNumber: Current campaign number.

       • projectId: From the newly created project.

       • boundaryCode: From the target sheet.

       • targets JSON: From the target sheet.

       • isActive: Set to true.

Logic for createFacility

About Entity table

The campaignFacility table is used to track facilities associated with a specific campaign. It contains the following columns:

• id: A unique identifier for each record.

• facilityId: The unique ID of the facility.

• campaignNumber: The campaign number, the facility is associated with.

• boundaryCode: A code representing the geographical location or boundary of the facility.

• isActive: A flag indicating whether the facility is active (true) or inactive (false).

These columns help manage and track facilities in relation to campaigns, enabling efficient updates, deletions, and creations based on the campaign's requirements.

Steps

1. Delete Facility:

Logic:

• For each facility marked as inactive in the sheet:

  1. Search for active facilities in the campaignFacility table that match the facilityId and parentCampaignNumber.

  2. If matching records are found, set isActive = false to deactivate those facilities.

2. Update Facility:

Logic:

• Check for Boundary Change:

  1. For active facilities in the sheet that have a facilityId, check if the boundaryCode has changed compared to the existing records in the campaignFacility table.

  2. If a boundary change is detected:

     • Create a new row in the campaignFacility table with the updated boundaryCode, setting isActive = true and associating it with the current campaignNumber.

  3. No other updates are needed beyond the boundary code change.

3. Create Facility:

Logic:

• Check for New Facility in Sheet (Active Facility):

  1. For each row in the sheet, if facilityUsage is active and a facilityId is provided:

     • If the facilityId is provided:

       • Check if the facility already exists in the campaignFacility table.

         • If the facility exists and isActive = false, update isActive = true and create a new row for the current campaignId.

         • If the facility does not exist, create a new row in the campaignFacility table with the relevant details: facilityId, campaignNumber, boundaryCode, status, capacity.

     • If no facilityId is provided:

       • Create a new facility using the Facility Create API.

       • After creating the facility, insert a new record into the campaignFacility table with the relevant details: facilityId, campaignNumber, boundaryCode, status, capacity.

Logic for createUser

About Entity table

The campaignUser table is used to track users associated with a specific campaign. It contains the following columns:

• id: A unique identifier for each record.

• userserviceid: The service ID of the user user for mapping.

• campaignNumber: The campaign number, the user is associated with.

• phoneNumber: The phone number of the user.

• Role: The role of the user within the campaign.

• boundaryCode: A code representing the geographical boundary or location of the user.

Steps

1. Delete User:

Logic:

• For each user marked as inactive in the sheet:

  1. Search for active users in the campaignUser table that match the userserviceid and parentCampaignNumber.

  2. If matching records are found, set isActive = false to deactivate those users.

2. Update User:

Logic:

• Check for Boundary Code Change:

  1. For users in the sheet that are active and have a userserviceid, check if the boundaryCode has changed compared to the existing records in the campaignUser table.

  2. If a boundary code change is detected:

     • Mark the old record as inactive (isActive = false) and associate it with the current campaignNumber.

     • Create a new record with the same userserviceid, the new boundaryCode, and set isActive = true.

• No Other Update Use Cases:

  • Currently, there are no other update use cases for user data beyond the boundary code change.

3. Create User:

Logic:

• Check for New User in Sheet (Active User):

  1. For each row in the sheet, check if the usage is active and whether a userserviceid is provided:

     • If a userserviceid is provided:

       • Check if the user already exists in the campaignUser table.

         • If the user exists and isActive = false, set isActive = true and update the boundaryCode in the new row for the current campaignNumber.

         • If the user does not exist, create a new record in the campaignUser table with the relevant details: userserviceid, campaignNumber, name, phoneNumber, role, employment, boundaryCode.

     • If no userserviceid is provided:

       • Create a new user using the Employee Create API.

       • After the user is created, insert a new record into the campaignUser table with the relevant details: userserviceid, campaignNumber, name, phoneNumber, role, employment, boundaryCode.

Logic for mapFacility

Steps:

1. Retrieve Data:

   • Fetch all projects from the campaignProject table for the current campaignNumber.

   • Fetch all facilities, both active and inactive, from the campaignFacility table.

2. Handle Active Facilities:

   • For each active facility:

     • Extract the boundaryCode of the facility.

     • Match the boundaryCode with the boundary codes of the projects in the campaignProject table to find the associated projectId.

3. Check for Existing Mapping (Active Facilities):

   • Use the facilityId and projectId to search for an existing mapping in the project-service.

   • If no existing mapping is found, make a create mapping API call to the project-service to create a new mapping between the facility and the project.

4. Handle Inactive Facilities:

   • For each inactive facility:

     • Extract the boundaryCode of the facility.

     • Match the boundaryCode with the boundary codes of the projects in the campaignProject table to find the associated projectId.

5. Check for Existing Mapping (Inactive Facilities):

   • Use the facilityId and projectId to search for an existing mapping in the project-service.

   • If the mapping exists, make an update mapping API call to the project-service to deactivate the mapping (i.e., mark it inactive).

Overview of Actions:

• For Active Facilities:

  • Search for a mapping between the facility and the project.

  • If no mapping is found, create a new mapping.

• For Inactive Facilities:

  • Search for a mapping between the facility and the project.

  • If the mapping exists, update the mapping to make it inactive.

---

Logic for mapUser

Steps:

1. Retrieve Data:

   • Fetch all users from the campaignUser table for the current campaignNumber.

   • Fetch all active and inactive users from the system.

2. Handle Active Users:

   • For each active user:

     • Extract the boundaryCode of the user.

     • Match the boundaryCode with the boundary codes of the projects in the campaignProject table to find the associated projectId.

3. Check for Existing Mapping (Active Users):

   • Use the userId and projectId to search for an existing mapping in the project-service (or similar mapping service).

   • If no existing mapping is found:

     • Make a create mapping API call to the mapping service to establish a new mapping between the user and the project.

4. Handle Inactive Users:

   • For each inactive user:

     • Extract the boundaryCode of the user.

     • Match the boundaryCode with the boundary codes of the projects in the campaignProject table to find the associated projectId.

5. Check for Existing Mapping (Inactive Users):

   • Use the userId and projectId to search for an existing mapping in the project-service.

   • If the mapping exists, make an update mapping API call to the mapping service to deactivate the mapping (i.e., mark it inactive).

Overview of Actions:

• For Active Users:

  • Search for a mapping between the user and the project.

  • If no mapping is found, create a new mapping.

• For Inactive Users:

  • Search for a mapping between the user and the project.

  • If the mapping exists, update the mapping to make it inactive.

---

Logic for mapProduct

Steps:

1. Retrieve Product and Project Data:

   • Get a list of all products with pvarId from the deliveryRules of the campaignProject table.

   • Retrieve all projectIds related to the given campaignNumber from the campaignProject table.

2. Generate Product-Project Combinations:

   • Create a list of combinations with each pvarId paired with the corresponding projectId.

3. Check for New Mappings:

   • For each pvarId x projectId combination:

     • Search the mapping service (e.g., project-service) to check if the mapping between the product (pvarId) and the projectId already exists.

     • If no mapping exists:

       • Make a create mapping API call to the mapping service to establish a new mapping between the product and the project.

4. Delete Existing Extra Mappings:

   • Check for products that are missing from campaignNumber but exist in parentCampaignNumber.

   • Make their pvarId x projectId combinations.

   • Make extra mappings inactive via the mapping update API.

---

5. Retry Logic identifiers

1. Project Retry:

   • The retry mechanism for a project will be based on the target boundary code. If a failure occurs, the system will attempt to retry the project creation or update by matching the target boundary code associated with the project.

2. User Retry:

   • The retry mechanism for a user will be based on the phone number. In the event of a failure, the system will retry the process by identifying the user through their unique phone number, ensuring that the retry logic targets the correct user entry.

3. Facility Retry:

   • The retry mechanism for a facility will be based on a combination of the facility name, status, facility type, and capacity. In the future, this retry logic can be expanded to use a generated unique facility name or other identifiers to better target specific facilities during retry operations.

Notes

• Sheet Columns as Table Columns We might need to use sheet columns as table columns to reduce coupling between sheets and to generate all data related to a campaign for parent-to-child updates.

• Facility Retry Identifier The facility retry identifier should be updated to have a standard identifier for retries.

• Campaign ID vs. Campaign Number Since we receive parentCampaignId, having entity tables with a column as campaignId could be beneficial.

---

Tunings Required for 0.4

1. Request Handling:

   • Requests should be made only at the Controller level, avoiding internal service layers for such calls. This will ensure better separation of concerns and improve the overall structure of the codebase.

2. Strict Model Enforcement:

   • Strict models must be enforced consistently throughout the codebase. This will ensure better validation, predictability, and reduce the likelihood of errors or inconsistencies.

---

File Organization

1. Utility and Helper Files:

   • Utility and helper files should follow specific naming conventions and responsibilities to maintain clarity and ensure proper organization:

     • campaignUtils: Exclusively handles campaign-related utilities.

     • microplanUtils: Exclusively handles microplan-related utilities.

---

Enhancements

1. Locale Issue in Uploads:

   • A solution needs to be designed and finalized for handling locale-specific uploads. The solution will include a signature or unique identifier embedded within the generated template. This signature will ensure that the template can accept only the locale and campaignId for specific campaign . This enhancement can be scoped as a 0.4 enhancement to provide better internationalization support for users in different regions, ensuring that the correct locale-specific data is handled during the upload process.

2. Auto Migration on Build:

   • Introduce automatic database migrations during the execution of yarn run dev or yarn prod. This can be scoped as a 0.4 enhancement to streamline both the deployment process and the development workflow.