Technical Design Document: Manage Resources API Flows

Overview

This document outlines the various API flows and interactions for the Project Factory, focusing on resource data creation with retry logic, template generation, campaign update flows, and data search and download processes. The aim is to manage and automate various aspects of resource and campaign data efficiently using the Project Factory services.

Project Factory Resources Data Create with Retry Logic

Description: This flow describes how the Project Factory handles resource data creation with retry logic. It ensures data consistency and reliability by retrying operations if failures occur.

Sequence of Operations:

1. Client Request Initiation:

   • The client sends a request to the Project Factory with filestoreid and type.

2. MDMS Service Interaction:

   • Project Factory retrieves the type schema from the MDMS Service.

   • MDMS Service returns the schema, which is then validated.

3. Data Validation:

   • The Project Factory validates the data against the MDMS schema.

   • If validation is successful, it informs the client that data processing has started.

   • If validation fails, an error message is returned, and the user must resubmit.

4. File Handling:

   • The Project Factory requests a file download from the FileStore Service.

   • Upon receiving the file, JSON data is created from the file of the desired type.

5. Data Creation and Retry Logic:

   • Data is validated according to the schema.

   • Normal Create:

     • Data is created in the respective service, and status is tracked.

     • A loop continues until retry attempts exceed the maximum retries, or no failed data remains.

   • Bulk Create with Retry:

     • Data is created in the respective service, and the status is checked.

     • A search is conducted using the generateApi search.

     • The loop continues under the same conditions as the normal create process.

6. Database Update:

   • The file is enriched with the created ID and sent back to the FileStore Service.

   • The updated file is received, and data is persisted in the database with a status of 'created' and the processed filestoreid.

Template Generate API Flow (forceUpdate = true)

Description: This flow is triggered when a client sends a generate request with forceUpdate = true. It checks for existing data and creates a new row with the status inProgress if necessary.

Sequence of Operations:

1. Client Request Initiation:

   • The client sends a generate request with the specified type and forceUpdate = true.

2. Validation and Data Check:

   • The Project Factory validates the type against predefined types and hierarchytype.

   • A check is performed to see if previous data exists. If it does, the data is marked as expired.

3. Database and Response:

   • A new row is created with null fileStoreId and status inProgress.

   • The database is updated, and a success response is sent back to the client.

4. Data Fetch and Storage:

   • The Project Factory fetches the template from the MDMS Service based on the type.

   • Data is consolidated and stored in the FileStore Service, which returns a filestoreid.

   • The database is updated with the new filestoreid.

Template Generate API During Update Campaign Flow (forceUpdate = true)

Description: This flow is initiated when a generate request is sent due to a boundary change in an ongoing campaign.

Sequence of Operations:

1. Boundary Change Trigger:

   • The Project Factory sends a generate request with campaignId, type, and forceUpdate = true.

2. Validation and Data Check:

   • The type is validated against predefined types and hierarchytype.

   • Previous data is checked, and if found, marked as expired.

   • The campaign is searched in the database based on the provided campaignId and its parent campaign ID.

3. Database and Response:

   • A new row is created with null fileStoreId and status inProgress.

   • The database is updated, and a response is provided.

4. File Update and Data Consolidation:

   • The Project Factory fetches file details from the FileStore Service using the parent campaign object.

   • The sheet is updated, freezing all the data.

   • If new boundaries are added, boundary relation data is fetched.

   • Data is consolidated, and stored in the FileStore, and the database is updated with the new filestoreid.

Project Factory Resources Data Search and Download

Description: This flow allows the client to search and download resource data based on a provided ID and type.

Sequence of Operations:

1. Client Request Initiation:

   • The client sends a request with the id and type.

2. Database Query:

   • The Project Factory checks the created resource based on the provided ID.

   • The corresponding resource details row is fetched from the database.

3. Response to Client:

   • The Project Factory sends back the created resource response to the client.

API Sequence Diagrams

During Create flow

During Update Flow

Conclusion

The outlined API flows are crucial for handling resource data creation, management, and retrieval efficiently. They ensure data consistency, integrity, and seamless interaction between various services, making the Project Factory a robust system for managing campaign data. Implementing the retry logic, data validation, and template generation processes helps maintain a reliable and scalable architecture.

Technical Documentation: Create Campaign Flow

Overview

This document outlines the flow for creating a campaign using various services in the system. The process involves interactions between multiple services, including the Project Factory, Project Service, Boundary Service, Facility Service, HRMS Service, MDMS Service, FileStore Service, and the database. This flow ensures that the required resources are validated and correctly set up before finalizing the campaign creation.

Actors and Participants

• Client: The entity (user or system) initiating the campaign creation request.

• CampaignManager (Project Factory): The main controller managing the campaign creation process.

• ProjectService: Manages project-related operations, including data creation and mapping.

• BoundaryService: Handles fetching of boundary relationships based on hierarchy types.

• FacilityService: Responsible for creating facility-related data.

• HRMSService: Manages the creation of employee data.

• MDMSService: Provides master data such as project types.

• FileStoreService: Manages the storage and retrieval of resource files.

• Database: Stores campaign-related data and status updates.

Sequence Flow

1. Initiate Campaign Creation

   • The Client sends a request to the CampaignManager to create a campaign with all required and valid resources.

2. Fetch Boundary Relationship

   • The CampaignManager requests the BoundaryService to fetch the boundary relationship based on the hierarchy type.

   • The BoundaryService responds with the relevant boundary relationship data.

3. Fetch Project Type Master

   • The CampaignManager requests the MDMSService to fetch the project type master data.

   • The MDMSService responds with the project type master.

4. Validate Resource Files

   • The CampaignManager checks the Database to see if the input resource files have already been validated.

   • The Database responds with the validation status of the files.

5. Validation Check

   • The CampaignManager checks if all file templates and data are validated:

     • If validation is successful, the CampaignManager informs the Client that the campaign creation process has started, and the user needs to track the status using an ID.

     • If validation fails, the CampaignManager returns an error message, indicating which validation failed, and the user must resubmit the request.

6. Resource File Retrieval

   • The CampaignManager interacts with the FileStoreService to search for the resource based on the filestoreid.

   • The FileStoreService responds with the valid resource files.

7. Process Resource Data

   • The CampaignManager processes the retrieved sheets and identifies any resource data that needs to be created.

8. Facility Data Creation

   • The CampaignManager sends a request to the FacilityService to create facility data.

   • The FacilityService responds that the facilities have been created successfully.

9. Employee Data Creation

   • The CampaignManager sends a request to the HRMSService to create employee data.

   • The HRMSService responds that the employees have been created successfully.

10. Project Data Creation

    • The CampaignManager sends a request to the ProjectService to create project data with a parent-child relationship based on the project type and delivery configuration.

    • The ProjectService responds that the projects have been created successfully.

11. Project-Facility Mapping Creation

    • The CampaignManager sends a request to the ProjectService to create mappings between projects and facilities.

    • The ProjectService responds that the project-facility mappings have been created successfully.

12. Project-Staff Mapping Creation

    • The CampaignManager sends a request to the ProjectService to create mappings between projects and staff.

    • The ProjectService responds that the project-staff mappings have been created successfully.

13. Update Campaign Creation Status

    • The CampaignManager updates the Database with the status of the campaign creation process.

Error Handling

• Validation Failure: If any validation step fails (e.g., invalid data in resource files), the CampaignManager sends an error message to the Client, indicating the failure. The user is required to fix the issues and resubmit the request.

• Service Errors: If any of the service interactions (e.g., BoundaryService, MDMSService) return an error, the CampaignManager stops the process and informs the Client of the failure, specifying the nature of the error.

• Database Errors: If the Database fails to update the campaign creation status, appropriate error handling and logging mechanisms should be triggered to handle the failure.

API Sequence Diagram

Conclusion

This document provides an overview of the create campaign flow, detailing each step and interaction between different services and the database. The flow ensures proper validation and creation of necessary resources, while error handling mechanisms provide robustness to the process.

Technical Documentation: Update Campaign Flow

Overview

This document outlines the flow for updating an existing campaign using various services in the system. The process involves interactions between multiple services, including the Project Factory, Project Service, Boundary Service, Facility Service, HRMS Service, MDMS Service, FileStore Service, and the database. The update flow ensures that the required resources are validated and updated correctly before finalizing the campaign update.

Actors and Participants

• Client: The entity (user or system) initiating the campaign update request.

• CampaignManager (Project Factory): The main controller managing the campaign update process.

• ProjectService: Manages project-related operations, including data creation and mapping updates.

• BoundaryService: Handles fetching of boundary relationships based on hierarchy types.

• FacilityService: Responsible for creating and updating facility-related data.

• HRMSService: Manages the creation and updating of employee data.

• MDMSService: Provides master data such as project types.

• FileStoreService: Manages the storage and retrieval of resource files.

• Database: Stores campaign-related data and status updates.

Sequence Flow

1. Initiate Campaign Update

   • The Client sends an update request to the CampaignManager with all required and valid resources.

2. Fetch Boundary Relationship

   • The CampaignManager requests the BoundaryService to fetch the boundary relationship based on the hierarchy type.

   • The BoundaryService responds with the relevant boundary relationship data.

3. Fetch Project Type Master

   • The CampaignManager requests the MDMSService to fetch the project type master data.

   • The MDMSService responds with the project type master.

4. Validate Resource Files

   • The CampaignManager queries the Database to check if the input resource files have already been validated.

   • The Database responds with the validation status of the files.

5. Fetch Parent Campaign Object

   • The CampaignManager queries the Database to fetch the parent campaign object based on the parentCampaignId.

   • The Database responds with the parent campaign object.

6. Validation Check

   • The CampaignManager checks if all file templates and data are validated:

     • If validation is successful, the CampaignManager informs the Client that the campaign update process has started, and the user needs to track the status using an ID.

     • If validation fails, the CampaignManager returns an error message, indicating which validation failed, and the user must resubmit the request.

7. Deactivate Parent Campaign

   • The CampaignManager updates the Database to mark the parent campaign object as inactive.

8. Retrieve Resource Files

   • The CampaignManager interacts with the FileStoreService to search for the resource based on the filestoreid.

   • The FileStoreService responds with the valid resource files.

9. Process Resource Data

   • The CampaignManager processes the retrieved sheets to identify any resource data that needs to be created.

10. Facility Data Creation

    • The CampaignManager sends a request to the FacilityService to create facility data.

    • The FacilityService responds that the facilities have been created successfully.

11. Employee Data Creation

    • The CampaignManager sends a request to the HRMSService to create employee data.

    • The HRMSService responds that the employees have been created successfully.

12. Search for Parent Project

    • The CampaignManager sends a request to the ProjectService to search for the previous or parent project.

    • The ProjectService responds with the project object.

13. Create New Projects for Added Boundaries

    • The CampaignManager copies the project content from the parent project while creating new projects for newly added boundaries.

    • The CampaignManager sends a request to the ProjectService to create project data for newly added boundary data and map it according to its parent.

    • The ProjectService responds that the projects have been created successfully.

14. Update Facility Data

    • The CampaignManager identifies what exact data needs to be updated for facilities.

    • The CampaignManager sends a request to the ProjectService to create project-facility mapping if any new facilities are added.

    • The ProjectService responds that the project-facility mapping has been created successfully.

    • The CampaignManager sends a request to the ProjectService to search for the previous project-facility mapping.

    • The ProjectService responds with the project-facility mapping data.

    • The CampaignManager sends a request to the ProjectService to update the project-facility mapping.

    • The ProjectService responds that the project-facility mapping has been updated successfully.

15. Update Staff Data

    • The CampaignManager identifies what exact data needs to be updated for staff.

    • The CampaignManager sends a request to the ProjectService to create project-staff mapping if any new staff are added.

    • The ProjectService responds that the project-staff mapping has been created successfully.

    • The CampaignManager sends a request to the ProjectService to search for the previous project-staff mapping.

    • The ProjectService responds with the project-staff mapping data.

    • The CampaignManager sends a request to the ProjectService to update the project-staff mapping.

    • The ProjectService responds that the project-staff mapping has been updated successfully.

16. Update Campaign Status

    • The CampaignManager updates the Database with the campaign update status.

Error Handling

• Validation Failure: If any validation step fails (e.g., invalid data in resource files or missing required resources), the CampaignManager sends an error message to the Client, indicating the failure. The user is required to fix the issues and resubmit the request.

• Service Errors: If any of the service interactions (e.g., BoundaryService, MDMSService, ProjectService) return an error, the CampaignManager stops the process and informs the Client of the failure, specifying the nature of the error.

• Database Errors: If the Database fails to update the campaign update status or mark the parent campaign as inactive, appropriate error handling and logging mechanisms should be triggered to handle the failure.

API Sequence Diagram

Conclusion

This document provides a comprehensive overview of the update campaign flow, detailing each step and interaction between different services and the database. The flow ensures proper validation and updating of necessary resources, while error handling mechanisms provide robustness to the process.

Overview

This service is used to create a Project (Campaign), create required resource data, and create a mapping relation between them based on the boundary data.

Dependencies

1. Project

2. Facility

3. Product

4. HRMS

5. MDMS

6. Boundary

7. Localisation

8. Access Control

9. IdGen

10. Individual

11. User

API Specification

Base Path: /project-factory/

API Contract Link

Data Model

DB Schema Diagram

Table Details

```dbml
Table eg_cm_campaign_details {
  id varchar(128) [primary key ,not null, unique]
  tenantid varchar(64) [not null ,note: 'Tenant identifier']
  campaignname varchar(250) [not null ,note: 'Name of the campaign']
  projecttype varchar(128) [not null ,note: 'Type of project']
  startdate bigint [note: 'Start date in epoch']
  enddate bigint [note: 'End date in epoch']
  campaigndetails jsonb [note: 'Campaign specific details']
  status varchar(128) [not null ,note: 'Status of the campaign']
  parentid varchar(128) [note: 'refering to the previous campaign id']
  action varchar(64) [not null ,note: 'Action type']
  campaignnumber varchar(128) [not null ,note: 'Campaign number']
  hierarchytype varchar(128) [not null ,note: 'Hierarchy type']
  boundarycode varchar(64) [note: 'Boundary code']
  projectid varchar(128) [note: 'Project identifier']
  createdby varchar(128) [not null ,note: 'Created by user ID']
  lastmodifiedby varchar(128) [note: 'Last modified by user ID']
  createdtime bigint [note: 'Creation timestamp']
  lastmodifiedtime bigint [note: 'Last modification timestamp']
  additionaldetails jsonb [note: 'Additional details']
}
```

```dbml
Table eg_cm_campaign_process {
  id varchar(128) [primary key, not null, unique]
  campaignid varchar(128) [not null,note: 'Foreign key to eg_cm_campaign_details.id']
  type varchar(128) [not null,note: 'Type of campaign process']
  status varchar(128) [not null,note: 'Status of the campaign process']
  details jsonb [note: 'Detailed information of the process']
  createdtime bigint [note: 'Creation timestamp']
  lastmodifiedtime bigint [note: 'Last modification timestamp']
  additionaldetails jsonb [note: 'Additional details']
}
Ref: eg_cm_campaign_process.campaignid > eg_cm_campaign_details.id // many-to-one

```

```dbml
Table eg_cm_generated_resource_details {
  id varchar(128) [primary key,not null, unique]
  filestoreid varchar(128) [note: 'File store ID']
  status varchar(128) [not null,note: 'Status of the resource']
  type varchar(128) [not null,note: 'Type of resource']
  tenantid varchar(128) [not null,note: 'Tenant identifier']
  count bigint [note: 'Count of resources']
  createdby varchar(128) [note: 'Created by user ID']
  createdtime bigint [note: 'Creation timestamp']
  lastmodifiedby varchar(128) [note: 'Last modified by user ID']
  lastmodifiedtime bigint [note: 'Last modification timestamp']
  additionaldetails jsonb [note: 'Additional details']
  hierarchytype varchar(128) [not null,note: 'Hierarchy type']
  campaignid varchar(128) [note: 'Foreign key to eg_cm_campaign_details.id']
}
Ref: eg_cm_generated_resource_details.campaignid > eg_cm_campaign_details.id // many-to-one

```

```dbml
Table eg_cm_resource_activity {
  id varchar(128) [primary key,not null, unique]
  retrycount int [note: 'Number of retry attempts']
  type varchar(64) [not null,note: 'Type of activity']
  url varchar(128) [not null,note: 'URL for the activity']
  requestpayload jsonb [note: 'Request payload']
  tenantid varchar(128) [not null,note: 'Tenant identifier']
  responsepayload jsonb [note: 'Response payload']
  status bigint [note: 'Status code']
  createdby varchar(128) [note: 'Created by user ID']
  createdtime bigint [note: 'Creation timestamp']
  lastmodifiedby varchar(128) [note: 'Last modified by user ID']
  lastmodifiedtime bigint [note: 'Last modification timestamp']
  additionaldetails jsonb [note: 'Additional details']
  resourcedetailsid varchar(128) [not null,note: 'Foreign key to eg_cm_resource_details.id']
}
Ref: eg_cm_resource_activity.resourcedetailsid > eg_cm_resource_details.id // many-to-one

```

```dbml


Table eg_cm_resource_details {
  id varchar(128) [primary key,not null, unique]
  status varchar(128) [note: 'Status of the resource']
  tenantid varchar(128) [not null,note: 'Tenant identifier']
  filestoreid varchar(128) [note: 'File store ID']
  processedfilestoreid varchar(128) [note: 'Processed file store ID']
  action varchar(128) [not null,note: 'Action type']
  type varchar(64) [not null,note: 'Type of resource']
  createdby varchar(128) [note: 'Created by user ID']
  createdtime bigint [note: 'Creation timestamp']
  lastmodifiedby varchar(128) [note: 'Last modified by user ID']
  lastmodifiedtime bigint [note: 'Last modification timestamp']
  additionaldetails jsonb [note: 'Additional details']
  campaignid varchar(128) [note: 'Foreign key to eg_cm_campaign_details.id']
}

Ref: eg_cm_resource_details.campaignid > eg_cm_campaign_details.id // many-to-one


```

Web Sequence Diagrams

Data Create API

Generate Data API

Process Create Campaign API

Process Track api