Error Handling

Error Handling Mechanism & Processed Error Sheet Generation

Overview

The CRS Data Loader implements a structured, multi-layered error-handling mechanism to ensure that only clean, validated, and schema-compliant data is inserted into the system. When errors occur at any phase of data upload, the Notebook automatically generates a Processed Error Excel Sheet that contains detailed status information for every row.

This enables users to identify issues quickly, correct the data, and re-upload without repeating successful entries.

---

1. When Errors Occur

Errors can occur during any of the three validation layers:

1. Excel-Level Validation

• Missing mandatory fields

• Invalid pattern (regex failure)

• Incorrect data type

• Duplicate values

• Invalid URL format

• Unmapped dropdown values

2. MDMS Schema Validation

• Field does not match schema pattern

• Unsupported value in enum fields

• Cross-sheet mapping error

• Missing parent reference

• Invalid tenant code

• Duplicate key in MDMS data

3. API-Level Validation

• Duplicate boundaries

• Invalid parent codes

• Department or complaint type already exists

• Invalid hierarchy for boundary

• SLA value is out of the acceptable range

• Non-existent department referenced in complaint

When any of these validations fail, the system does not stop the entire process. Instead, it records the status of each row and produces a processed error file.

---

2. Processed Error File: What It Contains

When errors are detected, the Notebook generates a Processed Error Excel File.This file contains every row from the original upload, along with system-generated status columns.

Standard Columns Added to the Processed Error File

Column Name	Description
_STATUS	Indicates whether the row succeeded or failed (SUCCESS / FAILED)
_STATUS_CODE	Numeric code representing the error or success (e.g., 201, 400, 409)
_ERROR_MESSAGE	Human-readable error message explaining why the row failed

Example Output

Department Name	Designation Name	_STATUS	_STATUS_CODE	_ERROR_MESSAGE
WATER DEPARTMENT	engineer	SUCCESS	201	Created successfully
ELECTRIC DEPT	LME	FAILED	400	Department name does not match schema

---

3. Error Types and How They Are Generated

A. Excel-Level Errors

These are validated before sending data to the backend.

Example messages:

• “Mandatory column ‘Tenant Code’ is missing”

• “Invalid URL format”

• “Resolution Time must be numeric”

B. Schema-Level Errors

Generated based on MDMS schema rules.

Example messages:

• “Department does not exist for this tenant”

• “Field ‘tenantType’ must be one of: State, City”

• “Regex validation failed for ‘boundary code’”

C. API-Level Errors

Generated by backend services.

Example messages:

• “Parent boundary code not found”

• “Record already exists”

• “Hierarchy definition missing for tenant”

• “Invalid value for SLA hours”

---

4. How the Notebook Generates the Error Sheet

The process followed internally is:

1. Read the uploaded Excel file.

2. Validate each row (Excel → Schema → API).

3. Maintain an internal list of results for each row.

4. Write the results back into the same row order.

5. Add _STATUS, _STATUS_CODE, and _ERROR_MESSAGE.

The file is automatically shown as a Download Link in the Notebook output.

---

5. User Actions After an Error File Is Generated

The steps for the user are:

Step 1 – Download the Processed File

Click the download link shown in the Notebook output.

Step 2 – Open and Review All Failed Rows

Look at _ERROR_MESSAGE and correct only failed records

These columns must not exist in the re-upload file.

Step 3 – Save Corrected File

Save it as:

• tenant_master_corrected.xlsx, or

• employee_master_corrected.xlsx, etc.

• common_master_corrected.xlsx,etc.

Step 4 – Re-upload Only the Corrected File

Upload the corrected file in the same phase cell.

Step 5 – The System Skips Previously Successful Rows

Only rows marked FAILED are processed again.

This incremental mechanism ensures:

• No duplication

• No reprocessing

• Faster onboarding workflows

---

6. Advantages of the Error Handling Mechanism

• Ensures clean and stable master data

• Allows modular re-upload instead of restarting the full upload

• Captures detailed row-level errors

• Prevents schema violations from entering the system

• Makes the onboarding workflow efficient and repeatable

• Matches the DIGIT platform’s standard boundary and MDMS ingestion patterns