> For the complete documentation index, see [llms.txt](https://docs.digit.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digit.org/local-governance/v2.7/platform/configure-digit/configuring-digit-services/indexer-configuration.md). # Indexer Configuration ## Overview Indexer uses a config file per module to store all the configurations pertaining to that module. Indexer reads multiple such files at start-up to support indexing for all the configured modules. In config we define source and, destination elastic search index name, custom mappings for data transformation and mappings for data enrichment. ## Sample Configuration Below is the sample configuration for indexing TL application creation data into elastic search. ```javascript ServiceMaps: serviceName: Trade License version: 1.0.0 mappings: - topic: save-tl-tradelicense configKey: INDEX indexes: - name: tlindex-v1 type: licenses id: $.id, $.tenantId isBulk: true timeStampField: $.auditDetails.createdTime jsonPath: $.Licenses customJsonMapping: indexMapping: {"Data":{"tradelicense":{},"ward":{},"tenantData":{}, "history":{}}} fieldMapping: - inJsonPath: $ outJsonPath: $.Data.tradelicense externalUriMapping: - path: http://egov-location.egov:8080/egov-location/location/v11/boundarys/_search queryParam: hierarchyTypeCode=REVENUE,boundaryType=locality,codes=$.tradeLicenseDetail.address.locality.code,tenantId=$.tenantId apiRequest: {"RequestInfo":{"apiId":"org.egov.pt","ver":"1.0","ts":1502890899493,"action":"asd","did":"4354648646","key":"xyz","msgId":"654654","requesterId":"61","authToken":"d9994555-7656-4a67-ab3a-a952a0d4dfc8","userInfo":{"id":1,"uuid":"1fec8102-0e02-4d0a-b283-cd80d5dab067","type":"EMPLOYEE","tenantId":"pb.amritsar","roles":[{"name":"Employee","code":"EMPLOYEE","tenantId":"pb.amritsar"}]}}} uriResponseMapping: - inJsonPath: $.TenantBoundary[0].boundary[0] outJsonPath: $.Data.ward - path: http://egov-workflow-v2.egov:8080/egov-workflow-v2/egov-wf/process/_search queryParam: businessIds=$.applicationNumber,history=true,tenantId=$.tenantId apiRequest: {"RequestInfo":{"apiId":"org.egov.pt","ver":"1.0","ts":1502890899493,"action":"asd","did":"4354648646","key":"xyz","msgId":"654654","requesterId":"61","authToken":"d9994555-7656-4a67-ab3a-a952a0d4dfc8","userInfo":{"id":1,"uuid":"1fec8102-0e02-4d0a-b283-cd80d5dab067","type":"EMPLOYEE","tenantId":"pb.amritsar","roles":[{"name":"Employee","code":"EMPLOYEE","tenantId":"pb.amritsar"}]}}} uriResponseMapping: - inJsonPath: $.ProcessInstances outJsonPath: $.Data.history mdmsMapping: - path: http://egov-mdms-service.egov:8080/egov-mdms-service/v1/_search moduleName: tenant masterName: tenants tenantId: pb filter: "[?(@.code == $tenant)]" filterMapping: - variable: $tenant valueJsonpath: $.tenantId uriResponseMapping: - inJsonPath: $.MdmsRes.tenant.tenants outJsonPath: $.Data.tenantData ``` ## **Variable List** The table below lists the key configuration variables. | Variable Name | Description | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | serviceName | Name of the module to which this configuration belongs. | | summary | Summary of the module. | | version | Version of the configuration. | | mappings | List of definitions within the module. Every definition corresponds to one index requirement. Which means, every object received onto the kafka queue can be used to create multiple indexes, each of these indexes will need configuration, all such configurations belonging to one topic forms one entry in the mappings list. The keys listed henceforth together form one definition and multiple such definitions are part of this mappings key. | | topic | The topic on which the data is to be received to activate this particular configuration. | | configKey | Key to identify to what type of job is this config for. values: INDEX, REINDEX, LEGACYINDEX. INDEX: LiveIndex, REINDEX: Reindex, LEGACYINDEX: LegacyIndex. | | indexes | Key to configure multiple index configurations for the data received on a particular topic. Multiple indexes based on a different requirement can be created using the same object. | | name | Index name on the elastic search. (Index will be created if it doesn't exist with this name.) | | type | Document type within that index to which the index json has to go. (Elasticsearch uses the structure of index/type/docId to locate any file within index/type with id = docId) | | id | Takes comma-separated JsonPaths. The JSONPath is applied on the record received on the queue, the values hence obtained are appended and used as ID for the record. | | isBulk | Boolean key to identify whether the JSON received on the Queue is from a Bulk API. In simple words, whether the JSON contains a list at the top level. | | jsonPath | Key to be used in case of indexing a part of the input JSON and in case of indexing a custom json where the values for custom json are to be fetched from this part of the input. | | timeStampField | JSONPath of the field in the input which can be used to obtain the timestamp of the input. | | fieldsToBeMasked | A list of JSONPaths of the fields of the input to be masked in the index. | | customJsonMapping | Key to be used while building an entirely different object using the input JSON on the queue | | indexMapping | A skeleton/mapping of the JSON that is to be indexed. Note that, this JSON must always contain a key called "Data" at the top-level and the custom mapping begins within this key. This is only a convention to smoothen dashboarding on Kibana when data from multiple indexes have to be fetched for a single dashboard. | | fieldMapping | Contains a list of configurations. Each configuration contains keys to identify the field of the input JSON that has to be mapped to the fields of the index json which is mentioned in the key 'indexMapping' in the config. | | inJsonPath | JSONPath of the field from the input. | | outJsonPath | JSONPath of the field of the index json. | | externalUriMapping | Contains a list of configurations. Each configuration contains keys to identify the field of the input JSON that is to be enriched using APIs from the external services. The configuration for those APIs also is a part of this. | | path | URI of the API to be used. (it should be POST/\_search API.) | | queryParam | Configuration of the query params to be used for the API call. It is a comma-separated key-value pair, where the key is the parameter name as per the API contract and value is the JSONPath of the field to be equated against this parameter. | | apiRequest | Request Body of the API. (Since we only use \_search APIs, it should be only RequestInfo.) | | uriResponseMapping | Contains a list of configuration. Each configuration contains two keys: One is a JSONPath to identify the field from response, Second is also a JSONPath to map the response field to a field of the index json mentioned in the key 'indexMapping'. | | mdmsMapping | Contains a list of configurations. Each configuration contains keys to identify the field of the input JSON that is to be denormalized using APIs from the MDMS service. The configuration for those MDMS APIs also is a part of this. | | path | URI of the API to be used. (it should be POST/\_search API.) | | moduleName | Module Name from MDMS. | | masterName | Master Name from MDMS. | | tenantId | Tenant id to be used. | | filter | Filter to be applied to the data to be fetched. | | filterMapping | Maps the field of input json to variables in the filter | | variable | Variable in the filter | | valueJsonpath | JSONPath of the input to be mapped to the variable. | [![Creative Commons License](https://i.creativecommons.org/l/by/4.0/80x15.png)​](http://creativecommons.org/licenses/by/4.0/)All content on this page by [eGov Foundation](https://egov.org.in/) is licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.digit.org/local-governance/v2.7/platform/configure-digit/configuring-digit-services/indexer-configuration.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.