DSS has two sides to it. One is the process in which the data is pooled into ElasticSearch and the other is the way it is fetched, aggregated, computed, transformed and sent across.
As this revolves around various data sets, there is a need to make this configurable. This ensures that the process can be configured easily in case a new scenario is introduced. This document explains the steps on how to define the configurations for the analytics side of DSS for W&S.
Technical Details
Analytics: Microservice responsible for building, fetching, aggregating and computing the data on ElasticSearch into a consumable data response. This is later used for visualizations and graphical representations.
Analytics Configurations: Analytics contains multiple configurations. we need to add the changes related to FSM in this dashboard-analytics.
Configuration file link:
Below is a list of configurations that need to be changed to run FSM successfully.
Chart API Configuration
Master Dashboard Configuration
Role Dashboard Mappings Configuration
Chart API Configuration: Each Visualization has its own properties. Each Visualization comes from different data sources (Sometimes it is a combination of different data sources). In order to configure each visualization and its properties, we have a Chart API Configuration Document.
In this, Visualization Code, which happens to be the key, will be having its properties configured as a part of the configuration and are easily changeable.
ChartApiConfiguration.json sample data for the W&S is given below:
Master Dashboard Configuration: Master dashboard configuration is the main configuration which defines as which are the dashboards which are to be painted on the screen.
It includes all the visualizations, their groups, the charts which come within them and even their dimensions as what should be their height and width.
Role Dashboard Mappings Configuration: The master dashboard configuration which was explained earlier holds the list of dashboards which are available.
Given the instance where role-action mapping is not maintained in the application service, this configuration will act as a role-dashboard mapping configuration. In this, each role is mapped against the dashboard that they are authorized to see. This was used earlier when the role-action mapping of eGov was not integrated.
Later, when the role-action mapping started controlling the dashboards to be seen on the client side, this configuration was just used to enable the dashboards for viewing.
{
"_comment": "Holds mapping for each role with and its associated dashboards",
"roles" : [
{
"_comment":"This role is super role which can access all the available dashboards: [other/new roles are suppose to be added]",
"roleId": 6,
"roleName" : "Admin",
"isSuper" : "",
"orgId": "",
"dashboards": [
{
"name": "National Urban Relatime Dashboard",
"id": "NURT_DASHBOARD"
}
]
}
]
}
MDMS configuration to be added: common-masters/uiCommonConstants.json
National DSS consists of the following charts in National Urban Real-time Dashboard:
Overview Card
Onboarded States
Under Implementation
Live States
Live ULBs
Total Collections
Target Achievement
Total Applications
Total Citizens
SLA Achievement
Project Status
Onboarded ULBs vs Live ULBs
Property Tax
Total Collection
Properties Assessed
Trade License
Total Collection
Total Applications
Municipal Grievances
Total Grievances
SLA Achievement
Water & Sewerage
Total Collection
Total Connections
Building Permissions
Total Collection
Total Permits Issued
Fire NOC
Total Collection
Total Fire NOCs Issued
User Charges
Total Collection
Total Receipts
Overview Card: The overview graph contains multiple data information as below in the selected time period.
Under Implementation: This represents the No of states in the nation which are in the status of under implementation for DIGIT.
Live States: This represents the number of states across the nation which has DIGIT as Live.
Live ULBs: This represents the Total No of ULBs Live across states within the nation which has DIGIT as Live.
Target Achievement: This represents the target achieved in percentage. This is calculated by the formula - (Total Collection/Target Collection)*100%, across the modules and States.
Total Applications: This represents the total number of applications across the modules and states.
Total Citizens: This represents the total number of citizens registered across modules and states.
SLA Achievement: This represents the total number of citizens registered across the modules and States.
Project Status: This Graph Represent the states int he nation with their status of the DIGIT implementation into 4 different states Live, Under Implementation and Onboarded and None
Onboarded Vs Live ULBs - This Graph represents the Onboarded ULBs count vs Live ULBs count month wise for a given period.
DrilDown of the Project Status for a given State Ex: Punjab
Project Status shows the list of the Live ULBs module wise for a given state. Bar chart shows the onboarded vs live ULBs count month wise for a given state.
All the modules specific metrics discussed below are queried from the data in their respective National Dashboard Elastic Search index.
Property Tax
Total Collection - This metric represents the Total Collection for the Property Tax Service across the ULBs and States with in the nation.
Properties Assessed - This metric represents the Total No of Properties Assessed by Property Tax Service across the ULBs and States with in the nation.
Trade License
Total Collection - This metric represents the Total Collection for the Trade License Service across the ULBs and States with in the nation.
Total Applications - This metric represents the Total No of Trade Licenses processed by Trade License Service across the ULBs and States with in the nation.
Municipal Grievances
Total Grievances - This metric represents the Total No of Grievances created across the ULBs across the states in the Nation
SLA Achievement - This metric represents the Percentage of the Grievances Addressed/Solved with in the Grievances created across the ULBs across the states in the Nation.
Water & Sewerage
Total Collection - This metric represents the Total Collection for the Water & Sewerage Service across the ULBs and States with in the nation.
Total Connections - This metric represents the Total No of Connections issued across the Water & Sewerage across the ULBs and states with in the Nation.
Building Permissions
Total Collection - This metric represents the Total Collection for the Building Permissions Service across the ULBs and States with in the nation
Total Permits Issued - This metric represents the Total No of permission issued across the ULBs and states with in the Nation
Fire NOC
Total Collection - This metric represents the Total Collection for the Fire NOC Service across the ULBs and States with in the nation
Total Fire NOCs Issued - This metric represents the Total No of Fire NOCs issued across the ULBs and states with in the Nation
User Charges
Total Collection - This metric represents the Total Collection for the Collection Service across the ULBs and States with in the nation
Total Receipts - This metric represents the Total No of Receipts issued across the ULBs and states with in the Nation
Newly introduced property:
isRoundOff: This property is introduced to round off the decimal values. Ex: if the value is 25.43 by using isRoundOff property in configuration we will get it as 25. if value is 22.56 round of value will be 23.
This can be used for insights configuration as well for overview graph.
Common Properties available:
Key(eg: fsmTotalrequest) : This is the Visualization Code. This key will be referred to in further visualization configurations.
This is the key which will be used by the client application to indicate which visualization is needed for display.
chartName : The name of the Chart which has to be used as a label on the Dashboard. The name of the Chart will be a detailed name.
In this configuration, the Name of the Chart will be the code of Localization which will be used by Client Side.
queries : Some visualizations are derived from a specific data source. While some others are derived from different data sources and are combined together to get a meaningful representation.
The queries of aggregation which are to be used to fetch out the right data in the right aggregated format are configured here.
queries.module : The module / domain level, on which the query should be applied on. Water and sewerage is W&S.
queries.indexName : The name of the index upon which the query has to be executed is configured here.
queries.aggrQuery : The aggregation query in itself is added here. Based on the Module and the Index name specified, this query is attached to the filter part of the complete search request and then executed against that index
queries.requestQueryMap : Client Request would carry certain fields which are to be filtered. The parameters specified in the Client Request are different from the parameters in each of these indexed documents.
In order to map the parameters of the request to the parameters of the ElasticSearch Document, this mapping is maintained.
queries.dateRefField : Each of these modules have separate indexes. And all of them have their own date fields.
When there is a date filter applied against these visualizations, each of them has to apply it against their own date reference fields.
In order to maintain what is the date field in which index, we have this configured in this configuration parameter.
chartType : As there are different types of visualizations, this field defines what is the type of chart / visualization that this data should be used to represent.
Chart types available are:
metric - this represents the aggregated amount/value for records filter by the aggregate es query
pie - this represents the aggregated data on grouping. This is can be used to represent any line graph, bar graph, pie chart or donuts
line - this graph/chart is data representation on date histograms or date groupings
perform - this chart represents groping data as performance wise.
table - represents a form of plots and value with headers as grouped on and list of its key, values pairs.
xtable - represents a advanced feature of table, it has addition capabilities for dynamic adding header values.
valueType : In any case of data, the values which are sent to plot, might be a percentage, sometimes an amount and sometimes it is just a count.
In order to represent them and differentiate the numbers from amount from percentage, this field is used to indicate the type of value that this Visualization will be sending.
action : Some of the visualizations are not just aggregation on data source. There might be some cases where we have to do a post aggregation computation.
For Example, in the case of Top 3 Performing ULBs, the Target and Total Collection is obtained and then the percentage is calculated. In these kinds of cases, what is the action that has to be performed on that data obtained is defined in this parameter.
documentType : The type of document upon which the query has to be executed is defined here.
drillChart : If there is a drill down on the visualization, then the code of the Drill Down Visualization is added here. This will be used by Client Service to manage drill downs.
aggregationPaths : All the queries will be having Aggregation names in it. In order to fetch the value out of each Aggregation Responses, the name of the aggregation in the query will be an easy bet. These aggregation paths will have the names of Aggregation in it.
insights : It is to show the data with the comparison of last year with arrow symbols, it will show the data in how much % is increased or decreased.
_comment : In order to display information on the “i” symbol of each visualization, Visualization Information is maintained in this field.
Index Mapping of National Urban Relatime Dashboard:
The index that contains data for National Urban Relatime Dashboard is- common-national-dashboard
The following is the table that describes the properties mentioned above:
Attributes
Definition
Breakup
ulb
Can be a dummy value as the common national dashboard data is not at ulb level
Nil
state
The ULB name for which the data is ingested
Nil
ward
Can be a dummy value as the common national dashboard data is not at ward level
Nil
region
Can be a dummy value as the common national dashboard data is not at region level
Nil
status
The status of the DIGIT implementation like Live/UnderImplementation/OnBoarded
Nil
onboardedUlbsCount
The count of onboardeUlbs for a given state till that date
Nil
totalCitizensCount
The count of the citizen registered for a given state till that date
Nil
slaAchievement
The slaAchievment for a given state till that date
Nil
totalLiveUlbsCount
The count of lvieUlbs for a given state till that date
Nil
totalUlbCount
The count of all Ulbs for a given state till that date
liveUlbsCount
The count of Live Ulbs for a given state till that date
Breakdown by the serviceModuleCode
National urban real-time dashboard-national DSS consists of multiple graphs which represent the data of all modules aggregated together. Each graph has its own configuration which will describe the chart and its type.
Onboarded States:This represents the No of states in the nation which are in the status of onboarded for DIGIT.
Postman collection for National Urban Relatime Dashboard:
