Keep Analytics Data Updated
Business Problem
Users wants to see near real-time or latest data in analytic Reports and Dashboards. You want to avoid costly manual data imports and instead automate the process of keeping reports and dashboards up to date.
Components
Requirements
- Birst Admin access (for Orchestration)
- If you are using an Infor Cloudsuite, you can find the corresponding IFS roles in the CloudSuite Analytics documentation. For example, in M3 Analytics you need the ‘M3A Administration‘ role for Admin access.
Tutorial
This can be achieved by scheduling periodic updates. Let’s assume that we are extracting data tables from the Infor data lake to be used in Birst reports and dashboards. We have completed creating the dashboards and we want to update the data in Birst, automatically on a scheduled basis.
For this purpose, we make use of the ‘Orchestration Workflow’, a simple process with a few clicks.
We make use of ‘Extract Groups’ and ‘Publishing Groups’ to process batches of data
1. Creating ‘Extract Groups’
Extract Groups can be created in order to group the tables that are part of the Orchestration Workflow. Hence, if we want to include only 5 tables in our process, out of the 20 tables included in our connection to the data lake, we use Extract Groups.
Extract Groups are created in ‘Modeler Connect’ as shown below: we create an ‘Extraction Group’, name it, select the Data ‘Connections’ and then the individual tables within them.
2. Creating ‘Publishing Groups’
A similar concept is used for Publishing Groups. It helps us to select only those tables which we want to publish as part of ‘Orchestration Workflow’.
Publishing Groups are created in ‘Modeler Prepare’ as shown below: we create a ‘Publishing Group’, name it and then select the Staging Tables to be put in.
3. Creating ‘Orchestration Workflow’
We create an ‘Orchestration Workflow’, which can be run manually or automatically. For our business problem, we will run the workflow automatically by scheduling it.
While configuring ‘Orchestration Workflow’, we imagine the steps that we usually do when updating the data. First we create a Workflow and name it.
We add steps for extracting the data from the data lake using ‘Extract Groups’, and then publishing the data using ‘Publishing Groups’ or all the data tables.
4. Scheduling ‘Orchestration Workflow’
Finally, we schedule the workflow as per requirements. An important factor to consider is to allow sufficient time window for the workflow to complete before the user checks the new data. The example below shows that the workflow will run on 2nd day of every month at 12:00 AM EST.
Several schedules can be added as per requirements.
