Data update status
- James McLeod
Introduction
We’ve enhanced our data curation tools with the ability to sort datasets by date last updated to identify stale datasets and to trigger in app notifications when datasets haven’t been updated in an expected number of days
This is new functionality we’re keen to see how its used and what you would like to see added or changed, please reach out with comments
Setting the threshold and trigger notifications is intended for use with real time datasets
Any change in dataset including meta data will be treated as an update
Sort by Date last updated
In the datasets panel “Updated” is a column at the right, this can be clicked on to sort all datasets by newest or oldest update.
Notification if data not updated
For real time datasets that are expected to update with a regular frequency a threshold can be added which will trigger a notification if data hasn’t been updated in that expected timeframe.
The notification will show for everyone in the organisation
To clear a notification the dataset must be updated, this could include a change of meta data
To remove a dataset from generating notifications in this way update the Frequency to 0
Step 1
To set an update frequency to trigger notification a button is available in Next steps, “Update frequency”. Clicking this button will bring up a modal to enter a number. The number represents the days between updates.
For example, if you enter the number 1 you expect daily updates and after a day if there has been no update the notification will trigger. Similarly if you expect an update Weekly/Monthly/Quarterly/Annually then add the respective numbers 7/30/91/365.
If the data update is irregular, you can still use this where you can choose a number you expect as a maximum time between updates, for example if a provider updates a dataset weekly to monthly then input a value of monthly (30) to trigger a notification when the typical maximum time is exceeded.
Only numbers should be entered and if 0 is entered you will disable notifications for that dataset.
Step 2
When the update frequency is set the button will disappear but the data is captured in the card under “Set Update Threshold” and can be edited.
If you want to stop a dataset generating notifications, change its “Set Update Threshold” to 0
Step 3
When a threshold is set for any dataset if any threshold is exceeded the notification icon will shake and turn orange
Step 4
Clicking on notifications we have a new table that will list datasets where the threshold has been exceeded. Update the dataset to reset the notification.
FAQs:
I do not see the Update frequency button: The button will only show if you are on a real time dataset.
I have an old dataset that we stopped updating, how do I turn this off: Change the “Set Update Threshold” to 0.
We need to pause updates for a dataset how do I stop the notifications: We would recommend extending the update frequency to cover the pause or stopping notifications temporarily, just remember to reset the threshold when updates resume.
I can see the option with non-real time datasets, can I use it: Its not intended for static file storage while you can set a threshold its likely the notification would trigger and you would be constantly suppressing. If you can think of a use case where you would want to use it please let us know.
I think this would be more useful if we could… Please let us know we’re eager to discuss improvements to make your data journey easier.