A gap in the data is defined as a single missing or series of consecutive missing interval meter readings from a periodic channel.
For example:
- a periodic channel A Periodic Channel which is recording data every half hour - If there are any half hour periods without a reading, then this would be deemed a gap
- a periodic channel A Periodic Channel which is recording data daily - If there are any days missing a reading, then this would be deemed a gap
Note, it is not limited to half-hourly only, if your data is received every 15 minutes, or 5 minutes, then the system will support this.
Adding a Data Gap Strategy
...
Adding a Monitoring Strategy
From the Strategy Configuration Tab, click 
Choose the Strategy Type from the dropdown
Click OK to take you to the Monitoring Strategy Editor
Monitoring Strategy Details
The top section of the Monitoring Strategy Editor is common to all the different monitoring strategy types that can be setup.
The details that need to be entered are as follows:
...
The field is disabled and displays the Strategy Type being created
...
Choose the importance of any notifications that are created against this strategy. Available options are:
- Low
- Medium
- High
...
Choose from:
Fixed - allows you to drag and drop individual Channels. Click here for more details
Flexible - allows you to configure the set up to automatically include Channels, see below for more details
...
Enter email address(s) to send a summary notification to.
More than one email address can be entered, and will need to be separated with a comma.
...
Choose from
- Daily
- Weekly
- Monthly
Note, there are more options depending on which of the above you have chosen, see below for more details on the configuration options.
...
It is possible to define which periodic channels the strategy should run for. This will help you ensure your strategies continue to run as your energy estate evolves and assets change/get added.
A monitoring strategy can have a "context" of either:
- Fixed - assign the channels you want the strategy to run for on a case by case basis. Doing it this way will only run for the channels that you have specifically assigned so it will not run against new periodic channels that you might add unless you also specifically assign them to the strategy. Click here for instructions on how to do this.
- Flexible - configure the attributes of the channel that the strategy should run for when setting up the strategy (e.g. run for all channels, run for channels of commodity type x etc.). Doing it this way will help make sure that the strategy will run for the assets you want it to as they evolve or new assets are added - you don't have to specify the periodic channel when you add new ones.
Setting up Flexible configuration:
Choose Flexible from the Context dropdown,
Click on 
to show the Flexible Context Configuration popup
Either:
Tick Yes to run for all Channels
Alternatively, leave as No and choose your own configuration using the dropdown options based on the attributes of the channel and their relates assets.
The available options are: Commodity, Data type, Unit, Site Property Keys, Meter Property Keys, Channel Property Keys.
For example, the above configuration would include all Electricity and Gas Channels that have a unit set to kWh. As new channels that match these parameters are added to your estate, the monitoring strategy would automatically include them when it runs.
Click Save to return to the Monitoring Strategy Editor to continue configuring the Strategy.
...
If you want to be notified of new issues by email, you can choose how often you should get an email and who the email should be sent to.
Tick 
Enter the email address(es) you want to send the notification to. Any email addresses entered here will receive a summary of all notifications raised across the entire estate. Note: for more than one email, add a comma between the addresses
...
Recurrence: Choose from Daily, Weekly or Monthly (each option then has further choices for the configuration to configure exactly when the email will be sent).
Emails will contain a summary of new notifications that have been raised since the last email was received. It will also contain a summary of all the outstanding notifications.
Data Gap Configuration
The Item Editor also contains a Data Gap Configuration section that allows you to set the specific configuration to use when finding and managing the gaps.
...
Choose the offset between the end date of the gap and the current date.
This is to enable a lag time between the date that is being checked and when the check is being performed to give the chance for metering systems to correct themselves.
For example, a metering system may be set up to send a daily drop of information, and then a weekly summary for the previous 7 days to correct any potential errors. In this case, the offset could be set to 7 days to allow the weekly summary upload the chance to correct the data before any problems are identified.
...
Define the combination of data values that should be considered missing data.
Options include:
- Zero - No Error
- Checks for recorded values of '0' AND a status of 'No Error'. This represents data that the source metering system has sent as an actual meter reading value of zero. This should be used with caution as could represent valid data - it is intended to check meters which never expect to receive readings with a value of zero based on their operation.
...
Choose from:
- Do Not Auto Fill
- An open Notification will be created which will be visible in the Notifications Tab for you to deal with and resolve manually (note gaps can be resolved in bulk)
- Auto Fill Recurring Gaps
- Sigma will automatically 're-fill' any data gaps that have previously been manually resolved by a User as soon as they are identified. A resolved Notification will be created to provide a full audit.
- This is to ensure data gaps aren't re-created if the source data is re-loaded with the same gaps that a User has already resolved
- Sigma will automatically 're-fill' any data gaps that have previously been manually resolved by a User as soon as they are identified. A resolved Notification will be created to provide a full audit.
- Auto Fill All Gaps
- Sigma will automatically fill all gaps that are identified. A resolved Notification will be created to provide a full audit.
| Note |
|---|
Please note, where the system is configured to fill all gaps automatically, if it finds a channel with more than 100 different gaps that need to be filled at the same time, then the system will not attempt to fill the gaps, but will instead raise the notification with a status of "New" so they can be manually reviewed and the necessary action taken. This is to highlight a potential fundamental problem with the ongoing recording of the metering data, or where a strategy may be incorrectly configured. It prevents the system from manipulating the data which cannot then be reversed. |
...
Specify the number of weeks that should be used to create a period average when automatically filling data gaps.
More information about how this feeds into the system processing that it supports when filling the data can be seen here.
...
The number of preceding months that Sigma will check for gaps.
For example, if you choose 12, Sigma will check the previous 12 months for any gaps
Click OK to save the Monitoring Strategy.
Data Gap Notifications
| Table of Contents |
|---|
Data Gap Notifications
If data gaps are found for one of the strategies, then notifications will be raised that are visible in the Notification Tab.
Where If the strategy was configured to auto-fill all gaps for you, then the notification will still be raised in a resolved state, but marked as resolved, to provide visibility of what the system has done, providing a full audit.
| Excerpt | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Anchor |
|
| Info |
|---|
If you select notifications which include types other than gaps, then a prompt will be displayed to confirm which strategy type you want to work with as the options available are contextual to the type of data issue being worked.
|
- The Resolve Notifications popup will be displayed, showing a list of the Notifications being resolved
The graph at the top shows the periodic data for the channel linked to the notification selected. The red area shows where the Gap is.
- Drag the mouse around the area to zoom in
- Click
to reset the zoom back to the default
Actions - Data Gap
Update Period Average Setting
You can change the number of Weeks used as a basis for calculating the data to use when filling the gap, select all the Notification records it should apply to then click Update. This will update the period to use when calculating the averaged data to fill the gap.
| Tip | ||
|---|---|---|
| ||
Use the "Select All" toggle at the bottom of the notifications panel if you want to quickly select all notifications. |
Preview
If you want to see what the filled data will look like, click 
first.
This will show you what the filled gap would look like for the Notification that has been selected in the list. The filled period is highlighted in green.
- Drag the mouse around the area to zoom in
Fill
- Click
to fill the gaps.
The Resolve Notifications pop up appears:
A Note can be optionally entered if required and will be stored against the Notification.
- Click OK
The following confirmation popup will be displayed:
Click OK.
Cancel
- Click
to cancel without making any changes. You will be returned to the list of all Notifications
| Excerpt | ||
|---|---|---|
| ||
Adding Notes to NotificationsThis basic workflow allows you to capture information and update the status of the Notifications, as per your own business processes. This is expected to be used to help you keep track of any investigative and follow-up actions which are been taken as a result of the Data Monitoring Notifications. For example:
Highlight the Notification(s) you would like to update. Notifications can be selected in bulk, even across different strategy types.
The Status will default to In Progress and is not editable - the Resolve and Ignore features will update the status. If you want to update the Sub-status, select the required option from the drop down list. The default options are:
If you want to update the Note for the Notification(s), then Tick Update Note and enter the note in the text box.
Click OK to Save. Ignoring NotificationsThis will update the status of the Notification to "Ignored" and will also prevent the same notification (for the same period and Impact) from being raised by the system again.
If you want to update the Note for the Notification(s), then Tick Update Note and enter the note in the text box.
Editing Items
You can view the Channel / amend the interval reading Data associated to the notification, if required. Please note, this option is only available if a single Notification is selected.
Editing Monitoring StrategiesHighlight the Notification for which you would like to edit the underlying monitoring strategy configuration.
Exporting to Excel
|
