A gap in the data is defined as a single missing or series of consecutive missing interval meter readings from a periodic channel.

For example:

a periodic channel which is recording data every half hour - If there are any half hour periods without a reading, then this would be deemed a gap
a periodic channel which is recording data daily - If there are any days missing a reading, then this would be deemed a gap

Note, it is not limited to half-hourly only, if your data is received every 15 minutes, or 5 minutes, then the system will support this.

Adding a Data Gap Strategy

Adding a Monitoring Strategy

From the Strategy Configuration Tab, click

Choose the Strategy Type from the dropdown

Click OK to take you to the Monitoring Strategy Editor

Monitoring Strategy Details

The top section of the Monitoring Strategy Editor is common to all the different monitoring strategy types that can be setup.

The details that need to be entered are as follows:

Field	Default	Details
Type	N/A	The field is disabled and displays the Strategy Type being created
Name	Empty	Enter a name - mandatory
Description	Empty	Enter a description - mandatory
Priority	Low	Choose the importance of any notifications that are created against this strategy. Available options are: Low Medium High
Context	Fixed	Choose from: Fixed - allows you to drag and drop individual Channels. Click here for more details Flexible - allows you to configure the set up to automatically include Channels, see below for more details
Status	Off	Choose On or Off to determine whether the system performs the checks
Email	Empty	Tick if you want Sigma to send summary notifications that are raised on a regular basis.
To addresses	Empty	Enter email address(s) to send a summary notification to. More than one email address can be entered, and will need to be separated with a comma.
Send to Site Contact	Empty	Tick if you want to send an email to the Site Contact (entered in the Site Editor) with a summary of notifications that relate to the assets on their site only.
Recurrence	Monthly	Choose from Daily Weekly Monthly Note, there are more options depending on which of the above you have chosen, see below for more details on the configuration options.

Channel Context Configuration

It is possible to define which periodic channels the strategy should run for. This will help you ensure your strategies continue to run as your energy estate evolves and assets change/get added.
A monitoring strategy can have a "context" of either:

Fixed - assign the channels you want the strategy to run for on a case by case basis. Doing it this way will only run for the channels that you have specifically assigned so it will not run against new periodic channels that you might add unless you also specifically assign them to the strategy. Click here for instructions on how to do this.
Flexible - configure the attributes of the channel that the strategy should run for when setting up the strategy (e.g. run for all channels, run for channels of commodity type x etc.). Doing it this way will help make sure that the strategy will run for the assets you want it to as they evolve or new assets are added - you don't have to specify the periodic channel when you add new ones.

Setting up Flexible configuration:

Choose Flexible from the Context dropdown,

Click on to show the Flexible Context Configuration popup

Either:

Tick Yes to run for all Channels

Alternatively, leave as No and choose your own configuration using the dropdown options based on the attributes of the channel and their relates assets.

The available options are: Commodity, Data type, Unit, Site Property Keys, Meter Property Keys, Channel Property Keys.

For example, the above configuration would include all Electricity and Gas Channels that have a unit set to kWh. As new channels that match these parameters are added to your estate, the monitoring strategy would automatically include them when it runs.

Click Save to return to the Monitoring Strategy Editor to continue configuring the Strategy.

Email Options

If you want to be notified of new issues by email, you can choose how often you should get an email and who the email should be sent to.

Tick

Enter the email address(es) you want to send the notification to. Any email addresses entered here will receive a summary of all notifications raised across the entire estate. Note: for more than one email, add a comma between the addresses

Tick if you want to send an email to the Site Contact (entered in the Location tab in the Site Editor). If this option is selected, the site contact will receive an email containing a summary of notifications only for the sites that they are responsible for.

Recurrence: Choose from Daily, Weekly or Monthly (each option then has further choices for the configuration to configure exactly when the email will be sent).

Emails will contain a summary of new notifications that have been raised since the last email was received. It will also contain a summary of all the outstanding notifications.

Data Gap Configuration

The Item Editor also contains a Data Gap Configuration section that allows you to set the specific configuration to use when finding and managing the gaps.

Field	Default	Description
Offset (Days)	1	Choose the offset between the end date of the gap and the current date. This is to enable a lag time between the date that is being checked and when the check is being performed to give the chance for metering systems to correct themselves. For example, a metering system may be set up to send a daily drop of information, and then a weekly summary for the previous 7 days to correct any potential errors. In this case, the offset could be set to 7 days to allow the weekly summary upload the chance to correct the data before any problems are identified.
Also Include	All Unchecked	Define the combination of data values that should be considered missing data. Options include: Zero - No Error Checks for recorded values of '0' AND a status of 'No Error'. This represents data that the source metering system has sent as an actual meter reading value of zero. This should be used with caution as could represent valid data - it is intended to check meters which never expect to receive readings with a value of zero based on their operation.
Auto Fill Type	Do Not Auto Fill	Choose from: Do Not Auto Fill An open Notification will be created which will be visible in the Notifications Tab for you to deal with and resolve manually (note gaps can be resolved in bulk) Auto Fill Recurring Gaps Sigma will automatically 're-fill' any data gaps that have previously been manually resolved by a User as soon as they are identified. A resolved Notification will be created to provide a full audit. This is to ensure data gaps aren't re-created if the source data is re-loaded with the same gaps that a User has already resolved Auto Fill All Gaps Sigma will automatically fill all gaps that are identified. A resolved Notification will be created to provide a full audit. Please note, where the system is configured to fill all gaps automatically, if it finds a channel with more than 100 different gaps that need to be filled at the same time, then the system will not attempt to fill the gaps, but will instead raise the notification with a status of "New" so they can be manually reviewed and the necessary action taken. This is to highlight a potential fundamental problem with the ongoing recording of the metering data, or where a strategy may be incorrectly configured. It prevents the system from manipulating the data which cannot then be reversed.
Auto Fill - Weeks to Average	4	Specify the number of weeks that should be used to create a period average when automatically filling data gaps. More information about how this feeds into the system processing that it supports when filling the data can be seen here.
Check Period (Months)	12	The number of preceding months that Sigma will check for gaps. For example, if you choose 12, Sigma will check the previous 12 months for any gaps

Click OK to save the Monitoring Strategy.

Data Gap Notifications

If data gaps are found for one of the strategies, then notifications will be raised that are visible in the Notification Tab.

Where the strategy was configured to auto-fill all gaps for you, then the notification will still be raised in a resolved state, to provide visibility of what the system has done, providing a full audit.

Resolving Data Gaps

Where the strategy was configured not to auto-fill all gaps for you, then these can be manually resolved using the "Resolve" functionality available.

Highlight the applicable Data Gap Notification(s) in the list.

Click Action Notifications to open the Resolve Notifications popup

If you select notifications which include types other than gaps, then a prompt will be displayed to confirm which strategy type you want to work with as the options available are contextual to the type of data issue being worked.

The Resolve Notifications popup will be displayed, showing a list of the Notifications being resolved.

The graph at the top shows the periodic data for the channel linked to the notification selected. The red area shows where the Gap is.

Drag the mouse around the area to zoom in.

Click to reset the zoom back to the default.

Actions - Data Gap

Update Period Average Setting

You can change the number of Weeks used as a basis for calculating the data to use when filling the gap, select all the Notification records it should apply to then click Update. This will update the period to use when calculating the averaged data to fill the gap.

Tip

Use the "Select All" toggle at the bottom of the notifications panel if you want to quickly select all notifications.

Preview

If you want to see what the filled data will look like, click first.

This will show you what the filled gap would look like for the Notification that has been selected in the list. The filled period is highlighted in green.

Drag the mouse around the area to zoom in.

Fill

Click to fill the gaps.

The Resolve Notifications pop up appears:

A Note can be optionally entered if required and will be stored against the Notification.

Click OK.

The following confirmation popup will be displayed: