Introduction

The Importing activity allows you to import data into the database with various file formats. This data can be for Channels, Site information or Meter Readings. The importing can be scheduled to load at a desired frequency picking up data from an email account, FTP or FTPS.  To manually import, click here.

Opening the Importing Activity

• Click on Importing

You can choose to import data immediately or schedule it to import at a time suitable to you.  

Below will show you how to schedule the import.

To import immediately, click here.

Scheduling an Import

To schedule an Import:

• Click 

Adding an Importer

If the importer isn't in the Favourites list, you need to add it in.

• Click  to open the Importer Selector popup

• Enter the name of your Import source ) ie IMServ)
  • Click Search to see a list of Importers
  • Highlight the correct Importer
  • Click OK

• Click   
• Click 

The Importer will now show in the Favourites list.

Using the Importer

• Highlight the Importer you want to use

Importer Options

Once you have highlighted the Importer:

• Click  

Amend the Import Options as necessary:

• Use date range  - generally, no change is required
• Time Zone - Best practice is either GMT or Europe/London, depending on whether your data includes the clock change data or not (ie 1 hour extra in October, 1 hour less in March) 
  • It is recommended that you use the same Time Zone as the Channels you are importing data into, ie
    • If you data does NOT include the clock change data, use GMT
    • if your data DOES include the clock change data, use Europe/London

• Click   to move to the next step.

Scheduling the Importing via Email

Sigma is able to pick up files to import from email mailboxes and is able to support both Encrypted and Non-Encrypted versions of POP3 connection.

The secure version is often known as "Secure POP3" or "POP3S". When selected, Sigma's connection to the email inbox will be encrypted using SSL/TLS  (Secure Sockets Layer/Transport Layer Security).

When setting up an email importer, you will need to indicate whether the Encrypted or Non-encrypted version of POP3 should be used.  You will also need to specify the Port Number to use.  These details can be provided by your IT Team, but please see table below for commonly available email providers.

Gmail App Passwords

Gmail has made a security change on 30th May 2022 which prevents Sigma accessing these inboxes using the regular password.
You can log in to the inbox and generate an App Password for Sigma.
Updating the schedule with the app password will allow the schedule to continue working as before.
 

• Details on change from google. Click on the below link for more information:
  • https://support.google.com/accounts/answer/6010255?hl=en

• App Passwords.  Click on the below link for more information:
  • https://support.google.com/accounts/answer/185833

Email Import Option

To use the email import option, tick 

Setup the configuration options as per the below:

• Email Server -  enter the email server address
• Username - enter the email address
• Password - enter the password for the email address
• Protocol - choose either POP3 Encrypted or POP3 Non Encrypted
• Port -  enter the Port number, this will default to 995 (encrypted) or 110 (non-encrypted)
• Sender Address (Optional): This tells the system to only look for emails that have been received from a specific sender email address. Note a partial email address could be entered (e.g. @teamenergy.com would only search for emails received with the domain @teamenergy.com)
  • This is important if you are have more than one EDI Supplier sending emails to the one email address or you are using you normal email address for importing 

• Subject (Optional): This tells the system to only look for emails that have been received with a specific subject. Note a partial subject string could be entered and is case insensitive (e.g. EDI File would find any emails that contained the words "EDI FIle" in it, even if the actual subject was "June Edi File Resend").
  • This is important if you are have more than one EDI Supplier sending emails to the one email address or you are using you normal email address for importing 

It is recommended that you test the connection to the mailbox before the schedule is saved.  This provides comfort that the system is able to connect before it runs for the first time and will not fail because of incorrect configuration and will provide details of why the connect was unsuccessful.

• Click 

• Click   to move to the next step

Scheduling the Importing via FTP

FTP stands for File Transfer Protocol and is a standard method for transferring computer files over local networks and the internet. The FTP works by having a program where the data is being transferred from known as a Server and another program that connects to the server known as the Client that requests the file(s) to be transferred and copies the data over. There are several different programs that work as either Server, Client or both.
One of the benefits of FTP is that it allows access to the files to be restricted to only certain users rather than making these publicly accessible.

Sigma is able to support both FTP and FTPS. The main difference being the added layer of security for FTPS which ultimately ensures the secure transmission of files between the the remote server and Sigma. FTPS stands for "File Transfer Protocol SSL (Secure Sockets Locker)". The SSL reference relates to the security aspect - a cryptographic protocol that encrypts the data being transferred.

Sigma has a built in FTP Client that can connect to an FTP Server to import data automatically via a schedule. To set up the automated importing an FTP Server will need to be set up. There are a large number of FTP Server Software providers offering a range of different functionality, TEAM would recommend that you pick the most suitable option for your business in line with your IT policies.

Whichever FTP Server program is chosen this will need to be set up with access to the folder where the files that need to be loaded to Sigma are sent to and able to be accessed from the internet. A user name and password will also need to be created to access the data, this will be entered in to Sigma when setting up the schedule.
If you want to limit access to the FTP further most Server programs will allow access to be limited to a range of IP addresses. Please contact Team Support if you require a list of the IP addresses used by Sigma.

The key configuration details for FTP are:

• Host name -  the host name of the FTP server whereby the files are stored that need to retrieved
• Username - the username to be able to access the FTP server
• Password - the password to be able to access the FTP server
• Path - the directory path to where the files are stored and need to be picked up from the FTP server

 Contact your IT department for the above details

• Ignore root files - Tick if you want to go directly to the folder indicated in the Path (ie ignore the root files)
• Secure FTP? (FTPS) -  Tick if you use FTPS (File Transfer Protocol Secure)
• Implicit Secure FTP? -  Tick both FTP? and Implicit Secure FTP? if  using Implicit.

Implicit FTPS is actually more strict than explicit FTPS when it comes to establishing a secure connection. In fact, the entire FTP session is encrypted, in contrast to flexibility you have when using explicit FTPS

• Click 

If successful, you will get the following prompt:

If unsuccessful, you will get the following prompt:

(Contact your IT Department or provider of the FTP server for advice if the FTP connection could not be established).

• Click  to continue setting up the Schedule

Schedule Options

General Settings 

• Schedule Active - tick to create an active Import
• Name - prefilled but amend if required
• Description - add a description (mandatory)

Allow schedule to become active after this date, and run at this time

• Start Time - amend to time and date to start the Importing, preferably out of office hours

Recurrence Pattern

• Choose how often you want the Schedule to check the email account for unread emails with attachments
  • Choose from Daily, Weekly, Monthly etc

Range of recurrence

• Choose No end date, End after or End by.

• Click 
• Click 
• Click 

The Import will now run at the specified time and frequency for either POP3 or FTP (depending on your earlier choice).

Viewing the importing progress

The file will show in the Queued And Running Imports area (depending how long the import takes, sometimes it doesn't show)

Refreshing the Run History

• Click on the Refresh icon in the Run History section
• The screen will refresh to show the latest import, and its status (success or failed)

Click here for more details on the Run History section, including viewing run details and downloading log files.