Creating and adding to datasets with File Transfer Protocol (FTP) data ingestion

Owned by Steven Revill
Last updated: Jul 07, 2023 by James McLeod
2 min read

This article is to illustrate how to set up datasets using our FTP ingestion pipeline. This dataset type should be used for datasets that have regular additions (e.g. daily or monthly) and/ or sensor networks that have large volumes of data, multiple sites but have the same column headings.

Step 1 - Create a real-time dataset

Select the real-time data toggle to on.

Step 2 - Choose a publisher or publishers for the dataset

The next step is to use the API Key Writers section to create someone who can publish. You do this by selecting the management tab on your left hand menu bar.

To create an API writer in the first place go to the “Management Section” and add a new API Key. Remember to save the API key and API secret so you or your designated publisher can add their data.

The next step is to use the API Key Writers section to create someone who can publish.

Step 3 - Use your FTP publisher

In your FTP data publisher complete the following information

Host: ftp.usmart.io

Username: “Your API Key”

Password: “Your API Secret”

Port: 7003 unless advised otherwise

Choose the “connect” option on the FTP server. The folder should appear in the bottom right hand box. In the above example it would be b41c9d16-9af0-4b4d-abde-f5796af90c1a.

Simply drag and drop and your data will start appearing in the dataset.

Depending on the size of file it may take a few minutes for the data to appear but please let us know if you think there are any issues at all.