This article is to illustrate how to set up datasets using our FTP ingestion pipeline. This dataset type should be used for datasets that have regular additions (e.g. daily or monthly) and/ or sensor networks that have large volumes of data, multiple sites but have the same column headings.
Step 1 - Create a real-time dataset
Select the real-time data toggle to on.
Step 2 - Choose a publisher or publishers for the dataset
The next step is to use the API Key Writers section to create someone who can publish. To create an API writer in the first place go to the “Management Section” and add a new API Key. Remember to save the API key and API secret so you or your designated publisher can add their data.
The next step is to use the API Key Writers section to create someone who can publish.
Step 3 - Use your FTP publisher
In your FTP data publisher complete the following information
Host: ftp.usmart.io
Username: “Your API Key”
Password: “Your API Secret”
Port: 7003 unless advised otherwise
Choose the “connect” option on the FTP server. The folder should appear in the bottom right hand box. In the above example it would be b41c9d16-9af0-4b4d-abde-f5796af90c1a.
Simply drag and drop and your data will start appearing in the dataset.
Step 4 - Checking the data
Once your FTP publisher has registered the transfer as complete you should be able to see data appearing in the black “Preview” box at the bottom of the screen.
You will also be able to search the data by selecting Filter and then typing in text for the data you expect to see. For example, if you have just added data about site 1029 type that in the site column, and if it is between particular dates choose those dates in the timestamp fields.
Depending on the size of file it may take a few minutes for the data to appear but please let us know if you think there are any issues at all.