Creating and adding to datasets with live data writer/ command line

Owned by Steven Revill, created
Last updated: Jul 07, 2023 by James McLeod
5 min read

Data can be provided to uSmart via a command line or script using our live data writer functionality. The most popular command line solution, which is being used for uSmart data provision in this manner, is curl. Guidance can be found at https://curl.se/. Other command line options will also work if they comply to standard web protocol and can meet the requirements illustrated below.

Within your command line or script you will need to include your API key and API secret as provided (see how to guide on setting up an API key and secret).

Example

Your command line should include a number of key components and specifically:

  • User Credentials - your security details (API key, etc)

  • your target dataset

  • your data payload

The full command is shown unbroken at the bottom of this page.

User Credentials

An example of the initial connection elements, that defines data provided to uSmart live data writer and the necessary credentials, are illustrated as follows:

curl -X POST https://livedatawriter.usmart.io/liveData:write -H "Content-Type: application/json" -H "Accept: application/json" -H "api-key-id:c5zz5965-4374-4b94-b444-06544b9eba8e" -H "api-key-secret:05901843c371273b12c732280d91ce0e39392786017c8ddfe435ac6b3a25cba8" -d "

Defining the dataset

For live data writer to be enabled the real-time switch must have been selected when you created the dataset in the first instance. Each dataset in uSmart has an organisation Globally Unique Identifier (GUID) and a dataset GUID. The organisation GUID is shown in the “write URL” illustrated in the following image and is specifically the text after “org/” and before the folder details.

In the below example therefore the text in bold is the organisationGUID. https://livedatawriter.usmart.io/org/d1b773fa-d2bd-4830-b399-ecfd18e832f3/d0173990-d83c-4e6e-bd81-a71d190ef67e/write

The “Folder Name” is the datasetGUID.

Specifically, the code to determine the dataset correctly can be written as follows:

{"organisationGUID":"d1b773fa-d2bd-4830-b399-ecfd18e832f3","datasetGUID":"d0173990-d83c-4e6e-bd81-a71d190ef67e",

Adding the data

You will notice that the data payload code is within one open brace { already. Data is then added in the following manner:

"data": {"Bedrooms": "3", "Date_survey": "2020-09-08", "Elec_currentuse": "1200", "Existing_Cooking": "1", "Existing_Fires": "0", "Floor_Height": "2.6", "Floor_Type": "Suspended", "Gas_currentuse": "10200", "Glazed_Type": "Double", "Habitable_Rooms": "4", "Heat_Primary_Source": "Gas Boiler - Combi", "Heat_Secondary_Source": "None", "Heated_rooms": "4", "House_Age": "1722", "House_Env": "Urban", "House_Form": "Semi-Detached", "House_ID": "1", "House_SAP": "50", "House_Type": "House", "Main_Fuel": "Gas", "Mains_Gas_Flag": "Yes", "Multi_Glaze_Proportion": "80-100%", "Name_Survey": "urbantide", "No_Adults": "2", "No_Child": "0", "No_Occupants": "4", "No_Storeys": "2", "No_Underfloor": "0", "Outside_Access": "No", "Outside_Space": "No", "Postcode_1": "ZW12", "Roof_Type": "Pitched", "TS_Existing": "None", "TS_Existing_Size": "", "Total_Floor_Area": "80.34", "Wall_Type": "Cavity_Insulated"}}"

Within the payload you can see key elements including:

  • the text within the first quotation marks e.g. “Bedrooms” is the column header. If these are the first entry then they will be consumed as is. If they are resultant entries after the first then they must match exactly to enable data ingestion to be successful

  • the text within the next quotation marks e.g. “3” is the relevant entry for that row

  • and finally \n to move to the next column entry

  • to move to the next row, as long as the data is structured correctly, you will publish in the same method as above as the \n will return you to the next row of data and continue

  • Make sure you end by closing both braces

We understand that the above can be quite technical and if you are looking for any further assistance please contact us directly at the uSmart Service Desk.

Finally here is the full script from the above example:

curl -X POST https://livedatawriter.usmart.io/liveData:write -H "Content-Type: application/json" -H "Accept: application/json" -H "api-key-id:cc5zz5965-4374-4b94-b444-06544b9eba8e" -H "api-key-secret:05901843c371273b12c732280d91ce0e39392786017c8ddfe435ac6b3a25cba8" -d "{"organisationGUID":"69210836-6c2b-4a26-a0a0-b903bde0dc46","datasetGUID":"fcd44037-4ba7-406f-8227-251bab530a99","data": {"Bedrooms": "3", "Date_survey": "2020-09-08", "Elec_currentuse": "1200", "Existing_Cooking": "1", "Existing_Fires": "0", "Floor_Height": "2.6", "Floor_Type": "Suspended", "Gas_currentuse": "10200", "Glazed_Type": "Double", "Habitable_Rooms": "4", "Heat_Primary_Source": "Gas Boiler - Combi", "Heat_Secondary_Source": "None", "Heated_rooms": "4", "House_Age": "1722", "House_Env": "Urban", "House_Form": "Semi-Detached", "House_ID": "1", "House_SAP": "50", "House_Type": "House", "Main_Fuel": "Gas", "Mains_Gas_Flag": "Yes", "Multi_Glaze_Proportion": "80-100%", "Name_Survey": "urbantide", "No_Adults": "2", "No_Child": "0", "No_Occupants": "4", "No_Storeys": "2", "No_Underfloor": "0", "Outside_Access": "No", "Outside_Space": "No", "Postcode_1": "ZW12", "Roof_Type": "Pitched", "TS_Existing": "None", "TS_Existing_Size": "", "Total_Floor_Area": "80.34", "Wall_Type": "Cavity_Insulated"}}""