DATA STORIES | REST API | KNIME ANALYTICS PLATFORM

Hey Siri, run my KNIME Workflow

Workflow automation at your fingertips (literally!)

In my role at Forest Grove Technology, I was always trying to find a way to provide clients with innovative solutions and leverage tools they already have. In summary as little to no coding as possible.

When Apple released Apple Shortcuts in iOS 12, I was hooked testing out all the fun ways I can automate my day to day life…. getting up off the sofa is for suckers! I was playing around with Apple Shortcuts and realised it has full support for REST APIs and JSON. This got me wondering, could I trigger a KNIME Server workflow using Siri and the answer was absolutely!

A KNIME Workflow being triggered by Siri and reading the response.

The JSON for the example above is just a list of 3 actors and their names, date of birth etc. I’m using JSON as the input so you can copy and paste the text below and follow along but you’ll need KNIME Server Medium or Large, KNIME’s commercial offering to access the API.

{
  "Actors": [
    {
      "name": "Tom Cruise",
      "age": 56,
      "Born At": "Syracuse, NY",
      "Birthdate": "July 3, 1962"
    },
    {
      "name": "Robert Downey Jr.",
      "age": 53,
      "Born At": "New York City, NY",
      "Birthdate": "April 4, 1965"
    },
    {
      "name": "Margot Elise Robbie",
      "age": 28,
      "Born At": "Dalby, Australia",
      "Birthdate": "July 2, 1990"
    }
  ]
}

The KNIME Workflow

Simple KNIME Workflow to read REST data and write REST data.

The KNIME workflow was a simple one, in a production/commercial environment this could be much more complex, running ETL/ELT or Data Science workflows in real time.

Imagine asking Siri to run a KNIME workflow that executes a Spark cluster running a Data Science algorithm and getting the response on your Apple HomePod!

Reading the JSON Input

We could have just passed the JSON straight out and used Apple Shortcuts to do the JSON processing but where’s the fun in that? Below is a micro tutorial on getting the JSON into a readable table for processing.

1. Open a new workflow, I’ve called mine API Example and add in a Container Input (JSON) node, double-click to configure and copy then paste the data above. Make sure you paste over the existing {}.
2. Use the JSON Path node to select just ages and birthplace from the JSON input. I personally find it easiest to select the value you want, then select Add collection query to select this field.

First select the Age to select the field, then click Add collection query.

3. Repeat this for the Born At field and tick Remove source column to tidy up the output and click OK (Hint: hold down the Command or Alt button to OK and Execute).

4. Okay lets get this data back to JSON and out. Add a Columns to JSON node, double click to check everything is in order, you shouldn’t need to change anything and click OK.

5. Finally, add a Container Output (JSON) node which will act as the REST API’s outbound node. Double click to configure, keep everything as it is and click OK.

6. If you haven’t yet, run the workflow, give it a quick save and everything should look like it does in the above workflow screenshot at the start of this tutorial.

Deploying to KNIME Server

This is already covered well in KNIME’s guides on how to deploy a Workflow to the server. It’s super easy just right click on the local workflow and select Deploy to Server…

Specify what workflow group you want to save the workflow to and KNIME will take care of the rest.

Interacting with the REST API

Once uploaded, you can navigate to your KNIME Server in the KNIME explorer, right click on your workflow and you’re presented with a bundle of options.

KNIME Server options on an uploaded KNIME Workflow.

The one we’re most interested in is the Show API definition. KNIME has made interacting and understanding their REST API’s incredibly easy by leveraging Swagger. Swagger allows you to see a full breakdown of the REST interface, the various services available to us and how to interact with the Workflow we selected.

For this tutorial we’re going to focus on using the execution GET API to trigger our workflow and return values from the Container Output in JSON format.

Let’s setup our authentication to the KNIME Server, this will enable us to grab the encoded basic authentication key from the curl URL later in the process.

1. Select the Authorize button in the top right corner of the web page

2. Enter your username and password and hit the Authorize button again.

3. Select the POST execution API and click on the Try it out button. All within the Swagger interface you can see exactly what will be returned. I’m going to set the following parameters;

As you scroll down, you can see the input parameters of the job, this will be the JSON that we entered in the Container Input (JSON) node.

4. Hit Execute to view the curl, URL and response body from the API. All of this information we’ll leverage in our Siri shortcut. In particular we’ll use the Request URL, in my case;

http://192.168.0.232:8080/knime/rest/v4/repository/FGT/FGT-PoC/REST/Siri Experiment:execution?reset=true

and the Response body so we can understand the format and structure of the JSON response.

5. Keep this screen open so you can easily copy and paste it to your iPhone or iPad to create the Apple Siri Shortcut.

Creating the Apple Siri Shortcut

Now, I could have done 100% of the processing in KNIME and just passed the result “The average age is 45.67” to the shortcut but I wanted to show how you can use Apple Shortcuts JSON functionality to loop through each record and calculate statistics on the device itself.

Apple iPhones are extremely powerful and capable of processing large data sets.

If you haven’t done so already, download the Apple Shortcut app from the App Store. We’ll be stepping through each action we need to connect to our KNIME wokflow.

1. First let’s setup our connection. We need 2 actions, a URL action and a Get Contents of URL action.

Add a URL and Get Contents of URL action.

2. Grab the Request URL from the Responses section of the Swagger web page and paste it into the URL of the Shortcut

Grab the Request URL from Swagger.

Paste the URL into the URL action in Apple Shortcuts.

3. Expand the Advanced and Headers section so we can setup our Basic Authentication header.

Add a new header for Authentication in Apple Shortcuts.

4. Add a new header and enter in the Key section, Authorization and in the Value section enter Basic and replace the XXXXXX with your authentication key. You can find your key on the Swagger page within the curl response;

Basic XXXXXX

Grab your secret key from the Swagger curl command.

5. Run the shortcut as a test, you should get a JSON output as expected;

Run your shortcut and you should be getting a JSON response.

6. Now time to read the JSON output using the Get Dictionary Value actions to navigate through the JSON tree. The first one we need is to select outputValues.

Select outputValues.

7. Run the action and take note of the name of the output node, in my case it is json-output-65.

Take note of the name of the output node.

8. Add another Get Dictionary Value action to select the output node. Enter as the key the name of the output node you want to parse data for.

Select the node “json-output-65”.

Looping through each item

This final part has a few steps to go through to get Apple Shortcuts to loop through the list and calculate the average.

1. Add a Repeat with Each action and within the Repeat with Each add a List action.
2. We want to select as part of the list the ages field in the JSON response. To do this, delete the example One and Two and select Add new item.

Remove the example One and Two and select Add new item in the List action.

3. From the list of variables above the keyboard, scroll through the list and select Repeat Item.

Select Repeat Item from the list of variables.

5. Select the Repeat Item in the List and select as Text to change the value to Dictionary.

Select Repeat Item and then Select as Text to change the value.

6. Select Dictionary from the list of field types and change the Get Value for Key from None to ages.

Select Dictionary.

Enter the value ages. This corresponds to ages in the JSON file.

7. Within the Repeat with Each loop, add a Calculate Statistics action after the list to calculate the Average for all the values.

Setting the Siri Shortcut and Output

1. Add a Format Number action after the End Repeat to format the number to two decimal places.
2. Add a Show Result action after Format Number. Here you can add text to provide context such as “The average age is:” along with the Formatted Number variable. This is what Siri will read out loud.

Format the number and Show Result providing context for when Siri speaks.

3. Open the Shortcut Settings, select Add to Siri and dictate the name you want to give the shortcut and voilà, when you give Siri that phrase it will run the shortcut and give you the result.

Add a Siri Shortcut to trigger the Shortcut via voice.

And that’s it! You’ve integrated JSON, KNIME Server and Siri. You can expand this to pass inputs to KNIME Server.

Some ideas for a Siri/KNIME workflow:

• You can pass the username and password then use the Base64 encoder and pass that for authentication.
• Use text inputs and variables to pass parameters to your KNIME workflow.
• Use KNIME as a conduit to core systems, asking things like “Give me my product sales revenue”

Download the example Apple Shortcut

https://www.icloud.com/shortcuts/e22f3358b57442e88580175690ab66bd

Apple has a great tutorial on how to use Apple Shortcuts with API’s:

Request your first API in Shortcuts