GETTING STARTED | WORKFLOW CONTROL | KNIME ANALYTICS PLATFORM
KNIME — Cases, Switches and Catching Errors
An overview of the key capabilities for workflow control
Besides Loops another important tool in the Data Scientist’s inventory are switches and the ability to catch and handle errors. KNIME as your favorite low-code data analytics tool is well equipped in that regard.
First you have basically two types of nodes here:
- Switches where you set a rule and know what you are expecting
- Catching errors. Where something might go wrong and the program should react to it
You can combine the two if you want.
TL:DR: Just show me the sample workflows:
“Switch — Empty Tables Switches and Cases — Try-Catch”
“Switch — Examples of how to use Case When”
Similar to loops, you will have to create an enclosed set of nodes with a node at the start that will initiate the action (or the monitoring) and a node at the end that will capture the results. Like always a combination with Flow Variables or Loops is absolutely possible and will enhance your automation capabilities.
CASE Switch Start / End
KNIME has introduced these powerful nodes where you have the flexibility to work with nearly every kind of port type and send the work-stream to the branch you choose. The CASE Switch Start node can handle various port types:
The port number to continue with can be controlled by a Flow Variable that you can fill from the results of earlier workflows or you could let the user enter a value. Ports start with 0 (zero):
You can use all sorts of nodes that support Flow Variables to make your decision, like for example the Rule Engine Variable.
The last thing to do to finish your basic CASE Switch Start is close the task with a CASE Switch End node. You will have to decide what should happen then. So, for example, if you send through a data (table), you will have to handle RowIDs — and also provide an alternative stream of data:
You can also just use Flow Variables at the end and do completely different things in each branch of the switch. You will have to plan ahead what you want to do and also take care of edge cases and make sure your workflow will run through.
You can mix CASE Ports of different types
You can mix different types of ports in a switch construct. For example, you can start with a Flow Variable CASE Switch and then collect the results in a database connection that is taken from one of the branches:
Catch an Error in KNIME
The next thing is to handle errors in KNIME and continue (or stop) the workflow. A very basic node is the Empty Table Switch node that would allow you to identify (well) empty tables and stop the process or start an alternative one without having to deal with flow variables or specific rules.
Catch the Error and provide an Alternative
Key to working with this Try-Catch Concept in KNIME is to prepare the ‘wiring’ in a correct way. Most of the time you will put one of the Try nodes at the start of the block you expect to possibly fail and then provide an alternative of the same port type.
The most basic version would involve a Try node (Flow Variable or Table), then a data process that might fail (in this case failing it deliberately) and then providing an alternative. This alternative might have to have the same structure as the original data, if you want to continue your processes.
This alternative you provide also can be a Flow Variable so you have a wide range of options of what to do. You can even combine the two concepts of Try and Switch to handle errors and then provide a reaction to the error:
You can expand these KNIME concepts further by employing loops.
Loop until a Job is Finished
In the next example you make sure an Excel file is written to a folder (might be a remote folder or via a shaky connection) and the job continues until success (or after a fixed number of trials).
You could also just record which files have been written (or being downloaded) successfully, so you can deal with the problems later without stopping your task:
I hope you enjoyed this text and move on to improve your workflow handling and automation.
Further hints and examples of switches and cases you might explore
- To develop and debug it can help to select the scope of a loop (right click and select the scope — this might not work yet under the KNIME 5 GUI)
- It can help to encapsulate your inner workings in a Component or Metanode so it is easier for KNIME to handle the task — this might especially be true if you are dealing with nested loops
- Another operation that might come in handy is to wait for a file to appear in a folder to continue with a task
- A collection of more examples about switches on the KNIME Community Hub
In case you enjoyed this story you can follow me on Medium (https://medium.com/@mlxl) or on the KNIME Community Hub (https://hub.knime.com/mlauber71) or KNIME Forum (https://forum.knime.com/u/mlauber71/summary).
