GETTING STARTED | CONDA & PYTHON | KNIME ANALYTICS PLATFORM

How to Create Your Own Python Environment with KNIME: A Step-by-Step Guide

Learn how to install packages into your Python env, share that env, and play a round of Snake in KNIME

KNIME is an excellent data science platform that offers plenty of functionality to conduct tasks from the realms of data wrangling to data analysis and data science — without writing a single line of code!

For advanced users it comes with a pre-bundled Python environment. This environment includes packages that are useful for data visualisation, model training and even web scraping so in many case it covers all grounds without the need to worry about managing a custom Python environment or portability of workflows if shared with other users.

Packages included in KNIME bundled Python environment

But what if these packages are not enough? In this article we’ll walk through the steps to set up a custom Python environment using Conda or Miniconda by installing packages in addition to the meta packages KNIME requires on Windows.

You can also follow along this YouTube video which walks through the same process:

Prerequisites

Before we dive into the process of creating our own Python environment, let’s cover the prerequisites needed for KNIME. You will need to have installed two extensions:

1. KNIME Python Integration: This extension allows you to use the Python script node in KNIME.
2. KNIME Conda Integration: This extension enables the Conda environment propagation node, which is essential for selecting your own Conda environment.
3. Conda or miniconda installed and configured in KNIME: This makes sure that the environment we create can later be used inside KNIME. I have covered this process in detail towards the end of my article on CrewAI which you can find here.

Integrating Agent Frameworks into Low Code Tools: Making CrewAI work in KNIME

How about a round of Snake?

To demonstrate how to create a custom Conda environment with additional packages, we’ll use a rather unconventional, yet (hopefully) entertaining, example: We will make the game “Snake” run in KNIME.

For this I found an implementation of the game in Python which uses the package “pygame”. Needless to say that this package is not included by default.

You can find it by following this link at the very bottom of the page.

Let’s validate this with a quick test:

1. Grab a Python Script Node and drag it on your Workflow Canvas
2. Right-click the Input and Output ports and remove them
3. Right-click and configure the Python Script node
4. Delete the default code and copy & paste the code from the source above
5. Click “Run all”

Et voilà — we got ourselves a nice error proving it won’t run with a little additional effort.

Creating a Custom Conda Environment:

The KNIME Python Integration Guide provides the documentation we need. Just below the list of packages included by default is a section that covers how to create a Conda environment including the metapackages required by KNIME in addition to other packages you may need. Below is the line of code ready to copy that will create an environment named “knime_pygame” which includes the package pygame. In general you can edit the name by changing the text after the — — name and you can add any other packages at the very end after pygame.

conda create --name knime_pygame -c knime -c conda-forge knime-python-scripting=5.2 python=3.11 pygame

1. Copy the code above
2. Go to your Windows search bar
3. Type “CMD” and Hit Enter
4. A command line / terminal window will appear
5. Paste the code from above and hit enter
6. After a while you will be asked to confirm to go ahead with the installation — type “y” and hit enter
7. The installation will be complete once the terminal looks like below screenshot (and you can type again). At this stage you can close the terminal window

Windows Terminal Window — status after installation of environment is complete.

Using Your Custom Conda Environment in KNIME

Now it’s time to return to your KNIME window and search for the Conda Environment Propagation node.

1. Drag it onto your workflow canvas and right-click on it to configure.
2. Select the dropdown menu and search for knime_pygame, the environment you just created.
3. Confirm with OK
4. Right-click and execute the node
5. Once the traffic light shows green, connect the red variable port of the Conda Environment Propagation Node with the Python Script Node
6. Right-click the Python Script Node and Configure Flow Variables
7. Under “Python 3 command,” select the dropdown menu and choose the conda.environment variable.
8. Click OK to confirm

Let’s play some Snake!

Let’s play some Snake!

1. Right-click on your Python script node and select configure
2. In the bottom-right corner click “Run All”
3. A new window should appear where the Snake is running. You can navigate using arrow keys, eat fruit that spawn, and if you hit the wall, you die and your score will be displayed.

You can also launch it by executing the Python Script Node — you may have to check your Windows task bar for the new window where Snake is running

Snake running inside of KNIME!

Final Remarks

While playing Snake in KNIME might not have much practical value, this example illustrates the process of installing additional packages to your Python environment.

You can follow the same process to install other packages you may need into a custom environment and the best thing is: By using the Conda Environment Propagation node you can share your workflow confidently with other users (provided they are running Windows and have Conda set up correctly) as the environment will be automatically created when running the Workflow for the first time.