Python Learning Note — Streamlit

Background:
I’ve been searcing for language to process data from MCU. And ends up with Python could be a nice language.

I used to play with C# for real-time data visualization and recording. But since I am learning python, why not use it to pick up and visaulize data. I decide to name this series as “Learning Note” which means it will be a note instead of tutorial. So steps may skip and just talks about some important parts.

Material for this learning note

1. Python3.7 with Numpy and Streamlit installed.
2. Particle Photon + Si7021 module
   You can use the exercise that I created before: <LINK>

Target

The final target is to pick up the measured data from Si7021 and MCU will transfer it to the host computer. Finally, we will use python to pick up data and visualization.

Let’s have a look at Official Tutorial

Type following command in the console:

streamlit hello

We choose the “Plotting Demo”:

So …. If we need a “Chart”, we will need to create a chart first using:

chart = st.line_chart(last_rows)

Then, we can add more data to the same chart via:

chart.add_rows(new_rows)

These are 2 basic instruction all we need for showing a chart.

String partition:

The UART(serial port) from MCU is sending data in string format. Luckily, python has a strong capability for string processing.

The messages from MCU will look like this in RAW format:

We can read the message per line using the following command:

data = particle.readline()[:-2] 
#the last bit gets rid of the new-line chars

Then, we can check whether it begins with Temperature or Humidity. In this way, we can know the physical meaning of the number. We need to split the sentence by “space” using the following command first:

#Convert bytes to str data type
data_str = data.decode('utf-8')
data_str = data_str.split(" ")

For temperature:

if  data_str[0] == 'Temperature:':
#Convert str to float data type
            temp = float(data_str[1])

For humidity:

elif data_str[0] == 'Humidity:':
#Convert str to float data type
            humd = float(data_str[1])

Remember to covert the data type from string to float!!!

Otherwise, something terrible will happened. This bug will let you run the streamlit but create something weird.

Challenge

There are some challenges if you are a noob in Python like me.

First, the data array that you feed into streamlit chart is not a normal array. It has to be a NumPy array. So…what you should do is :

total_data = np.array([[temp,humd]])

“temp: and “humd” are data that we identify from the UART.

We will also need to show which line is temperature and which line is humidity. So, we also need:

chart_data = pd.DataFrame(total_data,
                          columns=['Temperature', 'Humidity'])

Then, we should be fine to create the chart.

We also need to use the following command to add new data each time we receive a data set.

chart.add_rows(chart_data)

Let’s check the array whether it fits our target or not:

Cool! It seems to match what we are looking for.

Code!

The full Python code will look like:

Several things I would like to mention:

1. Python is an interpreted language. So make sure that you create the chart with valid data first. That is why we use first_lock_1 and first_lock_2 to check whether we receive the first temperature and humidity data. If not, don’t create the chart because you have nothing to plot.
2. We also want to update the temperature and humidity at the same time. So data_flag_1 and data_flag_2 are signal flags to make sure new data is received.
3. Check your serial port in your computer, everyone will have the different serial port number. Make sure you change to correct port!

Result

Using the following command to run the streamlit python code:

streamlit run "your_python.py"

The fluctuation of the temperature and humidity are caused by I placed my fingers on the sensor. The X-axis is the data count. The Y-axis is the values.

To do list:

1. SAVE Button: I will add a button which can save the data to a file.
2. Different line: I wish to show the tolerance of the sensor which mentioned in the datasheet in visualize data.
3. Make the code “Pythonic”: I feel terrible in the current coding style. I will try to make it more readable and nice coding style.
4. Close serial port in a proper way: I try to use “serial.close()” function to close it. But it seems to create some issue.
5. Automatically search for the particle serial port. I believe it can be achieved since particle command does have this feature.
6. Real-world timestamp instead of data count for the x-axis.

Leave your comment below if you have other ideas for the to-do list!