Tell me more Internet of Things — Part 3 — MindSphere cost and value comparison
MindSphere — device onboarding, data storage and visualization. How to connect your device to MindSphere via the MindConnect library (C and Node.js) and visualize the data.
Part 1 | Part 2 | Part 3 | Part 4| Part 5 |Part 6 | … | Part n
In Part 2 of this series we explored Google IoT for our environmental sensor device. The average cost per device was determined to ~0.7 €/device per year. There was even a solution which was entirely free (in case you wont get any paying participants / customers 😏).
Here we do the same but with Siemens MindSphere. Thanks to my friends in Siemens I have access to an account. It’s not for free and the pricing is simply sorted in to three different plans (small, medium, large). All necessary increases beyond the plans allowances are not transparent. As our scaled use case is well above some of the plans limits I try my best to estimate the costs.
My approach is based mainly on
Certainly there is not so much third party tutorials compared to Google Cloud. I try to implement the most straightforward solution and where necessary point out the necessary adaptation when scaling up.
Overview of implementation and results
The solution is based on onboarding the device
- via the MindConnect Library M1
- On our raspberry pi device we use the @mindconnect/mindconnect-node.js library for onboarding and sending the data to our registered mclib-asset.
- Show how you can use the publicly available C library to create your custom agent.
- Creating an assets for receiving the data (agent-asset or mclib-asset) M2
- Creating an asset for visualizing the data (referencing, data-mapping, the agent-asset).
- Visualizing the data via the Fleet Manager M3
This implementation does still not provide
- User Management — individual visualization and event suggestions.
- Automated data analytics for the user (e.g. out of the box functions like MindSphere’s own predictive maintenance app)
I am afraid you need to wait until MindSphere will offer free tier (developer) value plan accounts or pay the smallest plan (~3600€/year). We use the Graphical User Interface (GUI) for creation of:
- Aspects (which act for us as templates for the sensors variables) and
- Types [Basic Assets] (which act as templates for our raspberry pi edge (referencing all sensor variables)
- Asset 1 — “Visualization” asset referencing our newly created basic type(Visualization of onboarded asset).
- Asset 2 — “mclib" asset referencing the mclib core.mclib template.
We do need administrator rights (the standard user as well as the subtenant user does not have the necessary rights to create any of these objects)
We create just one subtenant which represents our collective pi-sensor data store (note: we have not yet introduced the user itself, which likes to visualize their own data). The subtenant can see only his dedicated assets (which looks like we need as many subtenants as users if we proceed to the next Milestone M4 — but that is for later articles to be examined.)
The current implementation (w/o users and individual visualization and event monitoring) has a comparable high price. That is a factor 100 compared what would Google Cloud IoT charge. Knowing that most of the functionality and services provided by the base MindSphere plan we just won’t use at the moment. Especially the amount of agents which need to simultaneously ingest data to our MindSphere tenant (account) drives the price, as we have to increase the plan to accommodate for more agents onboarded. Both 1.000 and 10.000 device scenarios are not covered by the base plans and I have extrapolated the cost to show the comparison.
The below costs are just an estimation based on (past) publicly available information and are both not released by Siemens nor reviewed K1.
Beside the limits for agents we exceeding asset instances for the 10.000 device scenario. The data ingest stays well below. The following table shows as well that we won’t utilize most of the other functions.
Looking to the soft factors — MindSphere is easy to set up. Of course the compilation of your own C agent takes time but the available node.js library is very easy to use for prototyping K3. When it comes to documentation I would like to see some more end to end use cases from the community, but there is already quite a bit of official docs and examples available including Siemens own forums (most of it internal though) K4. MindSphere provides out of the box visualization which is in my opinion far easier to set up compared to google data-studio (but as well much less customizable) K5.
Anyway it seems that MindSphere is not focusing on this IoT market which accommodates massive amounts of “agents” (like in our scaled use-case). MindSphere allows much more data ingest and assumes to be competitive with less devices ingesting with higher output.
When it comes to database handling or data storage we need to further see how we could implement our further project milestones using the MindSphere open platform.
Implementing the use case in MindSphere
Registering a new user as subtenant
The below picture story shows what the administrator of the tenant has to do to create a subtenant and a new user. Below the user screens for registering and login to MindSphere are shown.
Creating the aspects, types and assets
As mentioned earlier creating the aspects, types and the users-own sensor representation is the tenant admin job. The subtenants users does not have the rights to create those assets.
We need to create the communication asset (mclib-asset) which authorize our device to onboard and to exchange data via predefined datapoint ids. And we need to create an asset which maps those datapoints and is able to visualize the data.
Onboarding the sensor data
Our raspberry pi has no preferred protocol for exchanging data to another device. If you are in the process industry usually the sensors are connected via BUS (Profibus, Modbus) or via 4–20mA analog signals to I/O cards from there digitalized and onboarded for the automation program. If you have a Mindconnect Nano or a S7 automation program you could onboard with those devices / protocols directly (Either with the IoT Extension or directly via a MindConnect Nano) .
Edited 07.02.2019: Thanks Marion. The value base plan comes already with the IoT Extension which allows 5GB data ingest. If more is needed the upgrade to 20GB holds additional costs.
With the raspberry we connect to the internet via WiFi and will communicate via node.js MindConnect library (agent) to MindSphere (we will compile our own C agent too ). The agent is communicating with the mclib assets securely via a shared secret. That is necessary for our use case we certainly can’t integrate a MindConnect Nano from Siemens to onboard and data communication (Otherwise we would need 10.000 nanos 😅). Beside the less computational shared secret encryption we could choose RSA too.
C MindConnect library agent compilation, agent onboarding and timeseries upload
Thats the tough part — if you like to see why I think its easy to set up, skip to the below node.js implementation.
You get the latest mclib version here. Some of the below scripts and files I have shared here: https://github.com/jhab82/tmmiot-MindSphere-mclib-agent
After accepting the conditions for using the library you are allowed to download. I have saved my version in google drive and shared the link for me being able to get it via the command line like this:
@raspberrypi:~/tmmiot/MindSphere $
curl -L "https://drive.google.com/uc?export=download&id=YOURIDOFTHEFILE" > mclib.zip
Most of the raspbian images come with a long list of pre-installed software. Mine came with zip for unzipping our mclib.
The next steps are summarizing the C agent build explained in the README
@raspberrypi:~/tmmiot/MindSphere $
unzip mclib.zip
Creating a few directories
#!/bin/bash
mkdir tmmiotAgent
mkdir tmmiotAgent/build
mkdir tmmiotAgent/build/openssl
mkdir tmmiotAgent/build/curl
mkdir tmmiotAgent/build/mcl
mkdir tmmiotAgent/build/agent
sudo chmod 755 tmmiotAgent
Copy the mclib files to tmmiotAgent/MCL_Core
@raspberrypi:~/tmmiot/MindSphere $
cp -r MindConnect_Library_V3.1.2.0 MCL_Core
mv MCL_Core tmmiotAgent/
Building SSH
Downloading SSH
@raspberrypi:~/tmmiot/MindSphere $
wget https://www.openssl.org/source/old/1.0.2/openssl-1.0.2k.tar.gz
Unzip to tmmiotAgent
@raspberrypi:~/tmmiot/MindSphere $
tar xvzf openssl-1.0.2k.tar.gz -C tmmiotAgent/
Configure & install
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/openssl-1.0.2k $
./config — openssldir=/home/pi/tmmiot/MindSphere/tmmiotAgent/build/openssl shared -fPIC
sudo make install
Building libcurl
Download curl and unzip to tmmiot folder
@raspberrypi:~/tmmiot/MindSphere/
wget https://curl.haxx.se/download/curl-7.52.1.zip
unzip curl-7.52.1.zip -d tmmiotAgent
Configure & install
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/curl-7.52.1 $
LDFLAGS=”-Wl,-R/home/pi/tmmiot/MindSphere/tmmiotAgent/build/openssl/lib” ./configure — enable-http — with-ssl=/home/pi/tmmiot/MindSphere/tmmiotAgent/build/openssl — prefix=/home/pi/tmmiot/MindSphere/tmmiotAgent/build/curl — without-libssh2 — disable-ftp — disable-tftp — disable-file — disable-ldap — disable-rtsp — disable-dict — disable-telnet — disable-pop3 — disable-imap — disable-smb — disable-scp — disable-sftp — disable-smtp — disable-gopher — disable-manualsudo make install
Building MCLsudo
Create a shell script build_mcl.sh
#!/bin/bash
OPENSSL_DIR=”/home/pi/tmmiot/MindSphere/tmmiotAgent/build/openssl”
CURL_DIR=”/home/pi/tmmiot/MindSphere/tmmiotAgent/build/curl”
MCL_SOURCE_DIR=”/home/pi/tmmiot/MindSphere/tmmiotAgent/MCL_Core”
MCL_BUILD_DIR=”/home/pi/tmmiot/MindSphere/tmmiotAgent/build/mcl”
if [ -d ${MCL_BUILD_DIR} ]; then
sudo rm -rf ${MCL_BUILD_DIR}
fi
sudo mkdir ${MCL_BUILD_DIR}
sudo chmod 777 ${MCL_BUILD_DIR}
cd ${MCL_BUILD_DIR}
cmake -DCMAKE_PREFIX_PATH=”${OPENSSL_DIR};${CURL_DIR}” -DCMAKE_BUILD_TYPE=Release -DMCL_STATICLIB=OFF -DMCL_USE_LIBCURL=ON -DMCL_USE_OPENSSL=ON -DMCL_CREATE_DOXYGEN=OFF -DMCL_TESTING=OFF -DMCL_LOG_UTIL_LEVEL=MCL_LOG_UTIL_LEVEL_NONE ${MCL_SOURCE_DIR}
cmake — build . — target mc
Create, copy above and change to an executable file
@raspberrypi:~/tmmiot/MindSphere/
nano build_mcl.sh
chmod +x build_mcl.sh
Before we can execute we need to install CMAKE
sudo apt-get update
sudo apt-get install cmake
We have now following directory structure with all libs unzipped and installed to the build directory
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent $ ls
build curl-7.52.1 MCL_Core openssl-1.0.2k
Then we execute the build_mcl.sh
@raspberrypi:~/tmmiot/MindSphere/
sudo ./build_mcl.sh
Build target mc — okay. Let’s check the dynamic dependencies:
@raspberrypi:~/tmmiot/MindSphere $
ldd tmmiotAgent/build/mcl/build/Release/libmc.solinux-vdso.so.1 (0x7efa6000)
/usr/lib/arm-linux-gnueabihf/libarmmem.so (0x76f35000)
libssl.so.1.0.0 => /home/pi/tmmiot/MindSphere/tmmiotAgent/build/openssl/lib/libssl.so.1.0.0 (0x76ed0000)
libcrypto.so.1.0.0 => /home/pi/tmmiot/MindSphere/tmmiotAgent/build/openssl/lib/libcrypto.so.1.0.0 (0x76d44000)
libcurl.so.4 => /home/pi/tmmiot/MindSphere/tmmiotAgent/build/curl/lib/libcurl.so.4 (0x76d01000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76c6d000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76b2e000)
libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x76b1b000)
libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0x76af4000)
/lib/ld-linux-armhf.so.3 (0x76f78000)
Shows us that all libssl, libcrypto and libcurl are referenced by our own builds.
Building the custom agent
We create our agent folder and Make file
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent $
mkdir agent
nano CMakeLists.txt
We save following script for CMakeLists.txt
CMAKE_MINIMUM_REQUIRED(VERSION 3.5 FATAL_ERROR)
PROJECT(TmmiotAgentAppplication LANGUAGES C)
SET(CMAKE_C_STANDARD 99)
SET(CMAKE_C_STANDARD_REQUIRED ON)
FILE(GLOB SOURCES *.c)
LIST(APPEND AGENT_SOURCES ${SOURCES})
SET(MCL “/home/pi/tmmiot/MindSphere/tmmiotAgent/build/mcl/build/Release/libmc.so” CACHE INTERNAL “MCL” FORCE)
SET(MCL_INCLUDE_DIRECTORIES “/home/pi/tmmiot/MindSphere/tmmiotAgent/build/mcl/include/” CACHE INTERNAL “MCL_INCLUDE_DIRECTORIES” FORCE)
SET(AGENT_OUTPUT_DIR ${CMAKE_BINARY_DIR}/build)
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${AGENT_OUTPUT_DIR})
SET(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${AGENT_OUTPUT_DIR})
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${AGENT_OUTPUT_DIR})
ADD_EXECUTABLE(${PROJECT_NAME} ${AGENT_SOURCES})
TARGET_INCLUDE_DIRECTORIES(${PROJECT_NAME} PUBLIC ${MCL_INCLUDE_DIRECTORIES})
TARGET_LINK_LIBRARIES(${PROJECT_NAME} ${MCL})
We need to copy the available mcl headers to our build directory
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/MCL_Core $
cp -r include/ /home/pi/tmmiot/MindSphere/tmmiotAgent/build/mcl/
There are example source code which we will adapt to onboard our pi and then send timeseries data. Copy the example files for the onboarding to the agent
pi@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/agent $ ls
callbacks.c callbacks.h CMakeLists.txt onboard_dsc_upload.c
Adapt the onboad_dsc_upload.c file with your initial access token and further tenant information.
I have adapted the first code block from the example:
if (MCL_OK == code)
{
configuration->mindsphere_hostname = “https://southgate.eu1.mindsphere.io";
configuration->mindsphere_port = 443;
configuration->mindsphere_certificate = mindsphere_certificate;
configuration->security_profile = MCL_SECURITY_SHARED_SECRET;
configuration->proxy_hostname = NONE;//”http://192.168.0.1";
configuration->proxy_port = NONE;//3128;
configuration->proxy_type = MCL_PROXY_HTTP;
configuration->user_agent = “tmmiot agent v1.0”;
configuration->initial_access_token = initial_access_token;
configuration->tenant = “yourTenantName”;
configuration->load_function.shared_secret = (mcl_load_shared_secret_callback_t) custo$
configuration->save_function.shared_secret = (mcl_save_shared_secret_callback_t) custo$// Initialize mcl communication with the configuration.
printf(“Initializing mcl communication : “);
code = mcl_communication_initialize(configuration, &communication);
printf(“%s.\n”, mcl_log_util_convert_error_code_to_string(code));
}
And commented everything after the onboarding (no configuration creation…) simple onboarding.
/*
// If onboarding is successful, or the agent is already onboarded, initialize a store.
...
save_data_point_ids(data_point_id_1, data_point_id_2);
}
*/
Executing and building the agent
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/build/agent $
cmake /home/pi/tmmiot/MindSphere/tmmiotAgent/agent/cmake -build . -clean-first
Executing via
pi@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/build/agent $
./build/TmmiotAgentAppplication
The agent was onboarded successfully and the file registrationInformationFile.txt was created.
Creating the agent mclib configuration and data mapping via GUI
We have all the data points configured in order to later visualize the data we need to map the datapoints into our asset referencing the same data structure
Configuring the data (timeseries) upload
First we move our adapted onboard_dsc_upload.c to an archive. And then copy the timeseries example to our agent folder.
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/agent $
mkdir archive
mv onboard_dsc_upload.c archive/
cp ../MCL_Core/examples/timeseries_upload.c ../agent/
Further we adapt the timeseries_upload.c file and adapt it with our configuration and data ids.
char *configuration_id = “51830356–7180–4a0d-9f26–4681c623f89e”;
We start with just entering two datapoint T and H. The executable will then add two data points to the timeseries. You need to change the code on this point to allow initial token transfer (the example has a bug here). My example on github is changed already.
// If exchange returns unauthorized, then get new access token.
if (MCL_UNAUTHORIZED == code || MCL_NO_ACCESS_TOKEN_EXISTS == code)
{
Change the CMakeLists.txt file and rename the executable
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgent/build/agent $
cmake /home/pi/tmmiot/MindSphere/tmmiotAgent/agent/
cmake — build . — clean-first
The mclib asset gets all the necessary data points and mappings to the “visualizing”-Asset. All that is a bit too much drama for just a prototype. For our later product we would need to create a custom compiled agent for the hardware architecture. But for now we will proceed with the node.js library.
Nodes.js MindConnect library onboarding and timeseries upload
Compared to the C experience above this feels like a walk in the park. First the data mappings need to be set up like in in the above example with the c agent.
Installing the library
As the node.js library just got released under the MIT licence to the public you can easily install it via npm package manager. Create a new folder for your agent.
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgentNodeJs/$
npm init -y
npm i @mindconnect/mindconnect-nodejs -save
Create the node.js agent
I have created a script which sends every three minutes a new datapoint of our sensors.
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgentNodeJs/$
nano stream-pi-sensor-data.js
copy to stream-pi-sensor-data.js
Creating your agentconfig.json
As outlined earlier we need to store the boarding configuration. Which gets called via the above script. If the onboarding is successful a hidden subfolder get created .mc
which stores the shared secret.
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgentNodeJs/
nano agentconfig.json
just copy your onboarding information into that file.
Adding the hardware sensor libs
We can integrate the libraries/packages for the hardware sensors now
@raspberrypi:~/tmmiot/MindSphere/tmmiotAgentNodeJs/$
npm install sds011-client -save
npm install rpi-dht-sensor -save
Voilá — with root execution sudo node stream-sensor-pi-data.js
this is much easier compared to the above C agent steps.
I like your comments, corrections and suggestions 👌.