IoT considerations — Topology, Data Model and Languages
Continuing the series of things to consider when starting on an IoT project, let’s look at the topology of your system, how your data is represented, and languages and environments.
If you missed the previous articles, the overview with links to all the other content is here.
Topology
So, you’ve delved into your physical hardware, embedded environment, and decided if you’re using some wireless technology and/or battery powering your device(s).
But what next? What does the rest of your IoT system look like?
Well, when you think it through, unless you’re doing something particularly specific — e.g. peer to peer — then the topology is pretty well defined.
You’ll need some cloud services, user interface, apps, database(s), analytics, authentication and some administrative tools. Take a minute or two to sketch down what you’ll need and you’ll probably come to a conclusion it looks something like this:
Ok, that was easy! But defining the general topology is a breeze compared to figuring out how each part of it is to be done. There are many different ways to handle each component — and there are hundreds of competing technologies which could be used to implement each aspect.
So instead of focusing on the what, focus on the why. Each aspect of the topology should be chosen to fit three key considerations.
Rapid implementation
Time to market is a big issue. If you have a small team, this is essential. The difference between delivering or not could come down to how quick it is to build.
If you have a larger team, choosing technologies which are quick to build in lets you move faster and focus more on what you’ll be delivering to your customers rather than how.
Easy development and maintenance
Building the infrastructure is great, but if it is an unsupportable mess then it is no good.
Ease of development goes hand in hand with the speed of development, but make sure you choose technologies which require little or no ongoing maintenance or else you will face major problems down the line.
Scalability
If you’re only deploying one instance of your IoT system, then you can cut a few corners to just make it work. But what if you’re deploying ten, a hundred, a thousand or a million instances?
You need to pay close attention to this. If your infrastructure is slow or bloated, requiring a lot of resources to run, then you will need to quickly add more hardware, extra complexity and more cost.
If you think about it, all three of these considerations are interlinked — and are something you must bear in mind when deciding on every aspect of your technology stack.
How do you represent your data
So you’re starting to get some idea of how your system might look — but before you press ahead take some time to think about how the data in your system is represented.
This is essential as it will affect all aspects of your application — how the control code is written and runs, how the UI parses it to represent the system to the user, and how it is transferred from the real world to cloud, to apps, and vice versa.
A good data model / way of representing your system will allow the implementation to go more smoothly. A poor model will hinder you and cause delays in implementation, and extra ongoing maintenance.
Are you representing a simple application with just a few tracked parameters? Or are you trying to represent a highly complex system with hundreds of changing parameters, or where each deployment is different?
You could represent what you’re doing in many different ways. You might represent it using OO on the local device but ultimately as a flat record in a database in the cloud. Or you might look for a way of representing it the same way in both places.
Will your data schema never change? This would make it easy as you could implement some very rigid schemas. But what if it does change in future — will you have to make major changes everywhere to support it?
What if your data schema is very likely to change or evolve in future? Or what if it needs to be flexible — maybe each system is slightly different? In this case you’ll need to try and find something a bit more fluid.
Finally, there are many ways of representing or passing data as text — XML, JSON, CSV / Flat file — could one of these be of use to you? Regardless, you will need to think this one through carefully.
Languages and Environment
So you’ve thought through the topology and data model, and you’re starting to see an outline of where you need to go. But how will it be implemented?
What you already know -vs- a better fit?
It is tempting to look at what you already know and just press ahead with that.
Starting out on the Lattice Framework, the language I was personally most proficient with was C# — but the obvious embedded environment of choice at the time was embedded linux. These two didn’t really mesh well together, and besides, most fancy modern UI is based on Javascript. So, we started looking for alternatives.
We could run some C on our node devices, C++ on the embedded device, some C# on the servers and bake in some Javascript for extra UI functionality. How about apps — well that means Java and objective C or Swift too, right?
We could see how we could build it — but — if we had stayed the course and stuck with what we knew we would have ended with at least six different languages we needed to be expert in, and we also needed to build rapidly, scalably and with a small team.
So — we chose a better path. In the short term, it meant a couple of months re-tooling and re-skilling, trying to find a better match, but it was a good decision long term.
We now use just two languages right throughout the Lattice Framework, and those are syntactically so alike that anyone proficient in one will generally be able to work in the other too. This greatly simplifies everything — development, maintenance, code re-usability, team size, hiring, etc.
The takeaway is this — don’t just jump in with what you know — it may not be the best fit. Take some time to analyse.
Unmanaged vs managed code,
Unmanaged code is code which is directly compiled into machine code for a specific architecture. In it, you more or less directly assign space in memory when you create variables. If you forget to de-allocate the space after, then eventually you will run out of memory. A well known example is C/C++.
The code is lightweight, fast and very efficient — but the onus is on you as the programmer to be very precise — or else face things like buffer overflows, type mismatches and memory leaks.
Managed code is code which is not necessarily compiled fully for a specific architecture — instead it is built into an intermediate type of code which can run through an interpreter on different architectures. It is generally more forgiving as a lot of the specific considerations are abstracted away.
It takes care of memory by automatically tidying up unused variables when they are no longer needed. Type mismatches are less of a problem as the interpreter will generally handle them in a known way. It’s a lot quicker to write in, but can run more slowly.
You may end up using both — on embedded devices you might want to run unmanaged code as it is faster and lightweight. But for your cloud services, apps etc, you are far more likely to write managed code as it is quicker and easier to develop.
Asynchronous -vs- Synchronous processing
Should you consider using languages which do synchronous processing of code, or asynchronous, or both?
Synchronous processing is where each line of code runs one after the other, and the next line won’t run until the previous line completes. Conceptually, this is simple to write and follow. Historically, most code was written this way.
Asynchronous processing is much more complex but may be needed, particularly for your cloud services. Any call to a remove device, storage, database etc, is handled differently in this case. The code will not wait for the response before going to the next line of code, instead you will generally register some sort of a “callback” and deal with it when the data comes back.
Conceptually asynchrounous processing is tougher to follow but absolutely essential in the modern web.
For example — think of a web page requesting new data from a server using Javascript. Should the page lock while we wait for the data to come back? Absolutely not! Similarly, if your server receives a request for some data, should we lock up the server while we wait for data to come back from the database? No! Instead we should keep handling all the other requests in the meantime instead of queuing them.
Either is valid — and both have their pluses and minuses. So which should you target — or both? — and how does this affect your choice of languages?
Scalability
How do you plan to scale the code you are running in the cloud?
Will you do it vertically (more powerful server)? How well will this work — will there be bottlenecks in the code which will not scale with more power (e.g. large numbers of synchronous external database calls?)
How about horizontally (by adding more servers)? Is your code going to be happy running in multiple instances? Or will it fall over if some requests go to one server and others to the other? Do you need to share quickly accessible run-time data between instances — for example by using a cache such as redis or similar?
Finally — there may be cases where you want to maximise the resource usage in your current server — you may have some synchronous code which you want to run in the background, without blocking everything else. Could you do this using threads?
Traditional vs Web-centric environments
“Traditional” languages such as Java, C#, etc are extremely well known, well documented and have a lot of good, mature programmers who approach it from a computer science point of view — the solution needs to be elegant, readable, well documented as well as working well.
Many newer web-centric server environments now exist (Node.js, Python, Ruby, Others) and as they are perceived as “cooler” they are more likely to be of interest to younger, newer developers. Throw in some asynchronous processing and the results can seem a little more… freeform.
This is of course a gross oversimplification. Both groups of languages are equally powerful, and I’m committing some heinous stereotyping of the user groups. Of course, “traditional” languages can be applied to good effect on the web if well used, and modern web-centric languages can be used to create equally well engineered, elegant, well documented solutions.
So, if all else was equal you could go either way, right? Well — that “freeform” aspect of the web-centric languages is one of the things that attracts younger developers — they can be very rapid to prototype and build in — and as a result there are many hundreds or thousands of small, easily used open source libraries available for them — which makes it even quicker still.
So make sure explore both options before you commit to either.
“Full stack” approach?
So, after analysing all that, can you come up with a solution which meets all your needs, with as few languages you can get away with — e.g. a “Full Stack” approach?
Is it possible to find a language or environment where you can use it everywhere (or nearly everywhere) — in the embedded device, in the cloud, and for the user interface?
At least make sure you try — it could reap benefits in a more easily developed and maintained code base, with a high level of code re-usability, and the ability for anyone on your development team to jump into any aspect of your stack and feel at home.
Conclusions
It may seem easy at first — but there are a lot of choices when it comes to the topology, data and languages used. You could take any number of approaches to each aspect. Just make sure to think through the why first and not the how!
Next time out, I’ll delve into the thing which ties it all together — APIs and data transports.
Update — Part 5 is now available: APIs and data transports
—
Des Flynn is CTO of Lattice Research, who help companies to design, build, deploy, operate and service innovative and cost-effective IoT control systems to meet their customer’s needs. More information at www.lattice.ie
