IoT can be tough — where do you start?

One of the big problems anyone setting out to build an Internet of Things type product will face is the huge amount of things to consider, differing technologies and options available. There are many solutions to every aspect of the technology stack, so where do you start?

Over the next few weeks, I’ll be writing about some of the considerations we faced at Lattice Research when setting out to build our Internet of Things framework, and while your mileage may vary, I can give a few pointers on how you could approach these problems.

As I complete each post, I’ll add some links in-line and at the end of the article. But here is a broad overview of the questions I’ll be touching on.

What is the Internet of Things?

“Internet of Things” is a great buzzword — but fundamentally it’s just a logical intersection of two fields — client/server software development and accessibility and low cost of modern electronics.

Actual applications can vary greatly — most people are aware of connected thermostats such as Nest, or maybe of Philips Hue lighting or Belkin smart sockets — but it could also refer to wearable devices, sensor networks, control systems or many other things. It’s about gathering data from the real world, making it available via the web, analysing it and /or allowing the user to make a physical change by using an app or browser.

So, with that in mind — it’s easy to see where to start. How do you collect data or change things in the real world?

Physical Issues and Embedded Environment

To do something in the real world you’ll need to make or buy in some physical hardware. Eventually, this type of hardware may become commoditised. But for now, assuming what you want to do isn’t already available off the shelf, how would you make it?

There’s plenty of big vendors vying for market share in the IoT. Intel, Microsoft, QT, Arm, and many others are making big bets on the future of IoT. So, a good approach might be to hire some embedded developers and electronics engineers and jump right in — if budget were not an issue.

However many of these companies (and others) are also targeting “hobbyist” type devices which are very accessible for anyone with even the smallest exposure to electronics. These can be a great starting point for doing some simple proofs of concept or simply seeing what is possible.

Intel Galileo, Raspberry PI, Beaglebone Black, Arduino are all products aimed at the “maker” space. There’s plenty of tutorials available for all of them, and they’re so low cost that most people will just buy another one if they break them.

Of course, it’s a long road from a few simple mockups using these types of devices to a properly engineered, full fledged product. But they can make it easy to prototype a concept quickly and then get into the real development faster.

Will you have a single device or multiple devices networked together in the real world? For example, if you want to gather data from a remote sensor, you’ll probably need to navigate some wireless technologies. Where do you start — IP/Wifi, Bluetooth, Zigbee, Z-Wave, EnOcean, 6LoWPAN, Thread, and many others?

How about battery power? That leds to a whole different set of challenges. What about product certification — CE / UL testing?

Finally, you will need to think about how to enclose your electronics to use them in the real world. How will this look? How do you design it — or do you outsource this? Can you 3D print the prototypes or produce them using reductive manufacturing (plastic milling?) How about when it gets to production scale — will you need to tool for this and build in bulk?

Once you have decided what route you’re going — microcontroller based / microprocessor based / ARM or Intel based (and put the resources in place to engineer it), you’ll have to think about the environment too.

For example, your embedded device(s) will need to run control logic, connect to cloud, provide local data storage, possibly run a touchscreen or interface, have robust watchdog functionality to prevent unexpected lockups, and make sure everything is running correctly. Should you target Android, Windows or Embedded Linux?

I discuss some of these considerations in more depth in part 2 — Physical issues and embedded environment and part 3 — Wireless and Battery.

Topology, Your Data, Languages and Environments

So, you think you know what you need to do with your hardware. Where do you go from there? How should the rest of the system work? How is it arranged or laid out? In other words, what should the topology be?

If you work through the services you need it should be easy enough to arrive at a rough topology pretty quickly. But that is the easy step — as there are many different ways to handle each component. All your choices must support fast implementation, easy development and maintenance, and a high level of scalability.

How you handle the data in your system is equally important. You could represent what you’re doing in many different ways. You might represent it using OO on the local device but ultimately as a flat record in a database in the cloud. Will your data scheme never change? This would make it easy as you could implement some very rigid schemas. But what if it does change in future — will you have to make major changes everywhere to support it? Or maybe you need to try and find something a bit more fluid. Regardless, you will need to think this one through carefully.

When it comes to languages to use, the initial approach is to see if what you know already will do what you want. But if it doesn’t, there is a good chance there is a better fit out there. You’ll have to navigate whether to use unmanaged or managed code (you may need to do both), whether to use traditionally strong languages (Java, C++, C#, others) which are well supported or some more modern, web centric environments, which are a bit more freeform (Python, Node, Ruby, etc).

Should you look at using languages which do synchronous processing of code, or asynchronous, or both? The code running in the cloud will also need to be scalable — will this be done by scaling vertically (more powerful server), horizontally (more servers), or trying to maximise resource usage using threads?

Is it possible to find a language or environment where you can use it everywhere (or nearly everywhere) — in the embedded device, in the cloud, and for the user interface — e.g. a “Full stack” approach? This could reap benefits in a more easily developed and maintained code base.

Note — I take a closer look at these issues in part 4 — Topology, Data Model and Languages

APIs and transports

One of the tricks in the IoT is moving data seamlessly between different locations. Real world to cloud, cloud to user. So APIs and transports are at the heart of any IoT implementation.

There are some widely used web service types. HTTP REST is an obvious choice, and options such as SOAP/WSDL are also used, although less widely used as REST has gained popularity. But these could be thought of as “push/pull”, e.g. when you need to upload changes to the cloud you will need to specifically “push” them, and to pull down changes you will need to specifically “pull” them.

This is fine if your data changes are slow moving, but what if you need something that is more real-time or have specific needs to push out changes from the cloud to clients instantly? Well — real time transport of data needs something else.

If you think of it, it’s just messaging right? So maybe you could use a framework for real time communications, e.g. MQTT, XMPP, ØMQ, RabbitMQ, or WebSockets.

Perhaps your APIs could be a mix of both. For bulk getting or setting data, use a RESTful API, and for instant updates, pass smaller chunks of data using a real time transport?

Note — I take a closer look at these issues in part 5 — APIs and data transports

Cloud — Server side — SaaS vs PaaS vs IaaS

So you have your hardware, a data model and a way of passing the data about. But your cloud side services will need to run somewhere.

It’ll need to handle many different pieces of functionality — It will have to allow the hardware to connect, authenticate HW and clients, provide security, serve resources, run business logic, etc etc.

So how and where will this run? Should you buy some servers and put them in a rack in a datacentre (Self hosted)? How about using Amazon EC2 or Google compute or Azure to run some virtual servers (Infrastructure as a Service)? Does it make more sense to run your cloud backend server on some containerised type services like Heroku, or to use something like Salesforce.com to host your platform (Platform as a Service)? Or maybe a mix of these for different needs?

Note — I discuss this in more details in part 6: Cloud services — IaaS, PaaS, SaaS, build your own

Storage

At some point, you will need to store your data in the cloud. Traditionally developers would have pulled out a Relational DB such as SQL server, MySQL or PostgreSQL and designed some normalised tables to handle the data.

However — while this is still a good option for many applications, there are alternatives. NoSQL data stores are becoming popular as they can be more flexible.

Popular NoSQL data store types are Columnar / Big Table, Key/Value Store, Object or Document DBs, Graph DBs. These all have advantages and disadvantages, and will need to be chosen to offer the best support for your particular application and data model.

Should you choose a DB as a Service or self hosted? If you buy a DBaaS, how do you know your data is secure? Alternatively — if you host it yourself how do you ensure reliability, integrity and do you have the knowledge to make it secure?

Note — I discussed DBaaS in part 6, which is linked above, but I delve into the databases themselves along with file storage and handling historical data in Part 7: IoT considerations — storage and database, SQL, NoSQL, historical data

Frontend and Apps

You need to think carefully about your user interface. How is it to be built and served to the client? Should it be server side generated or run client side? How about apps? Do you need to build separate apps for IOS/Android/Windows Phone etc?

Generating it server side is a well known way of doing things. It can be secure, but needs a lot of resources to scale. It can also be slow, if the user has a slow internet connection, or if there are a lot of requests to the server or changes between different pages. It also cannot handle real time data very well, unless you add some client side code to pull in or update changes on the UI as necessary. Of course, you will still need to build apps from scratch.

Well, if you’re already doing a little client side javascript — could you run the entire UI client side using HTML5? Simply serve all the application resources at load up and the only subsequent server requests are for data from your API?

There are a lot of javascript frameworks for doing this — Backbone, Ember, Knockout, Angular, React, or you can write your own code from scratch using standard javascript/JQuery. Advantages for the user — really snappy UI, works well even on slow connections. Disadvantages for you — more complex topology, more security considerations (XSS? CORS? Authentication tokens/OAuth/Etc?)

But using a client side ui may have an additional advantage — If it is well designed it icould be easy to package it up into a multi-platform app using Phonegap / Cordova or Ionic, or Electron.

Note — I discuss this in more detail in Part 8 — Frontend and apps

Big Data and Analytics

You will obviously need to store some data historically in order to be able to query past states of your system.

How much data do you need — do you need to keep all your data as is for an extended period of time? If so, your storage needs will be vast and it will be slow to query.

Can you parse out only the important values and keep those only? In this case, you will need to think about how it is transformed. What data is actually of use? How are you going to store it and query it quickly?

Once you know how you are going to historical data, you will need to decide what you’re doing with it. Will you have tasks to analyse it for specific trends or errors? If so, how and where will these run?

Do you need to slice and dice it on the UI for the user? There are plenty of javascript graphing libraries to let you do this — for example D3.js, Highcharts, Morris, Flot. Google charts, etc etc — perhaps one or more of these could help.

Licensing

Even if you navigate all of the above and come up with a workable solution, there are other considerations.

Chances are, you’ll want to use a lot of open source code to accelerate your development process. But if this is a commercial operation, and you’re planning to build some proprietary IP, then beware — as open source is not all equal.

Different licenses hold the developer to different standards. MIT/BSD/Apache type licenses are free to use/modify, all that is required to comply with the license is attribution that the original code was used.

However other “non-permissive” licenses — particularly GPL — are more onerous. They may require you to release your modified source code to anyone who gets a copy of the binaries.

If your business plan revolves around giving your code away and making money from the services associated, then this may be fine — but may be a big no if you are trying to build a commercial product!

Note — I take a closer look at these issues in part 9 — Open source and licensing

Conclusion

The Internet of Things can be tough. The technical effort is significant, knowledge required is vast, the amount of time required to develop and bring such a product to market can be prohibitive.

I’ll be discussing some of the areas touched on in detail over the next few weeks — as I do I’ll add the links below. While I may not say “use X,Y and Z”, I hope it might stimulate some thought about how best to get started on your next IoT project!

Further Reading

Part 2 — Physical issues and embedded environment

Part 3 — Wireless and Battery.

Part 4 — Topology, Data Model and Languages

Part 5 — APIs and data transports

Part 6 — Cloud services — IaaS, PaaS, SaaS, build your own

Part 7 — IoT considerations — storage and database, SQL, NoSQL, historical data

Part 8 — Frontend and apps

Part 9 — Open source and licensing

—

Des Flynn is CTO of Lattice Research, who help companies to design, build, deploy, operate and service innovative and cost-effective IoT control systems to meet their customer’s needs. More information at www.lattice.ie