Simple and Robust {Computer — Arduino} Serial Communication
TL;DR:
Arduino built-in functions for sending/receiving data are not very handy and sturdy. We introduce a protocol to communicate (using serial port, bluetooth or sockets) with the Arduino (but not only) in a simple and robust way. We also release examples in various programming languages so you don’t need to bother about how to implement it ;).
In this post, we present a robust serial communication protocol to interface an Arduino with a computer (e.g. a Raspberry Pi). This protocol was designed to be simple, flexible and easy to use. We also release implementations in C Arduino, C++, Python and Rust.
Note: This communication protocol was used on robots for the French cup of robotics (where we finished 4th!) and recently on a AI Racer for both teleoperation and autonomous mode.
The github repo:
Background Knowledge
To fully understand this article, you need only some notions of programming (what is a bit? what is a signed int?), and a rough knowledge of what is Arduino (is it a fruit?).
Outline
In the first part, we briefly present what is serial communication and why do we need it. Then we introduce the “classical” approach to communicate with the Arduino and show its limitations. In the second part, we give an overview of the simple and robust communication protocol and then go into the details.
I. Problem Overview
What is serial communication and why do we need it?
Serial communication is the process of sending data one bit at a time, sequentially, over a communication channel or computer bus. Source: Wikipedia.com
This is the general definition of serial communication. In this article, serial communication will only refer to the fact of communicating using the serial port of the Arduino/computer.
To answer the why question, let’s start with an example. Imagine you have a temperature sensor that is connected to an Arduino and you want to retrieve the values to your computer. One simple solution is to connect the Arduino to your computer via USB and send the data using the serial communication (Arduino already has built-in functions to communicate with the serial port).You may also want to remotely control a motor, connected to the Arduino board. In that case, you need to send orders to it so it will change the motor speed: that is what was used to teleoperate a tiny racing car:
As you can see, serial communication is useful when you want to receive or send data from or to the Arduino; when you want to interface it with a computer, a Raspberry Pi or even another Arduino board.
Arduino Serial: Crash Course
Before we dive into the traditional approach to communicate with the Arduino, we present a quick recap on the few functions you need to know to understand the code. (If you are already familiar with it, you can skip this part)
To use the Arduino serial port, there is a built-in object called Serial. You first need to set the speed at which your computer will communicate with the Arduino (called the baudrate), it has to be the same on both sides. In your program, inside the void setup() function, you set the data rate in bits per second (baudrate) using Serial.begin(BAUDRATE). The valid values of BAUDRATE (e.g. 9600, 115200) are listed in the Arduino documentation.
The other methods you need to know about are:
Serial.available(): get the number of bytes (characters) available for reading from the serial port.Serial.read(): reads incoming serial data. It reads one byte (8 bits).Serial.write(): writes binary data to the serial port. This data is sent as a byte or series of bytes (byte array)Serial.println(): prints data to the serial port as human-readable text with a line break at the end
“Classic” Approach — Common Approach
The common approach with the Arduino Serial, is to send characters only, that is to say, send one byte (or 8 bits) at a time, because Serial.write()only allows to do that. Another approach for debugging is to use the Serial.println() method, that permits to write strings.
Let’s take the example of an autonomous car. We assume we have an on-board computer (e.g. a Raspberry Pi) connected to a camera and that does all the image processing and time-consuming computation. This computer is connected to an Arduino board that sends order to the motors (direction and speed). The computer needs to communicate with the Arduino to adjust the motor commands (should we go left/right? should we stop/go forward?)
The Arduino code that listens to the computer may look like that:
That works, sure, but that is not very handy. You may have the intuition that if we want to do something more complex, we are going to quickly reach the limits of that technique.
Limitations
So, what’s wrong with the previous technique? A lot of things !
First, there is a lack of readability: looking at the code, you have to understand that the character ‘a’ corresponds to the order of going forward. You have to make sure that at every place, where you want to read or send a command, the same character is used. This becomes clearly unmaintainable and error prone when the number of possible orders grows, or when you want to interface with different programming languages.
Second, it is not very flexible. Let’s say that instead of sending just a “GO_FORWARD” command, you want to send a speed value (between 0 and 100, 100 representing the max speed), or send encoders value (i.e. a count of wheel rotations to estimate a relative position) to a remote computer. With the previous technique, you can only send one byte a time — values in the range [-127, 128] — so you may end up sending strings instead, which is quite inefficient. For example, if you send the number 32768 (which is 2¹⁵) as a string, you need to send 5 bytes (5 characters or 5*8=40 bits) instead of 2 bytes!
But the main issue is not there. The “classic technique” for arduino serial forgot one important thing: the limited serial buffer size on the Arduino.
To keep it short, a buffer is like a waiting queue with a limited capacity. It keeps the incoming messages until they can be processed.
Because it has a fixed capacity, a limited size, if we send too much messages in a short amount of time, some messages will be lost! When you are doing teleoperation, that is not a big deal. However, if you are using Arduino with servomotors to grasp an object, and you need motor commands to be executed in a specific order, this can make a huge difference.
Also, from my personal experience, if you don’t account for the limited buffer size and flood the Arduino with messages, it will stop responding and sometimes reset. So, even for teleoperation, the method that we are going to detail is useful.
II. Serial Protocol
In this section, we first give an overview of the serial protocol, outline its design and then dive into the internal using a concrete example.
What is a communication protocol?
A communication protocol defines how two entities (e.g. two computers) can communicate. That is to say, it gives common rules — start the conversation with “hi”, end it with “bye” — so the two entities can understand each other.
General Idea
In the protocol we propose, there are two main phases: connection and then communication. The connection phase ensure that the Arduino and the computer are connected (e.g. usb cable is not disconnected, serial port is readable, …) and there is no communication problem (for instance, communicating at different speed). The communication phase is the normal one: the connection is established and the Arduino/Computer are sending/receiving messages.
Messages are composed of two parts: the order message and the parameters.
The order message tells the Arduino the type of action we want to perform, for example if we want to change the speed of the motor, we will send an order of type “MOTOR”.
The parameters messages contains all the relevant information for the action. In the case of a “MOTOR” order, the parameter is an int representing the actual speed we want to achieve (e.g. 100 for full speed, -100 for full speed going backward, …).
Solving buffer overflow
For now, we addressed the two first drawbacks of the traditional method. Still, the main issue of the limited buffer size remains.
To avoid filling up the buffer, one simple solution is to acknowledge the reception of each messages, and avoid sending new messages if the previous one were not received. That way, we won’t have a buffer overflow and no message will be lost!
In practice, we start with an certain amount of tokens, we lose one each time a message is sent, and gain one each time an acknowledgment is received. If the number of tokens goes to zero, then you cannot send messages anymore. You have to wait for an acknowledgment. This type of mechanism is called a semaphore and is quite useful when you have a resource with limited capacity (e.g. connection to a database).
That’s it for the communication protocol, there is no additional rules. To summarize it:
The Communication Protocol in a Nutshell
- Connection, you make sure the Arduino is connected
Then, for each message you want to exchange, you send:
2. The type of order (e.g. “MOTOR”)
3. The parameters (e.g. speed)
And then you wait for an acknowledgment of the Arduino to send new orders.
Keeping those rules in mind, in the next part, we dive into the implementation details of that protocol and present a concrete example in various languages.
Implementation Details
To ensure compatibility between the different entities that communicate, the finite set of possible “orders” is defined in an Enum:
Each order is associated to a byte, so you can have up to 256 different orders.
NB: those are only a suggestion, you can use your own name/type of orders/parameters
Parameters are represented as byte array, you can send int8, int16, int32, that is to say 8-bits, 16-bits or 32-bits long integers. That way, you can send numbers between -2,147,483,648 and 2,147,483,647!
In the different implementations we provide (for all the supported languages), the methods for sending data start with write_ followed by the type of data. For example, to send a 16-bits int, you need to use the write_i16() function.
Similarly, for reading data, we provide functions starting with read_ followed by the type of data. Again, to read a 8-bits int, you will use the read_i8() function.
Let’s practice with a concrete example.
Example
In this example, we want to change the direction and speed of the racing robot: we want the car to go forward at 56% of the maximum speed and steer with an angle of 156°.
In Python:
In C++:
In Rust:
As you can see, it’s pretty straightforward to send those orders. Also, the three implementation are quite close. Once you know how to use the serial protocol in one programming language, it is easy to use it in another one.
NOTE: we voluntary skipped the connection part for simplicity. However, in the github repository, we provide complete examples, including examples with bluetooth and sockets for Python.
Command Parser
As a bonus, I also developed a command parser in c++ that allows you to send orders interactively to the Arduino in the terminal. It can be very useful when you want to debug or calibrate a servomotor angle.
Possible Improvements ?
Even though the proposed communication protocol overcomes built-in Arduino methods issues, it has some limitations:
- there is no mechanism to retry if a message is not received (e.g. as in the TCP protocol)
- you need to have the same Order Enum (list of valid orders) and way of encoding/decoding parameters on both sides (Arduino/Computer)
- there is no checksum (unlike rosserial), so you cannot be sure that the message was not corrupted (still, the low-level serial communication protocol on which this one is built has a parity bit)
- this protocol does not account of the endianness of the system (it only works for little endian systems (majority of modern computers) for now)
- there is no method to send/receive float. However, this limitation can be easily overcome: you first scale and convert the float to an int, then you just need to rescale it and convert it back to a float at the reception. In practice, 32-bits int are sufficient for most cases.
- if out of sync after the connection (“hello” exchange), then it breaks. In practice, I never experienced that issue.
Conclusion
After showing the limitations of the traditional approach to communicate with Arduino, we presented a protocol that overcomes most of those issues.
Don’t hesitate to look at the Github repository, where we provide complete examples (you can also submit a pull request if you implemented that protocol in a new language):
We hope you enjoyed this article, please share it if you liked it!
If you have any question, any remark (or if you have found a typo), please leave a comment below ;).
Acknowledgments
I would like to thanks Dara Ly for the original idea of communicating with the Arduino via a command parser, and Xuan Zhang for fixing Arduino limited buffer issue.
Images Credits: Icons from Flaticons, Raspberry Pi image from Wikipedia
