Arduino Remote Uploader Redux

In early 2015 I started thinking about how I could upload Sketches to a remote Arduino. I had to lug around my notebook with a cable to update Sketches on my Arduino projects, scattered about the house. The end result is inertia took over I didn’t update my projects often: improvements came to a halt and bugs didn’t get fixed.

In the non-embedded world updating software on a remote computer is just obvious. But Arduino is not a computer and things are different in the embedded world; in the case of Arduino, it can only accept programs over a serial line, and it can’t update itself. Kinda lame right? But it costs just a few dollars, consumes only about 0.3W (compare to Raspberry Pi ~3–5W) and with power management it can run for years on a tiny battery.

I started looking at how the Arduino IDE programs an Arduino over USB, knowing that if I could mimic that process on a microcontroller, I would have a solution. So work began and somewhere around April I considered my project mostly complete. I built a few prototypes and even collaborated with a vineyard owner who wanted to use it for automating various aspects of wine production: pumps, controllers and doodads. Then, life got in the way and the project stalled.

This year I found some extra time around the holidays and decided to re-up on it. I had built an XBee garage controller years ago. Actually the garage controller was one of the motivating factors for this project. I decided this would be a good first candidate for my remote firmware solution.

One area of the project I overestimated was the ease in producing prototypes. I had never built a circuit on a prototype board before. How hard could it be? Well, the first board didn’t function after I had spent hours assembling it. It took a few more hours to troubleshoot, which is much more difficult to do when you can’t just pull a wire out or alter the circuit easy, like on a breadboard. But anyway, lots of learning, so that was good.

I looked briefly into what would be involved in ordering printed circuit boards, but it’s fairly pricey with low volume. I also looked into DIY PCB methods, including PCB milling and toner transfer. If I had a Shapeoko then I could mill boards and life would be awesome, but that’s a lot of cash. The toner transfer method looks unpleasant. You’d have to manually drill holes for all your components and that seems worse than dealing with prototype board. Plus messy chemicals and other hassles. In the end it turns out I actually enjoy building the prototypes when I have the time, perhaps like some enjoy knitting.

I powered up the XBee prototype board from almost two years ago. I couldn’t even recall if it worked. After some fiddling around with RXTX headaches, I ran

./ — sketch ../../..//resources/BlinkSlow-atmega-328–16Mhz.cpp.hex — serial-port /dev/tty.usbserial-A6005uRz — baud-rate 9600 — remote-xbee-address “0013A200408B98FF” — radio-type series2

And omg

Successfully flashed remote Arduino in 51s, with 0 retries

The garage door circuit is fairly simple: there’s a transistor for controlling the relay and a 1N4004 diode for protecting against back EMF spikes. Then there’s the Remote Arduino components: Arduino, EEPROM and some headers/jumpers mixed in. I’m using an XBee for wireless and an Arduino XBee shield for power, socket and serial.

Next up I needed to add the garage door components to the board. I wanted the ability to reset the XBee from the Arduino since the radio has a tendency to disassociate from the network over time and requires a power cycle (perhaps solved with the firmware upgrade). The proper way to do this would be to use a transistor or mosfet to drive the reset low momentarily, but I was out of components so to wing it I’m just connecting the Arduino pin directly to the XBee and hope I never set that pin high.

New and improved Garage Door Controller with full remote firmware capabilities!

The door control 1 and 2 pins connect to the garage door head unit (no polarity). The relay closes that circuit momentarily to activate the door. The open-door magnetic sensor connects to D3 and ground. Similarly the close-door magnetic sensor connects D4 and ground. When those sensors close they drive the Arduino input low; when open the input reads 5V.

In hindsight I should have used spring terminals to connect the door wiring to the board as the headers are a bit janky in accepting wire. Next time I’ll use one of these:

I made a couple improvements to the XBeeUploader sketch (the sketch that facilitates the firmware flashing). Originally it had the other sender XBee’s address hard-coded, but since the message is request/response, it can just get the other radio’s address when it receives a packet and reply with that address. It’s best to enable encryption since it’s possible someone could load arbitrary firmware on your device. The XBeeUploader sketch shouldn’t need to change from here on, except for bug fixes and that’s good because it’s the one thing that can’t be updated over-the-air.

I’m using a Modern Device RBBB as the firmware programmer. I could have used an Arduino Pro, or really any Arduino but this board is compact, has power built-in and I had it already. I had forgotten it doesn’t auto-reset on sketch uploads. After you click the upload button in the IDE you have to hit the reset button on the board when you see “Overriding baud rate…” in the Arduino IDE. This RBBB has an updated Optiboot bootloader installed and it looks like an Arduino UNO to the Arduino IDE. I have a USB BUB (v1) TTL that makes programming it a snap.

I soldered up all the components in and uploaded my test blink test, but it was dropping packets all over the place. After lots of troubleshooting, I realized I accidentally flipped the USB/XBEE switch on the shield when I was adding headers. A simple fix but I burned a few hours thinking I had incorrectly wired the board. I added a LED to fire when the relay activated, but this turned out to be problematic since the LED sucked too much current and there wasn’t enough oomph to drive the relay, so that had to go. Lastly I uploaded a simple sketch to verify the relay and the door sensors would fire and things looked good.

Next up I replaced my dev/test radios with the radios I had been using in the garage door application. With these radios I could upload sketches but the app would not respond to any commands (e.g. door status). The only difference between these radios and the dev radios was firmware version so I attempt to bring them up to date.

Digi provides a mac app (XCTU) for firmware updates but I found it only sort of works. I was unable to update the firmware — it would consistently crash at varying percentage completes during flashing with message: “Could not program the radio module -> getData: failed”. So not super helpful.

I tried this about a dozen times. I switched USB cables, xbee sockets, USB ports, rebooted, enabled hardware flow control, etc, but no matter it would consistently fail. After more tinkering I found I was able to upgrade to the subsequent version, but not the latest (about 10 versions ahead), but only if I selected the XBee Recovery tool (Update firmware failed consistently). While dealing with this nonsense I realized the Sparkfun XBee Explorer reversed several pins: the reset pin on XBee is pin 5 but for some awful, ungodly reason the Sparkfun board swaps them and uses pin 6. It also swaps pins 1 and 2. Now I could actually reset the radio when XCTU told me to, but that was just one factor in this mess. After lots of intermediate upgrades and failures I finally got them all upgraded to the latest firmware. Oh, and now it worked! I’m not sure why since it was working before but just going to accept this and move on.

About a year ago I built a socket/serial server to relay packets over TCP and then to USB-serial XBee. This is useful to share access to an XBee across processes/machines, because otherwise only one process can own the serial port and the app must live on the same machine as the device. The only change to the client code to send via sockets vs directly to an USB connected XBee is one line to indicate the host and port of the server:

xbee.initProviderConnection((XBeeConnection)new SocketXBeeConnection("pi", 9000));

And after that the api operates the same whether the xbee connect via serial port or across the network. It’s on github and published to maven central.

Finally in the pizza dough box and ready for install

Current versions

  • Arduino IDE 1.0.5, although later version should also work
  • xbee-arduino commit eee466d56451ad651ba52ede7902b378b7a0c847
  • xbee-api commit 8c4bdd6418001949ed7c2fc73ba9a3e16f556472
  • arduino-remote-uploader commit 4740f98de801b43b8793817476839f0b3abd1284
Show your support

Clapping shows how much you appreciated Andrew Rapp’s story.