Embedded MongoDB 4.0 on Raspberry Pi

My Raspberry Pi Development and Test Environment

Introduction

With the explosion of IoT devices there is a growing demand for light and efficient local embedded databases. Of course you can use SQLite — often with some form of ORM — but if you’re already using JSON as a serialization and interchange format in your software, then being able easily store and query your native JSON objects directly with the database is an incredibly attractive proposition. And given that MongoDB is the most popular JSON Document database out there, being able to employ it for this use case is ideal.

As part of our new MongoDB Mobile solution, we built a low level component that provides an embedded version of MongoDB 4.0 — essentially MongoDB as a library. This lower level building block is an internal part of the purpose built full stack solution — MongoDB Mobile — but it’s also possible to use this low level component directly if and when it makes sense to do so.

For more details on the components we’ve built as part of the MongoDB Mobile project, see the slides from my MongoDB Mobile talk at MongoDB World 2018.

In this post I’ll walk you through how I got the MongoDB 4.0 Embedded SDK built, installed, and working on my Rasberry Pi. Then I’ll provide you with a sample application that demonstrates how it can be used for your own IoT application¹.

[1] Update (Dec 2, 2018): For more recent follow-up work, please see my mongo-embedded-sample github repo.

Platform Details

Hardware: Raspberry Pi 3B+, with a Samsung EVO 32GB micro SD card

OS: Raspbian Stretch Lite — Debian 9 variant specifically for Pi’s (32bit / armv7l OS)

You’ll need to get Raspbian Stretch booting on your device. The output of cat /etc/debian_version should return 9.x.

Note: The Raspbian kernel is for the armv7l CPU arch, regardless of whether your Pi is ARMv7 (Pi 2) or ARMv8 (Pi 3); ARMv6 (Pi 1 and Pi Zero) is out of scope.

MongoDB 4.0 Embedded SDK Installation

Setup the Build Toolchain and Environment

Next you will follow a variant of the instructions here to get the MongoDB 4.0 build environment and toolchain in place. The installed gcc and python packages are good. But you will need to install the following additional packages:

shell# sudo apt-get install scons libssl-dev python-pip libffi-dev libcurl4-openssl-dev cmake

We’ll then need to install the python requirements:

shell# sudo pip install -r ./buildscripts/requirements.txt
shell# cat > ./python-requirements.txt <<EOF
cheetah3==3.1.0
pyyaml==3.13
typing==3.6.4
EOF
shell# sudo pip install -r ./python-requirements.txt

Build and Install the C Driver

You can download the latest C driver source here. We can then build and install it (in /usr/local/) this way:

shell# tar xzvf mongo-c-driver-1.12.0.tar.gz
shell# cd mongo-c-driver-1.12.0 && cmake . && make install

Build and install the Embedded SDK

First you’ll need to download the latest MongoDB 4.0 source tar here (“Download Source: tgz” link).

shell# tar xzvf mongodb-src-r4.0.2.tar.gz
shell# cd mongodb-src-r4.0.2
shell# ./buildscripts/scons.py --link-model=dynamic --install-mode=hygienic --disable-warnings-as-errors --enable-free-mon=off --js-engine=none --dbg=off --wiredtiger=off --use-system-mongo-c=on --allocator=system CPPPATH=”/usr/local/include/libbson-1.0 /usr/local/include/libmongoc-1.0" LIBPATH=”/usr/local/lib” CCFLAGS=”-mabi=aapcs-linux -march=armv7-a” install-embedded-{dev,test} -j1

Now you can go and relax as it will take a while for the build to finish. 🙂

Note: with -j > 1 you’re likely to encounter virtual memory exhaustion unless you setup a larger swap partition.

Now we can install Embedded MongoDB (libraries and headers) in /usr/local:

shell# cd build/install
shell# sudo cp -R * /usr/local

Next we can perform a quick sanity check to ensure the build is installed and working as expected using the embedded test binary:

shell# /usr/local/bin/mongoed --version
2018-09-05T13:30:46.905-0400 I -        [main] MongoDB embedded standalone application, for testing purposes only
2018-09-05T13:30:46.912-0400 I CONTROL  [main]
2018-09-05T13:30:46.912-0400 W CONTROL  [main] 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.
2018-09-05T13:30:46.912-0400 I CONTROL  [main]
2018-09-05T13:30:47.004-0400 I CONTROL  [initandlisten] MongoDB starting : pid=839 port=27017 dbpath=/data/db 32-bit
2018-09-05T13:30:47.005-0400 E -        [initandlisten] No such file or directory

Now lastly we can save the self-contained Embedded SDK (C Driver + Embedded MongoDB) as a tar package:

shell# cp -R /usr/local/include/lib{bson-1.0,mongoc-1.0}/ ./include/ && cp -R /usr/local/lib/lib{bson-1.0*,mongoc-1.0*} ./lib/
shell# tar czvf mongodb-embedded-4.0.2.tar.gz * && mv mongodb-embedded* ~/

The resulting binary package — ~/mongodb-embedded-4.0.*.tar.gz — is then what you can copy to all of your other Pi devices. After copying the tarball to another device, you can simply install it in /usr/local:

shell# tar xzvf mongodb-embedded-4.0.*.tar.gz -C /usr/local

An Example Application

Now that we have the Embedded SDK installed, let’s create a simple “guestbook” app to demonstrate how to use it.

Then we can build it this way:

shell# gcc -I/usr/local/include/libbson-1.0/ -I/usr/local/include/libmongoc-1.0/ -I/usr/local/include/mongo/embedded-v1/ -L/usr/local/lib -lmongoc-1.0 -lbson-1.0 -lmongo_embedded_capi -lmongo_embedded_mongoc_client -w -o iot_guestbook iot_guestbook.c

Finally let’s test our guestbook app:

shell# ./iot_guestbook --help
Usage: iot_guestbook [name] [message]
shell# ./iot_guestbook "Matt Lord" "Hello from Pi-ville!"
{ "message" : "Hello from Pi-ville!", "from" : "Matt Lord", "date" : { "$date" : "1536424056000" } }
shell# ./iot_guestbook "Lily Lord" "Hello from the Poconos"
{ "message" : "Hello from Pi-ville!", "from" : "Matt Lord", "date" : { "$date" : "1536424056000" } }
{ "message" : "Hello from the Poconos", "from" : "Lily Lord", "date" : { "$date" : "1536424062000" } }

Conclusion

Now you have a MongoDB 4.0 Embedded SDK (~/mongodb-embedded-4.0.*.tar.gz ) that can be used on Raspbian Stretch based Raspberry Pi 2 (ARMv7-a) and Raspberry Pi 3 (ARMv8-a) devices.

As you can see this is still very much a DIY setup and it is not commercially supported today. But hopefully it provides you with a reference guide that allows you to begin testing the Embedded MongoDB 4.0 database on your IoT devices until we’re able to deliver a more purpose built full stack MongoDB IoT solution.

If you have any questions or run into any problems, please let me know here in the comments or via email. THANK YOU for using MongoDB!