Introducing Dyno 🦕

8 min readFeb 23, 2019

Update: the second article in this series is now available!

Dyno is a new Swift library under construction to:

Provide a functional, Reactive, safe and easy-to-use interface to the Amazon AWS DynamoDB database.

Don’t worry if that doesn’t make much sense — or if it makes sense but doesn’t sound useful for you — because as well as practical steps I’ll also be discussing ideas and techniques which will hopefully be adaptable elsewhere.

Along the way, it will:

integrate the official AWS Python boto3 libraries with Swift
add robust, async handling
output data via an reactive Observable stream
provide RxSwift extensions to both macOS and iOS which integrate results into your apps in a simple, declarative way.

I will also look at adding some additional features:

offline data storage and syncing
extension to some of DynamoDB’s advanced capabilities.

The side aim is to show how the powerful functional tools that I’ve been looking at in prior articles can give great practical benefit.

I’m also planning to develop this library as part of writing these articles. The Dyno source code is available on github (I’ll be continuing to adapt it as I go, branching the code to match these articles: this article is under the introduction branch) So if you are interested in even just some of these topics, follow on — and comment, PR and star as appropriate 😀!

Linux / non-Apple / server-side folks, I haven’t forgotten you: a lot of what’s in here will be hopefully of relevance, even if some of the details will change.

This Article

In this article we’ll do some basic introductions to the concepts; then get set up with AWS DynamoDB. This will be a lot of initial configuration, but then we’re done: and we will see how to use PythonKit to get access to our new database from Swift.

I can’t wait to get started! 🦕

Meet the Characters

DynamoDB

DynamoDB is a highly available and resilient cloud-hosted NoSQL database from Amazon Web Services. It scales enormously, but is also free for small use cases, so it’s perfect to get started with.

Boto3

Boto3 is the official Amazon SDK for communicating with DynamoDB. It uses Python: other languages (like Java and Go) are available, but no Swift. What we’ll do in Dyno is make use of the semi-official Python interoperability developed by the Google Tensorflow team (which itself relies on Swift 5 features, so these articles are Swift 5 only. We’ll discuss how to deal with that shortly).

I should also point out that there is AWS support for Swift and DynamoDB in the shape of the AppSync platform, which is a sophisticated interaction library and will be a great choice for some apps. However: it only exists for iOS; it’s very heavyweight and complex; and it doesn’t use reactive programming techniques — so I am looking for something with more compatibility, lighter weight, and more ‘functional’.

Observables and Reactive programming

Reactive programming describes streams of data (‘Observables’ : which might be button clicks in a UI, data packets on a network — or results retrieved from a cloud data store) in a functional way, allowing us to manipulate them using operators . This allows us to use a declarative style to develop whole applications. There are a couple of reactive frameworks available for Swift; I’m using RxSwift because I’m more familiar with it.

By returning data from DynamoDB as a stream of Observables, we can then attach the results to a data table. But more than that: we can show spinners when the data is loading; failure messages in case of error; and many other neat features simply by observing the stream. The code does take a bit of getting used to, but it’s well worth it:

…these 4 lines of code: check for the application either refreshing its data store or storing new data; show a spinner while data is being transferred; display any errors; and display the results of a refresh in a table view. That’s a lot of power for not much code!

Reactive Extensions

RxSwift adds a number of extensions to common UI objects, to allow them to interact with Observable streams. Dyno will add some more, including to make them work with common macOS UI elements and providing some handy operators (eg –-> to bind an Observable stream and ..> to dispose)

Getting Started

Setting up AWS

To get started, you’ll need to set up an account with AWS if you have not done so already. This is a bit of a convoluted process, taking you through a number of security options. As part of the setup, you’ll be given an access key ID and a secret access key : keep note of these (you can’t re-retrieve your secret access key) as we’ll need them shortly. You might also need to set up a region, depending on where in the world you’re located. Finally you’ll need to provide a credit card so they can charge you: but don’t worry, we shouldn’t be hitting any of the capacity charges and should remain in the free tier!

Setting up DynamoDB

Once that’s done, and you have access to the AWS Management Console, search for DynamoDB and you’ll get to a dashboard:

Click Create Table and type in a table name. Let’s call this Dinosaurs and put in a primary key called id. Everything else can stay the same. Click Create to actually create the table.

Note: it’s best practice to not use names as primary keys in databases — stick to meaningless identifiers instead

You’ll switch back to the dashboard and the table will be created — this can take a little while, but once that’s done you’ll jump to the Tables view with the Dinosaur table selected. I would recommend making a change on this view, to avoid billing costs:

Go to the Capacity tab and change the Read/Write Capacity Units to 1

That’s all we’ll need AWS for for now, let’s get cracking with some code!

Python and Boto3

To actually talk to the new database we just set up, we’re going to use the boto3 library from Amazon. There are some instructions here but basically you need to go to a Terminal and type:

> sudo easy_install pip
> pip install boto3 --user

(You might not need the first command if pip is already on your system; and in the second you might also need a flag --ignore-installed six to ignore previously-installed packages)

Next, set up a credentials file. Create a directory ~/.aws and create a file called credentials in there which looks like this:

[default]
aws_access_key_id=[your access key id from earlier]
aws_secret_access_key=[your secret access key from earlier]

I also had to set up a config file like this for the Region my table was installed in:

[default]
region=us-east-2

Once all that setup is done, let’s try this out.

> python

>>> import boto3
>>> dynamodb = boto3.resource("dynamodb")
>>> table = dynamodb.Table("Dinosaurs")
>>> table.scan()
{u'Count': 0, u'Items': [], u'ScannedCount': 0, 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId'...}}

… you should receive a message with an HTTPStatusCode of 200 indicating that the request was OK, even though there were no rows returned. If this doesn’t happen – check carefully, particularly your credentials settings.

Finally, some Swift (part 1)

The final thing we’ll tackle in this article is getting this working in Swift. For that, we are going to leverage a library called PythonKit, which is a wrapper built by Pedro Vieto leveraging the Python interoperability library built by the Tensorflow team at Google. That team have introduced some features to the core Swift language to make working with Python much simpler — those features have made their way into Swift 5… and that means we’re going to need do some more setup and get hold of Swift 5 !

The easiest way to do that (assuming you’re reading this before the official release of Swift 5 in Spring 2019) is to download the latest Xcode Beta from the Apple developer site. Once that’s downloaded and installed (it’s not quick, I’m afraid) you’ll need to switch the command-line Swift tools to use your new download:

> sudo xcode-select -s /Applications/Xcode-beta.app/Contents/Developer

Now, we’re going to use the Swift Package Manager to do the hard work of creating an XCode project with the two major dependencies we’ll need: PythonKit and RxSwift.

So, create a new directory called Dyno ; go into that directory and type the below to set up a new SPM directory structure:

> swift package init --type executable

We’re using an executable structure for now, so we can get started (hurray!)

Find the Package.swift file that was created, and edit it so that it looks somewhat like this:

// swift-tools-version:5.0
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "Dyno",
    dependencies: [
	.package(url: "https://github.com/pvieito/PythonKit.git", .branch("master")),
    .package(url: "https://github.com/ReactiveX/RxSwift.git", "4.0.0" ..< "5.0.0")
    ],
    targets: [
        .target(
            name: "Dyno",
            dependencies: ["PythonKit","RxSwift"]),
        .testTarget(
            name: "DynoTests",
            dependencies: ["Dyno"]),
    ]
)

Then, fetch the dependencies and create/open an Xcode project using the below commands:

> swift package fetch
> swift package generate-xcodeproj
> open Dyno.xcodeproj

Finally, some Swift (part 2, with actual Swift this time)

In main.swift add the following:

import PythonKit

let boto3 = Python.import("boto3")
let dynamodb = boto3.resource("dynamodb")
let table = dynamodb.Table("Dinosaurs")
print( table.scan() )

And amazingly that will work in the same way as the Python code previously, pulling the (admittedly non-existent) data from the table and displaying the connection information.

You might also want to look at how similar the code looks to the original Python version, and indeed experiment with the boto3 library to do some more operations on that table – wouldn’t adding some dinosaurs be nice?

table.put_item(
    Item:[
        "id" : "1",
        "name" : "Emojisaurus",
        "colour" : "blue"
   ]) 

print( table.scan() )

🦕

For Next Time

Phew! That’s a lot of setup and configuration out of the way, though we did get something interesting happening in the end. Next time we’re going to actually build more of a library with APIs to hide that Pythonnessss 🐍 and provide a Swift interface! 🏎

UPDATE: the next article is here: https://medium.com/@JLHLonline/dyno-aws-swifter-1ecaa1249924