AWS Greengrass — the Missing Manual
The Greengrass API is a pile of spare parts without wiring instructions — so here’s a map, an explanation, and some code
AWS Greengrass is a great technology. If you’re reading this, you likely already think so too. If you don’t — check out re:Invent keynote and AWS Greengrass intro, and you’ll come back convinced and inspired to hack.
To master Greengrass beyond the tutorials, to define and deploy setups in automated, reliable, repeatable way, one must hit the AWS API or CLI — we long live in the bright DevOps world.
There is a big problem here: while AWS documentation on Greengrass lists individual calls, it gives no clue on where to start. There is no map to show how to wire the calls together. All parts are in place, but the assembly manual is missing. Or, was missing — until this post.
Getting Started
I’ll begin with the high level: concept definitions, a map of the model, some theory on how things work … but my millennial readers are eager to learn by doing, so let’s get started with some working code. No worries, you’ll still find the map and concepts below.
This is the code to create a Greengrass group with a single core.
As a bonus, get this code in a Jupyter Notebook for your convenience and immediate gratification!
When you click a single “Create Group” button in the AWS console to create Greengrass Core, the code provided below is what happens bend the scenes. It’s quite a lot … 7 calls across 3 AWS services — 5 entities created explicitly, 2 implicitly.
Creating the group is just the beginning — you’ll still need to:
- create Lambda functions with AWS Lambda
- register them with IoT via Lambda definitions
- define resources for your Lambdas to access
- configure loggers and register devices
- setup MQTT subscriptions for every point-to-point communication
- deploy the whole shebang to a Greengrass core
And by the way, all of this must be properly installed and configured, carry the right certificates, and run on your edge device. If it sounds a bit complex, it is because it is a bit complex. Time to elevate to the concepts and get oriented.
The Lifecycle Workflow
First, let’s review the steps of a Greengrass operation lifecycle workflow.
- Create Greengrass group definition in AWS IoT — a number of calls to AWS API to create and connect all necessary entities.
- Setup Greengrass core “on the edge” — configure a box Raspberry Pi or other compatible device, or VM / Docker for dev & testing. Download and install a proper version of greengrass, put certificates in place, adjust `config.json`, and launch the Greengrass daemon.
- Deploy — tell AWS to push Greengrass group definition to the Greengrass Core runtime(s).
- Clean up — the steps 1–3 produce a slew of artifacts that would pollute your AWS account if not properly recycled.
The Model
Next let’s understand the model. The group definition — step 1 above — is by far the most complex. To help you navigate it, I created a “Greengrass Entity Map”. While the map is not a topologically precise representation of AWS API, it would help you grasp the model and not be lost.
Greengrass is intertwined with other AWS services — most notably IoT, Lambda, and IAM & security which are color-coded on the map. Take a look at the map. Spot the objects created by the code above. Explore the rest. Enjoy!
The LoggerDefinition and ResourceDefinition blocks are not shown on the map to reduce clutter — they are relatively straightforward so you can extrapolate now that you see the pattern.
Greengrass is conceptually a part of AWS IoT, and visually it’s a section of IoT in AWS console. But at the API level it is a separate service with distinct model — quite different from IoT in the concept, conventions, and even naming.
The Entities
In Greengrass, each entity is versioned and immutable. Any modification is done via updating an entity with a newly created version. Versions carry all the data and references to the other AWS services.
Group Version
The focal point of the model is a Group Version. A group manifests in the AWS console only once a Group Version is created. But to create a Group Version, you must first create a pinch of other entities and supply them to create_group_version call.
Greengrass Core
The Greengrass Core has dual nature: on the IoT side, it as a “thing”, or “device” for connection and communications, and is represented by IoT “thing” in the IoT registry, along with attached certificates, role and policies. On the Greengrass side, it is exposed as a CoreDefinition/Version.
Lambda Functions
Lambda functions are first created in AWS Lambda, and linked to Greengrass via FunctionDefinition/Version. It also adds some extra Greengrass-specific Lambda parameters — like pinned flag to make Lambda long running, device access policies, or extra environment variables.
To be linked, Lambdas must be published, and shall be referred by a qualified ARN — an ARN with a version or an alias (recommended). I wish AWS API doc told me about this— it doesn’t, and API throws a puzzling “functions definition is invalid or corrupted” message if you supply an unqualified ARN as FunctionArn.
Subscriptions
Subscriptions are straightforward but too verbose to define: I find it irritating as it takes a bunch of them to define all communication between devices, lambdas, and cloud. How to deal with it? See “file-based approach to Greengrass setup” down below.
Device Definition
Device Definition represent devices connected to the core as links to IoT Things — these Things must be set up in AWS IoT. Consider using ELF to quickly simulate a device for you dev environment. Logger Definition tells Greengrass core what to log locally and what to send to CloudWatch. Resource Definitions are there to define Lambda access to resources of Greengrass core device group-wide, to attach to Lambda Function Definitions.
Oh, and did you notice Machine Learning resources in Greengrass AWS Console now that AWS ML Inference is generally available?
Enough said about the map: explore it, track and find the correspondent AWS API calls to create and connect the entities. I considered putting the exact methods on the map but decided to favor aesthetics, leaving you some space and joy of discovery.
Deploying Greengrass
Finally, a word on file-based approach to Greengrass deployment. As I played with Greengrass, I wanted — but missed—the three things I am so used to enjoy with Ansible, Terraform, or Serverless framework:
- File-based Greengrass group definition
- One-command to create, deploy, and clean things up
- Simple, representable, repeatable dev boilerplate with Greengrass Core on a VM (Docker? even better!)
I missed it, so I wrote one. Please welcome greengo — a simple file based Greengrass configuration tool.
Take it as your Greengrass dev boilerplate — adjust greengo.yaml to your liking and hit greengo create. Or use is as a cheat-sheet to find out how to call the right boto3 methods in the right order with the right parameters, as you track the chains on the map above.
That’s it. I hope that the map above, the code examples, the explanations and greengo tool/cheat-sheet will fill out the blanks in AWS doc and help you operate Greengrass in automated, repeatable way, DevOps style.
Happy green-grassing!
If you find this manual helpful or interesting, I take your claps 👏 for tips . Please share your thoughts in the comments here, or follow me on Twitter @dzimine for discussions and stories on IoT, DevOps, and Serverless.
