6 tips for developing with AWS Greengrass and CloudFormation — Part II

These might just help you get a jump over some of the hurdles you may encounter using these services

Second things second

Continuation from Part I

If you’re reading this then you know that I am gone by now. Well, you would if this was a cheesy movie. It’s not, so you’re likely to have landed first on Part I of these development tips. If not and you’d like to know where the motivation for these articles came from, make sure to check that one out.

Otherwise all you need to know is that in this text I delve into three more development tips for developing with the AWS offerings Greengrass and CloudFormation. If you’re to understand what you’ll read, I expect you to have some working knowledge of both of those services. Read through or take this article as a reference if you ever run into any of these issues:

1. Handling IoT rules and rule permissions from CloudFormation;
2. Streamlining the test of code changes in a physical Greengrass IoT edge device;
3. Accessing local Greengrass IoT edge device files for simple operations.

Let’s dive right into it.

The tips

Note: for brevity I’ll refer to Greengrass as GG and to CloudFormation as CFN throughout.

Handling IoT rules and rule permissions from CloudFormation

Let’s say you have a GG group physically deployed to your IoT edge devices. Now you might want to send some information from them back to the cloud (maybe you even already set up a GG Subscription for it). Plus, if you’re working with Lambda microservices, it is likely you want to trigger a cloud function in reaction to something published from your devices. Perhaps you have even got to the point where you figured out the way to go is with AWS IoT Rules.

How do you incorporate that with your CFN-based development? By defining those IoT rules in a CFN template, naturally. Let’s have a look at the code for one such example rule:

Where there is dev, we could instead use a Sub for an environment parameter.

This rule named devProcessSignalRule triggers on any message that is published into the /dev/+/process/signal MQTT topic (+ stands for a wildcard, in this case I wanted all my IoT cores to send a message to their respective topic e.g. /dev/core1/process/signal, but the same Lambda function to handle all the messages in the cloud). It then invokes a Lambda function with an ARN that is imported from another stack (dev-process-signal-function-arn is the export name), passing the message payload as the event function argument. After CFN deployment I was able to go to the AWS IoT service in the AWS Console and indeed see the rule there:

Nonetheless, I faced a problem: the rule didn’t seem to call the Lambda function into action. As many messages as I sent to the correct topic and no matter what the message, I did not see any logging on CloudWatch nor the effects that I was expecting on my RDS database. Intriguing.

The rub had to do, as many times happens while connecting services, with permissions. Specifically, Lambda function permissions. Although my GG group role had permission to publish to the AWS IoT service, the AWS IoT service itself was not whitelisted in my Lambda function as an authorised invoker.

As per the fix, you can find it below. In the end it was as simple as adding an AWS::Lambda::Permission resource on the same CFN template as the ProcessSignalRule IoT rule above, allowing the rule to trigger the intended Lambda. After deploying the template everything started working nicely. Hopefully this spares you a headache.

First takeaway: to handle IoT rules and rule permissions from CFN, make sure you add a Permission resource on your template to allow your IoT rule to trigger your target functions.

Streamlining the test of code changes in a physical Greengrass IoT edge device

After my CFN template and cloud deployment for the GG group were all set up (see the previous tips for quite a lot of information about that), my usual development workflow became as follows:

1. Modify Lambda functions’ code however suited my needs;
2. Deploy the CFN template to the cloud with the updated function definitions;
3. Deploy the group to my physical device;
4. Test the changes I just made.

However, having to go through this painstaking process every time I wanted to make even the slightest change to any function soon became tiresome, to the point I felt like there had to be a better way. After dedicating some time to investigate the GG folders living on my physical device, I found out something interesting.

Inside the IoT edge device, the Lambda functions’ code sits in the folder /greengrass/ggc/deployment/lambda/. For example, the folder /greengrass/ggc/deployment/lambda/[example function Lambda ARN]/ contains the entire code for some example Lambda function.

Let’s assume you are connected to the device by SSH and imagine you want to test a small change to the file index.js of this Lambda function. But you don’t want to go through the effort of deploying a new GG group version with the updated Lambda to the cloud and then the device. One option is to simply navigate to the mentioned deployment folder on the device and run:

vim index.js

If you’re familiar with vim, you can use it to edit the file as you please. If you’re not, you can use another editing tool or instead edit it in your local environment with your favourite editor, but then you need to somehow place it inside the device. In the same scenario, you can use SCP on Mac or Linux (or do something similar with Putty and PSCP on Windows) to pass files to the device by going (in your machine) to the folder where they are located (in this case the folder where index.js is placed) and running:

scp index.js [user]@[IP address]:/greengrass/ggc/deployment/lambda/[example function Lambda ARN]/

(Obviously user and IP address refer to the same user and IP address you use to SSH into your device.)

This will override the index.js in the deployment folder of the example function on the device in favour of the new one.

In both cases (in-place editing and file transfer), GG will not pick up the changes in the code until you restart its daemon (meaning the device will not be executing the new code you want to test). Luckily GG provides an easy way to do that. Simply run these two commands on the device:

/greengrass/ggc/core/greengrassd stop
/greengrass/ggc/core/greengrassd start

Momentarily all the Lambdas should be executing with their new code. If you’re crafty with bash you can even write a script on the device side to automate the whole thing: get all the files you want from your local machine into the device’s correct location, and then restart the daemon, every time you make some changes. Up to you.

When you’re happy with your outcomes you can get your changes properly deployed through the AWS services, updating the Lambda ARNs in your CFN stack for GG, to make double sure everything still works properly.

Second takeaway: to streamline the test of code changes in a physical GG IoT edge device, make the code changes directly to the deployed files and restart the GG daemon on the device.

Accessing local Greengrass IoT edge device files for simple operations

The beauty of GG is that it runs on containers. Coming from an Electrical / Embedded Engineering background, the fact that the same kind of Lambda function that runs in the cloud can also run in an (albeit very powerful) Linux device just fascinates me.

For this tip we face the flip side to this beauty, which sometimes is practicality. One of my requirements was that I had to write two Lambda functions that needed access to the local device file system, one to reboot the device and another to upload the GG system file /greengrass/ggc/var/log/crash.log (which for some reason is not synced to CloudWatch logs — follow this link for more info) daily to an S3 bucket.

They seemed like simple enough tasks until I got to actually develop and test them. The difficulty comes from the fact that the GG container bus cannot access the system controller binaries, so I was never able to run the reboot command or get access to the GG deployment files from within a function.

You can resolve this by carefully inspecting the configuration options available to GG Lambda functions. The Containerization and Run As are of special importance in this case. The first allows you to run the functions as a regular Linux process, while the second lets you choose the UID and GID access identity.

So if you have some resources in your local system that are not accessible from the container (such as the crash.log file) or that require access from an identity other than the default GG user and group (such as the reboot executable), you can specify them with these options on your GG CFN template function definitions.

Just be extra careful about following the AWS recommendation not to run functions as the device root user, as that poses a security threat. For something as simple as a reboot though, you can probably get away with it with little to no risk.

Third takeaway: to access local GG IoT edge device files for simple operations, change the Containerization and Run As configurations accordingly to suit your needs.

Wrap up

I honestly hope that this collection of assorted development learnings I took away from my recent work with AWS Greengrass and CloudFormation are useful for someone out there under a similar situation.

And of course, feel free to drop me a comment with any doubts, questions, suggestions or feedback you might have. I’d love to hear it if you have any improvements regarding anything I went over in this couple of articles.