Mendix on AWS for citizen developers

SJ Meyer — Mon, 28 Mar 2022 10:02:41 GMT

A few months ago I spoke at a Mendix Meetup about Mendix and AWS, and how you can leverage the power of AWS to achieve different objectives for your Mendix users. In this article, I will take a look at Citizen Developers in an organization and ways to overcome some common challenges they face.

Three key things that are important to citizen developers: fast iterations, flexibility, and keeping running costs as low as possible. Citizen developers typically build applications for internal or limited public use. Therefore they don’t have to worry about factors like high availability, scalability, or backups and failover. Once the application gains traction or approval from stakeholders, sure those aspects become important. But not at this stage of the product lifecycle. Another important aspect, especially for internal business applications, is security.

I am a big proponent of developers understanding the tools they work with and the environment their applications will be running in. That being said, citizen developers focus on the product and business logic. What we are trying to achieve here is to support them in their process. The infrastructure aspects discussed below should therefore be set up, configured, and maintained by the infrastructure or DevOps competency.

Proposed Solution

The diagram above is a simplification of the proposed architecture. It does not show where application logs will go (CloudWatch Logs) nor does it show how the network is configured. But that is not too important for this discussion.

Application containers will run on AWS Fargate and use an Amazon Aurora Serverless database cluster. Simple Storage Service (S3) is used to store application static files and can also store the application deployment package. Route 53 is a DNS management service and is not strictly required, but it is what we use. When a container task is launched, the container image will be pulled from ECR. Application secrets like database credentials and API keys are pulled from Secrets Manager. While application configurations, such as log level and environment-specific configurations, are pulled from the SSM Parameter Store. Integrations with Secrets Manager and Parameter Store are specified in the “Task Definition”, which is where you can specify some configurations around the container, so there is no manual pulling and setting of secrets and environment variables.

Speed

First, I’d like to look at fast iterations. One of the most annoying things as a developer is when a change takes 2 minutes to make, but it takes 15 minutes to deploy the change. Well, for production workloads that’s not negotiable. Quality is important, and running all the necessary test suites takes time. Citizen developers on the other hand are constantly tweaking and testing, and don’t want to go through the entire build pipeline just to see if their change worked as expected. With Docker containers and a few AWS services, the solution becomes simple. Containerizing applications from the start will have the additional benefit of simplifying the transition from POC to an SDLC-driven production level application.

Typically containers are built to hold the application and all of its dependencies. That single entity will then make it through the various stages, from Dev through to Production. However, I propose a slightly modified approach. We can rather build a container that holds everything the Mendix application needs to run, without the actual application deployment package. Upon container start, the entry point script will pull the deployment package from Amazon S3, and start the app. Yes, this will increase application start-up time. While this is undesirable for Production it doesn’t really matter at this stage. Instead of the process calling for the developer to run a pipeline with container build and deployment steps, all they have to do is upload a new deployment package. You can even take it a step further and set it up so that the container pulls the deployment package straight from the Mendix Build server via the Build API.

I ran a quick test to check the difference in speed between the pipeline approach and my proposed solution. It typically takes 10 to 15 minutes between the pipeline execution start and a running container in the test environment. Within that time the deployment package was pulled from Amazon S3 onto the build server, the container was built and pushed to the container repository (ECR), and a task reached running state on AWS Fargate. Compare that to around 3 minutes from the point of container kick-off to being able to serve traffic. That might not seem like a lot of time, but when you are “in the zone,” the last thing you want is for something to break your speed. I want to stress that this should never be done for applications that run in production environments, but for this use case it is useful.

Flexibility

Now let’s shift our focus to flexibility. Let’s say the application developed is gaining traction within the organization and gaining users fast. Or the test data set is quite large, and growing. This is causing the application to run slow and get bogged down. It would be unfortunate if environmental constraints were to have a negative impact on stakeholder approval. It would be nice if we could throw more resources at the problem with the click of a button. Well, we can! With AWS Fargate and Aurora Serverless, it really is a case of just a click of a button and restarting the application.

With Fargate, you specify the amount of memory the container needs, how many CPUs you would like to allocate, and where in the network the container should run. There are no servers to configure, manage, or scale. Aurora Serverless can scale as CPU load or connections increase, while the storage layer will automatically scale as your dataset grows.

Note; even though Aurora Serverless can auto-scale based on CPU load or connections, we have found that it is simpler to forcibly change the database cluster size and restart the application.

Cost

With most of the services used in the proposed architecture, costing is simple. If it’s running, you pay for it. If it’s not, you don’t. This means that costs can be kept to a minimum by simply turning off an environment when it’s not in use. This step can even be automated. By making use of some simple metrics like the CPU or memory utilization of the container, or logged-in users on the application, you can set up CloudWatch Alarms that can trigger a scaling event. This scaling event can, for example, turn off the application when no one is using it for more than 30 minutes. When there are no active connections to an Aurora Serverless database, it too will turn itself off after a specified amount of time. With a little bit of work, you can configure the application to auto-start when someone calls the URL, which will in turn auto-start the database.

The automated way of doing things takes a bit of work, but I have found that it is usually worth it. And if you use CloudFormation you only have to do it once. AWS also offers two options for saving costs on Fargate workloads in the form of “Spot” pricing and “Savings Plans.” Spot pricing is roughly 75% cheaper than normal Fargate, but the container can go down at any time. There is a 2-minute warning, but it is not really suitable for applications that can’t afford downtime. Savings plans are for when you know that you will use a certain amount of Fargate resources each month across all workloads. You can then purchase a savings plan for, say, 10 CPU and 20 GB memory each month, committing for either 1 or 3 years. You can also choose between no upfront, partial upfront, or all upfront payment of the savings plan.

Security
When it comes to security there are a few deployment decisions that will determine the exact measures that need to be in place to keep the environment secure. Therefore I’m not going to talk about services like Guard Duty or Inspector here. Nor will I talk about secure network configurations, such as using VPC endpoints for services. Instead, I will focus on the security aspects of the specific services used. The proposed architecture makes use of a few “fully managed” AWS services — S3, Aurora Serverless, ECR, Secrets Manager, and Parameter Store. With these services we do not have access to the underlying operating system, we do not have to secure, patch, or update anything, we simply interact with the service and AWS handles all of that for us. The only security-related aspect we have to worry about is securing the API calls to the service, and in the case of Aurora Serverless, secure database connections. This is easily achieved by locking down the database so that it is only accessible from within the VPC, preferably only from the application, and by making use of IAM Roles to secure calls to AWS services.

AWS Fargate is the only service that requires work on the customer’s side. Since customers have full control over what is deployed within a container, it is the customer’s responsibility to ensure it is secure. As part of Elastic Container Repository (ECR), there is one free container image scan per image, per day. This feature performs an image vulnerability scan and reports on any vulnerable container OS packages and their severity. Setting up an automated daily scan and report is relatively simple and can ensure that the container OS is always in a secure state. If a vulnerability is detected, the DevOps competency can simply resolve the issue and push a new image. Application packages are not covered with this scan, but they can be built into the deployment pipeline.

These environments can be deployed to be only accessible within the corporate network if it is an internal project. Greatly reducing the risk and attack surface. The objective is to enable citizen developers to do their thing with little-to-no dependency on others. Making use of services that takes care of security and scalability concerns, therefore, makes sense.

The architecture diagram above shows a single citizen developer environment deployment. A challenge with this setup is serving webpages over HTTPS.

An SSL certificate will have to be installed on the container. A service like Lets Encrypt can help, but will require some configuration. Applications would usually be deployed behind an Application Load Balancer (ALB) and unless you require end-to-end encryption for all requests, you can terminate SSL at the ALB. Amazon Certificate Manager can provide free SSL certificates for use with the Application Load Balancer and will auto-renew certificates every 13 months. The reason why there is no ALB in the proposed solution is because it cannot be turned off when not in use. There is therefore an ongoing cost. If multiple citizen developer environments will be deployed, running an ALB becomes an option. The cost for the ALB can be shared among projects and a single ALB can route traffic to multiple environments.

Conclusion

This post looked at creating an environment for citizen development to function optimally. While it is perfectly fine to use services like AWS Fargate and Aurora Serverless in production workloads, much of this setup is not recommended for that purpose. The nice thing about everything covered here is that you can place it all into a CloudFormation template and spool up replicas of this environment. Which means each citizen developer in the organization can have their own environment to play with. It will run only when in use, and will scale based on the needs of each developer. And with applications already containerized, promoting the project out of POC phase becomes quite trivial.

From the Publisher -

If you enjoyed this article you can find more like it on our Medium page. For great videos and live sessions, you can go to MxLive or our community Youtube page.

For the makers looking to get started, you can sign up for a free account, and get instant access to learning with our Academy.

Interested in getting more involved with our community? Join us in our Slack community channel.

Mendix on AWS for citizen developers was originally published in Mendix Community on Medium, where people are continuing the conversation by highlighting and responding to this story.

Stories by SJ Meyer on Medium