Appengine + Gradle Power

Dynamic AppEngine Configurations using Gradle Part 2

In the previous story Dynamic AppEngine Configurations using Gradle Part 1, we discussed how we can easily deploy to various environments like (prod, stag, test) using simple gradle argument. If you haven’t read before, would recommend to read it before proceeding.

Now let’s take a look at how we can have multiple configuration files(cron.xml, appengine-web.xml, queue.xml) for each of those environments

Optional configuration files

These configuration files control optional features that apply to all the services in an application:

• dispatch.xml (routing)
• queue.xml (task processing)
• datastore-indexes.xml (querying)
• cron.xml (scheduling)

In order to have different configurations for each environment, we are going to create one folder called gaeconfigs in the root folder, and under which create one folder per environment, like below:

Folder structure for each environment

Now go ahead and add those configuration files for each environment and it looks something like this,

Environment specific configuration files

You can now edit or optimise configurations according to the environment, for example

• for queues, processing rate is 10/s for staging and for production it can be 100/s
• Data processing Cron should run every 5 minutes in production but for staging it can be every hour

What about common configurations for all environments?

Sometimes certain configurations don’t change for each environment like datastore-indexes.xml or you have same settings for all environments, in such case place those configurations directly under gaeconfigs folder, we will handle those as well. Like,

Common configurations

All set, now we need to use these configurations during deployment. We are going to do that using a gradle task,

gradle copy configuration task

Let’s understand what’s happening, First, we are declaring necessary directory path:

def configDir = "${project.projectDir}/gaeconfigs"
def targetDir = "${buildDir}/exploded-" + project.name + "/WEB-INF"
def envName = getDeployConfig().mode //live or stag etc

configDir: points to the gaeconfigs folder
envName: environment value like live or stag or alpha
targetDir: points to appengine explodedWar’s WEB-INF directory, which will be of the form, build/exploded-{your-project-name}/WEB-INF

Then we copy common configurations under the gaeconfigs folder like (datastore-indexes.xml, dos.xml) to the WEB-INF directory and this ignores environment folders, with following code:

from configDir
include '*.xml'
into targetDir

Then depending on the mode flag which is passed during task execution, we are coping configurations from mode specific environment directory into the WEB-INF directory,

// copying env based configs
from fileTree(configDir + "/" + envName)
include '*.xml'
into targetDir

For example, when the mode is -Pmode=live, files from gaeconfigs/live are copied. Finally, we tie this task to explodedWar task which will be called during the local run or during deployment

explodeWar.finalizedBy copyGaeConfig

That’s it. Easy right? Now for deployment just run following command, which automatically copies necessary configurations

gradle appengineDeploy -Pmode=live

Let’s look at another important configuration file which we didn’t cover yet.

appengine-web.xml

This is the primary file to specify information about your app and to identify which files in the app’s WAR are static files (like images) and which are resource files used by the application.

Unlike other configurations, this file is specific to individual module (service). Let’s look at a sample appengine-web.xml file,

appengine-web.xml file

Here we have defined service, static files and scaling configurations. The problem is what if when we need to change scaling config based on the environment we are in.

For that first, we are going to change appengine-web.xml file like below,

appengine-web.xml with placeholder properties

Noticed anything? check this one,

<instance-class>@INSTANCE_CLASS@</instance-class>
@SCALING_CONFIG@

we just replaced the static values with placeholder names @INSTANCE_CLASS@ and @SCALING_CONFIG@ which will be replaced during deployment with the help of the following task,

So in getGaeWebXmlConfig() function, based on the mode flag we are setting values for INSTANCE_CLASS & SCALING_CONFIG, notice for scaling we have added a whole bunch of settings only for live & alpha, but for staging, defaults are used (which also helps in keeping staging cost low).

And in the explodeWar task, we are reading the appengine-web.xml file and replacing placeholder properties with generated one and finally, it is written into the build directory.

Note: Original appengine-web.xml file will be untouched

That’s all. Now each time you run the following command

gradle appengineDeploy -Pmode=live

all configurations based on the environment is automatically configured and copied to build directory. For more check out the demo application in GitHub,

ramesh-dev/gae-dynamic-config-demo

Conclusion

That’s all the tricks for this story. Using these techniques you can also do other sorts of configuration and provisioning like copying application settings, flags, credentials etc. And if you look at it, we just made the deployment a whole lot easier.

Thanks for reading! Happy Deploying…

Here are another useful reference,

• Dynamic AppEngine Configurations using Gradle Part 1
• Auto Detect Environment in Google AppEngine
• Multi Module Projects with Google AppEngine and IntelliJ

This story is published in The Startup, Medium’s largest entrepreneurship publication followed by +393,714 people.

Subscribe to receive our top stories here.