Enterprise NodeJS On OneOps
Introduction
At Walmart, we’ve been building technologies to enhance the development of our eCommerce sites. OneOps, an open-source product developed at and backed by WalmartLabs, is a platform to make application lifecycle management easier and faster. When we were migrating our web applications to NodeJS, we found OneOps to be the natural choice to host our platform. In this post, I’ll go over some OneOps features and how we use them to implement hosting, deployment, and continuous integration (CI) for our NodeJS applications.
What is OneOps?
OneOps is an application lifecycle management (ALM) platform developed and open sourced by WalmartLabs. More specifically:
OneOps is a cloud management and application lifecycle management platform that developers use to both develop and launch new products faster, and to more easily maintain them throughout their entire lifecycle. OneOps enables developers to code their products in a hybrid, multi-cloud environment.
Follow OneOps on twitter for news and updates.
OneOps Features
From a user perspective, OneOps is fairly straightforward to use, either with the nice interactive Web based UI or the REST APIs. Under the hood, the OneOps platform offers many benefits automatically, but my focus will be on user facing features.
The first step is to understand phases in the lifecycle of hosting an app on OneOps. There are are four phases:
- assembly — Container for everything, like a workspace in an IDE.
- design — Blueprint of your platform to host your apps.
- transition — Prepare your blueprint and environment for deployment.
- operation — Monitor and control your deployed apps.
Creating Assembly and Design
You can start using OneOps relatively quickly. The first thing you need to do is create an assembly and design.
Using the UI, you can quickly create an assembly on the organization page. Below I am creating a new assembly for the org GlobalProducts.
OneOps allows you to save your design into a catalog and reuse them when creating a new assembly. Here I start with no design to show what it’s like to use the UI to create a brand new design. Below is the newly created assembly at the design screen which has no platform yet.
When adding a new platform, you need to select a pack. A pack in OneOps, is a Chef recipe that specifies the OneOps components to add to a platform. Out of the box, OneOps comes with many packs that are designed for different needs. The custom pack is a basic one that creates a basic Linux platform. In the image below, it shows all the component types that OneOps supports and the number of each components this platform contains. Note that the basic platform contains 1 compute, 1 fqdn, 1 os, and 1 secgroup. You also get a nice visual diagram with components and their dependencies.
From here, you can add other components to the platform, such as storage volumes and user. A user is very important as it allows you to login to the virtual machine. To add a new user, click on the + button and specify a Linux user, as shown in the image below.
Now it’s time to pick the components to implement a deployment process for our NodeJS apps. The candidates are the build or artifact components. Before I talk about those, let’s look at the requirements.
Platform for NodeJS Apps
Regardless of the hosting environment, for our NodeJS apps, there are a few basic requirements that are important for us:
- Deploying the application with a sealed binary artifact, which contains the entire application, including the node_modules directory. Publish the artifact to a storage repo using Nexus.
- The same NodeJS version as when the sealed binary was built must be used to run the app. The Node binary with the artifact does not need to be included. Instead, we use NVM to manage Node binaries on the VM.
- When deploying the application artifact, we can either retrieve it from the build machines or Nexus.
- To avoid public or private servers affecting our production deployment, we retrieve software packages from our private Nexus server, including NodeJS binaries.
OneOps comes with enough existing component types to implement a platform meeting the requirements above. For example, the artifact component is designed to retrieve a package from Nexus. This made it rather simple — just create an artifact component in the OneOps design.
Next is the scripts to deploy and start the application. OneOps has a build component that allows me to retrieve files from a git repo. I initially used that to retrieve the scripts but later found that even our GitHub servers can’t be a production dependency. So I changed to use artifact component to retrieve the deploy scripts since our Nexus server availability is higher.
Now that I’ve decided on the components for my platform, we need a single design that can be used for different apps. OneOps variables come to the rescue for this.
OneOps Variables
OneOps platform’s components are fairly flexible. You can add, edit, or even delete them easily, but generally making changes to components is more complex. To allow for simple customization, OneOps supports variables at global or design level. You can access any variables in component settings or scripts. Users can quickly make changes to the variables to customize the platform without having to understand the internals of the components.
With design variables, everything you need to create the design is there. Though you can make a single design, once you need to make adjustments to your platform, you’ll want to have a reusable setup script rather than manually adjust through the UI.
The Setup Script
After creating a new design, it can be added to the global catalog, allowing the ability to quickly create a new assembly using the same design. However, the catalog feature in OneOps is not very flexible. Once the design is saved to the catalog, it is not easy to view what’s in the design and you can’t have revision history of changes made to the design.
I decided to write a setup script in Bash that can use OneOps APIs to create new assembly, design, and platform. Here is a reference for the APIs. I wrote the script in Bash using cURL to make the API calls. In retrospect, maybe I should’ve used NodeJS, but at the time OneOps document was in a lot of flux and using cURL allowed me to try things out very quickly so it just stuck. Over time, this script has been updated many times and it allow our developer to quickly create a new OneOps assembly and design for a new app or just to experiment.
The Deploy Script
For the NodeJS platform implementation, I’ve decided to use artifact components to retrieve the app and deploy script. Since the deploy script can’t run until the app is retrieved, it has a dependence on the app artifact component.
To implement the deploy scripts, I had the option to develop them as ruby scripts that OneOps uses, but I decided to use Bash scripts since I am already familiar with that and OneOps supports executing Bash scripts from components.
The deploy script does a lot of small things to make sure the VM has everything needed to run the the app. Here is a list of some essential tasks it performs:
- Installs and setup the necessary software such as the NodeJS binary, NGINX, and PM2
- Make sure existing instance of the app is shutdown first
- Generate the startup and shutdown scripts.
- Cleanup on old versions of the app
- Start the app
The script is zipped up and uploaded to Nexus. The OneOps component start executing it after retrieving it.
The Complete Picture
Now, we have explored the features OneOps offers to allow us to build a platform to host NodeJS apps. Here is a components diagram of what the platform looks like. There are some minor details that are missing, but it’s got all the essentials. Again, this diagram is generated by the OneOps UI and it’s actually a clickable interactive diagram on the UI.
Any developer can follow simple instructions using the setup script to create a new assembly on OneOps in order to deploy a NodeJS app to the cloud environment.
Deployment and Operations
So far we’ve covered the assembly and design phases. Next is the transition phase. As the name implies, it’s basically a transition acting as a holding place for deploying the design to a particular environment. In our cloud, we have production, staging, and dev environments. Transition allows us to mirror and hold a copy of the design for each environment. Each copy can be modified and customized independently. If the main design ever gets updated, it can be pulled into a given transition to update its copy. For developers, most of their activities will be around the transition phase. When they have a new build to test on the dev environment, they go to its transition to update the artifact version and initiate a deployment.
After a transition is deployed to actual VMs, the operations phase is activated. Under this phase, we can check deployment history, VM health notifications, and VM information such as CPU, RAM, and OS metrics.
Monitoring
Our apps are written with instrumentations to allow detail analysis, but OneOps also offers “continuous tracking of metrics against thresholds. Auto-trigger healing, replacement, scaling, or escalation events.” We get periodic emails reporting on the health of our VMs and alerts if anything is not normal. When a VM becomes unhealthy and stop functioning, we receive alert and OneOps can automatically trigger the healing process and redeploy the VM.
Continuous Integration
At this point, our NodeJS platform on OneOps is operational. Our developers can easily create their own instance in the cloud for their testing and development, but for our normal development process, we want our builds to automatically deploy a successful build and kick off the functional tests. With OneOps APIs, this is all possible through automation scripts.
I wrote another Bash script that uses the OneOps deployment API and makes the OneOps deployment process typically done through the UI possible through remote commands. The build jobs for our NodeJS apps use this script to automatically kick off the deployment and functional test.
Adoption
After we made our OneOps NodeJS platform available, we had dozens of developers trying it out within two weeks, and they were able to quickly acquire compute resources for NodeJS environment to learn and test. Our team which worked on migrating an app to NodeJS, was able to setup multiple environments for development and QA testing for regression and continuous integration quickly. Currently, we have more than a dozen apps under development. Each team is managing their own cloud instances with OneOps UI and APIs, and are rapidly iterating through multiple integration cycles daily. The OneOps platform’s flexibility is key to helping our developers realize this fast turn around time.
Other Thoughts
OneOps is a very feature rich and flexible platform. There are many inherent benefits to using OneOps, including other features not utilized in our NodeJS migration. For example, there is a docker pack that is under development. Pack development gives more flexibility with native access to the Chef APIs. We are now working on converting our design into a native OneOps pack.
Conclusion
OneOps is a cloud management and application lifecycle management platform developed and open sourced by WalmartLabs. It’s versatile with many “out of the box” features that make deploying apps in the cloud convenient and flexible. We’ve successfully used it to host the migration of Walmart eCommerce sites to NodeJS at an Enterprise scale.
