DevOps Comes to Juniper Square

Our mission at Juniper Square is to make the world’s private capital markets more efficient, transparent, and accessible. See Making Private Markets Better for more.

-

The Juniper Square team continues to grow quickly, supported by strong fundamentals in our business. True to our value #SolveHardProblems, iteration is a key part of our improvement process — we are constantly evolving our organization as we scale.

It was inevitable that we would need to build out a stronger supporting infrastructure for our engineering organization as we scaled. Today we have over ten engineering teams, the newest of which is our DevOps/SRE team within our Platform track.

DevOps and SRE

The term DevOps, a combination of development and operations, has been around for over ten years now.

It was a response on how to bring the agile principles that developers were implementing into the operations arena. This includes practices such as iterating and releasing rapidly, breaking down silos of information, shifting from waterfall approaches to smaller, functional changes, automating repetitive tasks, and so on.

At its core, DevOps is a set of principles. It seeks to break down silos between developers and operations, reducing the time it takes for software and features to make it into the hands of customers.

SRE, or Site Reliability Engineering, has actually been around for closer to fifteen years, predating DevOps. It was originally coined by Google in an attempt to combine these same agile practices and principles across software development and IT operations.

One of the more noticeable differences between DevOps and SRE is that DevOps focuses mainly on broader principes, whereas SRE defines practical steps to achieve those principles. SRE could be considered a subset, or an implementation of, DevOps principles.

Consider, for example, the DevOps principle of “breaking down silos of information.” The SRE practices implementing that principle would include using the same tools for development and production, capturing operational actions in code, ensuring issues are surfaced to developers quickly, and creating a culture of shared responsibility for product success between developers and SRE.

Scaling and the need for DevOps

There is no hard and fast rule around when to form a DevOps or SRE team — each company grows and evolves in different areas at their own pace.

In the early days of an organization, engineering organizations tend to focus on hiring full stack engineers — developers with a broad array of knowledge varying from frontend to backend, and even some DevOps principles. It’s natural that some related practices and principles will find their way into an organization without a dedicated DevOps/SRE team. The advantage to this approach is that a small group of a few skilled engineers can quickly create applications and services that begin to bring value to an organization.

As a company grows and evolves, things become harder to manage.

As the user base grows, customers want additional features. The organization might have to add more engineers to keep up with demand. The code base becomes larger and more complex. The human brain is a wondrous thing, but there’s a limit to how much complexity and detail it can handle.

Following agile principles, teams typically start breaking things down into more manageable chunks with the objective of preserving velocity. Roles might start shifting from full stack developers to more specialized roles, perhaps first distinguishing between frontend developers and backend developers. Somewhere in this natural development cycle is where DevOps/SRE teams may come into play.

Typical decision points include:

• When your code base can no longer be managed by a single team
• When you want to free up your engineering teams to focus on writing software instead of building & managing infrastructure
• When organizational compliance or regulations require additional layers of command and control (i.e. segregation of duties)
• When there is too much entropy; when there are large numbers of errors that find their way into production

A product team may help close the gap between customers and engineers. A DevOps/SRE team helps close that loop back to the customer.

While DevOps and SRE may not interact directly with the customer, many core tenets of DevOps/SRE focus on the health of code running in production. If your software does not perform in production, customers will not be happy! Ensuring smooth production operations should be core to any business.

Key DevOps and SRE Principles

Context is everything

DevOps has an innate need to understand why. The context drives the decision-making process.

Understanding the underlying business drivers, architectural choices, appetite for risk, and other additional context is critical to making optimal decisions in a system that is continually changing and growing.

The context is not limited to deep dives into the details. A solid grasp on the big picture, from product team to the customer, and even organizational vision, is similarly critical.

In order to drastically improve release times or increase reliability, proposed solutions may look vastly different than the status quo. Masterful context helps conduct the appropriate evaluation, isolate the essential requirements, and come up with the most appropriate approach.

Provide value to your customers

First, who are the customers?

On the immediate end of our services are the developers. Beyond the developers are the end users, or consumers of the developed services. More tangential, the rest of the organization and even the DevOps/SRE team itself.

The second question, what value do we provide?

The first component, inline with our value #BuildCustomerTrust, is for end users. While some may refer to it as uptime, a more accurate description is closer to meeting contracted SLAs and providing a good customer experience, ensuring the service is up, responsive and performs as expected on any customer interaction.

The second key component is quick delivery of features and fixes. Developers should be enabled to deliver their work quickly and with minimal friction. DevOps helps accomplish this by giving engineers the tools and feedback they need to succeed, such as starter templates for services, continuous integration and deployment pipelines, appropriate access to service logs, access to customer facing metrics, and much more.

Sublinear scaling

Another key principle of a DevOps/SRE team is sublinear scaling. “The size and number of SRE teams should grow more slowly than the number of supported services.”

Consistency, simplification, automation, “cattle not pets” all work together to begin to provide this.

In the late 2000s with the housing market crash, companies were downsizing at a terrifying rate as they scrambled to cut costs, but were not willing to reduce services for fear of losing customers. The term “do more with less” became a popular mantra, typically implying to take on more work without reducing the current workload. SRE aims to do just that — reduce the load of the current workload to increase productivity.

Reduce risk

A common equation for assessing risk is

Risk = Blast Radius x Time x Probability

where

• Blast Radius assesses the reach of the event
• Time considers how long it takes to identify and/or recover from the risk
• Probability is the likelihood of the event

Given these three levers, DevOps/SRE seeks to adjust the given risk levels. A few strategies include:

Appropriate tooling
Linting, static code analysis, unit testing, security checks, and more should be worked into pipelines to help ensure the code is structured properly and not exposed to known vulnerabilities.

Disaster recovery planning
While it might be ideal to have a contingency plan for every possible situation (zombies? ice age?), time may be better spent focusing on the highest probability events that could compromise service level agreements. Having documented recovery plans for failed databases and server outages help prevent outages and data loss.

Containerization
Containers serve as a means to standardize the deployment and operation across applications. Containers are built off a snapshot of a code base, running that copy of the application anywhere, consistently. As the code base changes, new containers are built to deploy the new code. This practice, establishing a clear relationship and contract between development teams and DevOps/SRE team, helps reduce risk by ensuring builds and deploys are consistent across various environments such as to test, to stage, and to production.

Scalability
Having a product that brings a lot of customers is the dream for most companies. Having so many customers that your site is constantly down is usually their nightmare. Aim for the former, plan for the latter. Most modern infrastructure systems have multiple scaling mechanisms available, understand your current bottlenecks and how those mechanisms can be leveraged to help. Once you’ve addressed that bottleneck, find the next one and solve that one.

Observability
By providing immediate and actionable means to understand what is happening in production, teams are able to more quickly react to any issue that may occur.

Conclusion

DevOps and SRE play a critical role in a maturing organization.

Our DevOps/SRE team at Juniper Square is focused on delivering the above and has made significant headway.

Our DevOps/SRE team seeks to empower software development teams, providing guidance, infrastructure, and assistance to more quickly iterate, build, and deploy our work to our end customers.