Stop punishing your developers
Go to any medium-to-large size company and you’ll likely see the same story: A complex web of networks, VPNs, environments, access credentials, and domains. Each of these represent an additional hurdle for developers looking to create software for the company: Access must be granted, credentials issued, environments made accessible, and firewall ports opened before the developer will have the resources he or she needs to succeed. The whole process may take weeks before a single line of code can be written. Now ask yourself: Does it have to be this way? Let’s take a look at a common enterprise scenario to understand why this situation might occur — and whether or not the insanity is truly warranted.
The arguments for an internally managed and secured development pipeline
IT and risk managers everywhere will be quick to point out that development environment complexity is sometimes in place for important reasons, not least of which is software security. After all, we’re not just talking about meaningless 1’s and 0’s: The software being created may be intellectual property that makes or breaks the company. For software-intensive companies, a source code leakage to competitors or hackers could destroy the company’s competitive advantage and ultimately doom it to failure. Similarly, inadvertent or malicious injection of code could expose the company to catastrophic liability or a loss of consumer confidence.
Having an internally managed and secured development pipeline, proponents argue, is the only way to counter these risks. Doing so relies on two basic tenets: First, all parts of the development pipeline should be secured on company-managed and -secured environments; second, software should be developed on company machines that securely connect to a monitored internal network. This two-pronged approach, the theory goes, helps the company minimize the chances that the valuable software is lost, stolen, or manipulated to others’ ends.
A case study in developer frustration
The problem, of course, is that the benefits of “secure” software development come with a price. Let’s take a fictitious e-commerce company (let’s call it Acme Online) to demonstrate the ways in which a strict development pipeline can end up punishing developers and the company at large.
At Acme Online, they’re looking to create a cutting-edge redesign of their online shopping experience. They’ve hired a crack team of well-paid contract developers to get started on this ambitious project. Team members are scouring the backlog and preparing to start development on this exciting project. Unfortunately, the roadblocks to the team start popping up fast and furiously:
- Acme corporate security mandates that only company-secured computers can access the network, and so the developers’ shiny latest-generation MacBooks are shelved in favor of company-issued laptops. As contractors, these developers get the leftover previous-generation laptops that are large and slow, limiting their productivity.
- Part of the team is overseas, so some of the Acme-issued laptops will be shipped. Half of the team loses a sprint when the laptops are unexpectedly caught up in customs.
- The VPN is old and slow, and the connection sometimes drops during times of peak volume. Overseas developers, who rely exclusively on the VPN, are at its mercy.
- The developers are not given administrative access on their computers, so software tools that the team is accustomed to using must go through a lengthy security review and approval process. Some tools (e.g., Slack) are ultimately rejected entirely because they are hosted outside the Acme company firewall, and the team is forced to find alternatives.
- Setting up the team’s development environment involves submitting a ticket to a centralized queue. The service level agreement specifies 3–5 days before the request is addressed, and the team will spend several more days in back-and-forth discussions with the systems engineer who didn’t quite understand the original request.
- Integrations with 3rd party tools (e.g., remote testing services) require opening ports in the firewall, another submit-ticket-and-wait proposition.
- The internally managed Acme pipeline doesn’t yet support the latest version of Node.js, which is needed to support some cutting-edge libraries that will allow for a better customer experience. Instead, the user experience is watered down to accommodate the limitation.
Just weeks into the project, the scope has already been cut, the developers are struggling without the tools they need to succeed, the development environment still isn’t fully configured, and the projected cost and duration of the project is exceeding original estimates. It probably isn’t surprising, then, that the morale of the development team is low.
Was it worth it?
On the plus side, none of the (somewhat crippled) source code created by the developers could leak into the wrong hands… until it is deployed to Production, that is. The front-end source code created by this team consists of HTML and JavaScript libraries that are minimally compiled and can be easily downloaded, read and appropriated by the company’s competitors.
Meanwhile, like all good developers, the team sought to speed development and lower costs by leveraging several free, 3rd-party-developed user interface components and software development kits. While the use of these libraries helped keep the project on-track, it exposed the company to the potential for malicious code injection, a risk that the company’s “secure” pipeline could not effectively address on its own.
The end result is that, despite the burdensome process foisted upon the development team, Acme’s security measures failed to provide the intended security benefits.
Protect what matters
The lesson is clear: Security matters, but only when you secure the “right” things. In the case of Acme Online, their real competitive value comes not from their front-end user interface code, but from the back-end services code that codifies their business processes for pricing, inventory, product recommendations, and sales.
In this particular case, the contractors don’t actually need access to this highly proprietary back-end source code to achieve their objectives. What they do need, however, is access to secure, well-documented, and highly accessible development web services. This requirement could have been satisfied with far less onerous security:
- A publicly-facing set of web services that is accessible outside the company firewall, allowing development from anywhere
- SSL certificate authentication, ensuring that those services are accessible to only company-approved developers
- IP whitelisting, to minimize the chances of a denial-of-service attack from unauthorized computers
- API rate limiting, to ensure that no one developer (or malicious actor that has obtained a developer’s credentials) can attempt to reverse-engineer back-end algorithms by issuing thousands of API requests.
Finally, a clear company policy on the use of 3rd-party components, rigorous validation of the completed code, and appropriate safeguards for Acme’s IT and API infrastructure could be effective at combating the risks of malicious front-end code injection without impeding the progress of the development team.
With these security measures in place, the developers could have set up their own development pipeline, used the tools that they are comfortable using, and developed software anywhere and anytime they had an Internet connection.
This approach would also have significant security benefits by implementing the principle of least privilege: Developers would have access to only those resources necessary to complete their work, with no broader access to internal company systems or networks. The use of external-facing APIs in this manner replaces the outmoded and ultimately ineffective concept of a “company firewall” that does little more than divide network traffic into internal (‘good’) and external (‘bad’) sources.
When red tape is a good thing
The example above is no doubt a simplification of software development: Not all projects produce relatively non-proprietary front-end web code. When, then, is a highly-secured development pipeline, and the regulated and lengthy processes that come with them, a necessary evil? Situations that may meet these criteria include those where the code to be developed:
- Contains highly proprietary algorithms — Code that codifies an insurer’s rating algorithm or an investment bank’s investing strategy may call for a tightly controlled development pipeline. In these cases, the algorithm itself is a major source of the company’s business value.
- Must be strictly traceable— HIPAA-compliant software, among others, must undergo rigorous validation that maps directly to system specifications. A single, highly-secure, fully traceable development pipeline may be necessary when the development process must comply with certain laws and regulations.
- Exposes the company to significant risk — Certain types of software — such as that responsible for managing financial transactions— can have real-world impacts that expose a company to significant liability risk. Managing and effectively delegating accountability for this risk is especially necessary when the risk is high, contractors are involved, and the software to be developed lacks clear boundaries of ownership. A highly-regulated development pipeline can help a company effectively assess, manage, and mitigate this risk.
Development ops is not one-size-fits-all
The lesson to be learned is that no single development pipeline is right in every situation. For mission-critical software that requires the highest levels of security, processes and rules that inconvenience developers may be absolutely necessary. For some software, though, a more hands-off approach that gives the team greater flexibility is warranted.
Only by truly understanding a company’s competitive advantage and risk profile — and by being willing to tailor development processes based on the particular software being created — can an organization achieve the most effective balance between security and flexibility.
