There Are 2.08 Trillion Reasons to Build Better Software
It has been said before, but it needs to be said again — and again: Software can make or break your business.
If it’s high-quality, with security “built in” throughout the software development life cycle (SDLC), it can streamline your operations, protect your assets, and help you create and deliver products and services that can make you prosperous.
If it is written poorly, with little attention paid to security, software can make you an easy target for online attackers who can exploit its vulnerabilities to steal your intellectual property, your money, and your customers’ personal and financial information. It can destroy your reputation, leave you on the hook for legal and regulatory liabilities, and cause other disasters.
So you might think the vast majority of organizations would make the quality and security of their software a very high priority. Who wouldn’t want to be on the “make” side of make or break?
But you would be wrong. A report released last week by the Consortium for Information & Software Quality (CISQ), and cosponsored by Synopsys, titled “The Cost of Poor Software Quality in the US: A 2020 Report,” sets that cost at a staggering $2.08 trillion over the past year — more than the GDP of all but a dozen countries.
And that doesn’t even count an estimated $1.31 trillion in “technical debt” — accumulated software vulnerabilities in applications, networks and systems that have never been addressed. The report says that debt has been increasing at a rate of 14% since 2018. It isn’t added to the $2.08 trillion because it applies to future costs, not those from the past year. Still, that debt will have to be paid eventually.
If there is any marginally encouraging news in the report, it is that last year’s cost of poor software quality (CPSQ) is slightly less than CISQ’s 2018 estimate, originally put at $2.8 trillion but revised down to $2.1 trillion.
The more discouraging takeaway is that the major reasons for such staggering losses are the same ones that have existed, and that experts have warned about, for years.
In other words, organizations already know, or ought to know, what to do to cut their losses caused by bad software. Most of them just aren’t doing it.
Primary causes
There are three primary causes of those losses:
- The large majority (75%) of the CPSQ is software failure due to the failure to patch known vulnerabilities. The estimated $1.56 trillion is up 22% since 2018. Notice that these aren’t so-called “zero-day” vulnerabilities that nobody has discovered. They are known. There are patches available. They just aren’t being applied. That’s a bit like getting a recall notice that your car’s brakes could fail at any time, with an invitation to bring it in to the dealer, and not bothering to get them fixed.
- Second, at $520 billion, are legacy system problems, although that figure has declined from $635 billion two years ago.
- Third, at $260 billion, are unsuccessful development projects. And that number is trending the wrong way — it’s up 46% since 2018.
All three are different facets of the primary finding of the report: Software quality (which includes security) isn’t keeping pace in a world where development and operations have combined into DevOps and the speed of development has increased by orders of magnitude.
While there are multiple factors causing project failures, “one consistent theme has been the lack of attention to quality,” the report said, noting that organizations are not very good at “the balancing act” required to maintain quality and security in a DevOps environment.
This is a problem that, if not addressed, will lead to even greater CPSQ costs, for a number of reasons.
First, as numerous experts have been saying for most of the past decade, the software “attack surface” is explosively expanding.
The report cites Susie Wee, vice president at Cisco, who said in May 2017 that there were more than 111 billion lines of new software code being produced each year. And according to the National Institute of Standards and Technology, typical software contains an average of 25 errors per 1,000 lines of code.
That should be no surprise, given that we are getting to the point where, as numerous experts say, “everything is a computer.” The Internet of Things (IoT) is so last decade. It’s now the Internet of Everything (IoE). The IoE is not just your laptop, tablet, smartphone and smart watch. It’s your car, appliances, TV, door locks, heat, water — even clothes.
And when software is in everything, there are so many more ways it can hurt as well as help you. If hackers breach your home security system, you’re not safe. If they get control of your smart thermostat, your pipes could freeze while you’re away. Etc.
Beyond that, “products and services that were traditionally delivered through other means are now being run on software and delivered as online services,” the report says, noting the stark contrast between “the fall of Borders and the rise of Amazon.”
It also predicts that “while emerging technologies currently account for only 17% of overall global revenue, they are expected to drive nearly half of the growth in new revenue in the coming years.”
That means software is not only a crucial element in the success or failure of products and businesses, but of the overall economy itself. Saving $2 trillion could provide a major boost to the economy.
Fix it later, pay more
Second, it’s more expensive — by a factor of as much as 10 — to fix software defects after a product has been released than to find and fix them during development.
That message is constantly preached at security conferences and it has been documented for the past decade in the Building Security In Maturity Model (BSIMM), an annual report by Synopsys on the software security initiatives of dozens of companies, primarily in nine verticals.
Yet the CISQ report states that “The biggest bucket of CPSQ that we have identified is in operational failures ($1.56 trillion in the U.S. in 2020),” primarily caused by the failure to fix software defects during development.
So if there is a New Year’s resolution that every organization ought to embrace for 2021, it’s to improve the quality and security of their software. Especially since the template and the tools are available to do it.
As the BSIMM and numerous other reports, surveys, blog posts, and keynote speeches tell you, it requires implementing security measures at every stage of the SDLC.
Those security testing measures include:
- At the start, architecture risk analysis and threat modeling can help eliminate design flaws before a team starts to build an application or any other software product.
- While software is being written and built static, dynamic, and interactive analysis security testing can find bugs or other defects when code is at rest, running and interacting with external input.
- Software composition analysis can help developers find and fix known vulnerabilities and potential licensing conflicts in open source software components.
- Fuzz testing can reveal how software responds when it is hit with malformed input.
- Penetration testing, or “red teaming,” can mimic hackers, to find weaknesses that remain before software products are deployed.
Beyond those, the report also recommends that organizations:
- Analyze source code regularly prior to release to detect violations of quality rules that put operations or costs at risk. System-level violations are the most critical since they cost far more to fix and may take several release cycles to eliminate.
- Treat structural quality improvement as an iterative process to be pursued over numerous releases.
- Patch known vulnerabilities, a large percentage of which are listed as CVEs [Common Vulnerabilities and Exposures] by the National Vulnerability Database. Also eliminate the most egregious CWEs [Common Weakness Enumeration]. The top 25 CWEs are here.
“If all new software were created without those known vulnerabilities and exploitable weaknesses, the CPSQ would plummet,” the report said.
All these measures take time and money to implement. But the amount of time is decreasing, thanks to intelligent orchestration of software testing tools, an automated way to help developers do the right security tests at the right time without overwhelming them with false positives or irrelevant defect notices.
And the benefits of those investments are huge. “The biggest bang for CPSQ investment money would be in preventing most of those from occurring as early as possible (if at all), when they are relatively cheap to fix,” the report says.
Most important, better software security will yield multiple financial benefits including lower cost of ownership, higher profitability, better human performance levels, increased innovation, and more effective mission-critical IT systems.
Or, as noted at the start, an investment in good software can “make” your entire business.
