Leveraging Generative AI for the Software Industry
By Akash Bhatia, Julius Biedermann, Naila Dharani, and Nipun Misra
Generative AI is the hot topic in tech right now — and for good reason. ChatGPT hit 1 million users in five days, faster than Instagram (75 days) and Spotify (150 days). VCs have been investing heavily in this technology, and many new companies are popping up. While OpenAI’s GPT-3 leads the foundational model race right now, other players, such as Google, Meta, Anthropic, Cohere, and Character.ai, are hard at work creating their own models.
Generative AI’s capabilities go far beyond text and image generation, with applications in other modalities such as video, gaming, avatars, music, and even software code. Its wide range of potential applications, pre-trained models, and democratized access for users position it as a highly promising technology.
While we are at the peak of the hype cycle right now, many believe that generative AI is poised to disrupt almost every industry. Even the proud software development industry is starting to build generative AI capabilities into their own toolsets which will likely boost productivity by 2x, 5x, or maybe even 10x in the coming years.
In this article, we will focus on two key questions for the software industry:
- How can software developers leverage generative AI?
- What opportunities are there for the tech industry?
Leveraging generative AI across the software development lifecycle
We believe generative AI will be a massive productivity booster in software development within six main areas of the entire lifecycle: (1) Research and Design, (2) Code generation, (3) Security review and bug detection, (4) Documentation, (5) DevOps automation, and (6) ML Ops automation.
There are still gaps and limitations, but the good news is for many high value use-cases, many tools already exist and are likely to mature in the next few years (see exhibit 1).
Consequently, the market landscape of generative AI for software developers is becoming increasingly crowded. There are a growing number of emerging players, most of whom are focused on the app layer and are building upon a few foundational models (mostly GPT-3). We have built a running database of ~60 emerging players in this space; new, exciting players are appearing every month.
While it is still the early days, these tools are going to fundamentally change how software developers work and how they spend their time. Here are some commercial examples that are already available today:
· Researching code and data, or understanding existing code: AI2SQL can automatically write complex SQL queries, and SEEK can search and pull code from numerous data sources.
· Code Generation: Writing boilerplate code, repetitive functions, or tests will be massively accelerated by contextual tools. Github Copilot is probably the most popular and established tool right now with >1M paying users, as it is already improving developer productivity by 10–50%.
· UX/UI Design: Galileo AI translates natural language design prompts into design suggestions.
· Code documentation: Mintlify can automatically document code within seconds in a standardized way.
· Bug detection & vulnerability management: Metabob lets you find and fix code faster than human software developers are able to do today; it also identifies potential risks and suggests recommended solutions for them. Crucially, it has a dashboard that runs static code analysis across large code bases.
That said, these tools are not going to replace software developers — at least not anytime soon. But productivity will be multiples of what it is today, and developers will pivot to focus on very different tasks (e.g., architecture, ideation, and interacting with these tools). A 2022 MIT study[1] already shows that developers who used Github CoPilot spent ~50% of their time interacting with Copilot itself.
Gen AI Implications for software engineering leaders
While productivity gains are expected to be massive and developers are (mostly) going to be happier, the technology doesn’t come without risks: Buggy code, copyright liabilities, and security vulnerabilities, to name a few. Copilot, for example, is already being sued for violating copyright law; an NYU study[2] showed that Copilot generates code with security flaws about 40% of the time.
It will take time for these tools to fully mature, but engineering leaders should already start to think about how to responsibly leverage this new technology.
Here are some key action items:
1. Create a responsible AI strategy
2. Define clear governance for the use of generative AI
3. Establish key processes with quality controls
4. Support developers in using approved tools
5. Train teams appropriately to responsibly use AI tools
In the long run, engineering leaders will have to consider other strategic moves, such as shifting engineers to higher-level tasks and upskilling their teams to focus more on architecture, cybersecurity, and quality testing.
One thing is clear: Software development will not look the same in five years.
Leveraging Generative AI for B2B Tech players
We see several opportunities for B2B tech players across the generative AI stack (see exhibit 2). While most funding activity has been targeted towards foundational LLM models, many new startups are building solutions on top of them in the app layer, which is likely where most of the value creation is going to occur.
[1] “Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming.” MIT November 2022.
[2] https://cyber.nyu.edu/2021/10/15/ccs-researchers-find-github-copilot-generates-vulnerable-code-40-of-the-time/
Despite the growing number of emerging players in this space, we expect that there will not be a proliferation of foundational models and that approximately three to five will dominate. The Hyperscalers (Google, Microsoft, and Amazon) have the highest probability of winning out, given their scale advantage, access to data, and ability to generate partnerships (e.g., Microsoft’s investments in OpenAI). The Hyperscalers will also have opportunities to build apps with and without proprietary models, platforms or model hubs.
We expect that traditional software players will have the most opportunities in the app layer, and, of course, in augmenting their existing product suite with generative AI functionality. For example, ServiceNow is building automated workflow generation into its existing toolkit. We similarly expect the Hyperscalers to modify their existing products with generative AI capabilities.
For smaller startups, the app layer is likely the most promising avenue for investment. But for those who choose to be vertical-specific or operate in a particular niche, there is also a significant opportunity in building end-to-end apps with proprietary models (see exhibit 3).
Naturally, this field will continue to evolve. However, we expect that three core principles will hold throughout this generative AI development cycle: 1) data will remain a competitive advantage, 2) companies who establish credibility and expertise in the space early on are more likely to win, and 3) protecting yourself from exposure and risk will drive trust and adoption of this game-changing technology.
As with any new technology, generative AI will require time and careful consideration before its benefits can be maximized. In the meantime, software engineering leaders will have to balance unlocking productivity gains with minimizing risk, all the while ensuring they do not create overly bureaucratic processes. Tech is undoubtedly going to remain a provocative and disruptive industry, with potentially surprising winners and losers.
The authors would like to thank Surya Ambardar and Miriam Rollock for their help in conducting research for this article.
