Generative AI’s Impact on Software Development

AI Assistant (Image generated using Midjourney)

Generative AI and Large Language Models (LLMs) are transforming the way software engineers work. These tools have the potential to make engineers 2x productive if not 10x. While AI development has been active for a couple of decades, the democratisation of AI through ChatGPT has become an inflection point for the software industry.

The transformative impact of generative AI parallels groundbreaking milestones such as the use of the internet (Google and Stack Overflow) for coding, the advent of the iPhone which sparked a mobile app revolution, the rise of microservices architecture, and the widespread adoption of DevOps and cloud technologies. Personally, I believe these tools will empower engineers instead of replacing them. Generative AI will impact various aspects of the software development process, here is how:

1. Improve productivity and reduce the cost of development

One of the major uses of Generative AI is to improve the efficiency of current Software Engineering practices, thereby increasing productivity and reducing the cost of development.

• AI code assistants like Github Copilot and Amazon CodeWhisperer can be used to boost developer productivity. They do so through the auto-generation of language-specific or framework-specific boilerplate code. Additionally, they can add unit tests and generate test data. They are also useful in identifying and fixing security vulnerabilities.
• Generative AI tools enable quick prototyping, and consequently, rapid evaluation of ideas.
• Automation of mundane tasks such as creating pull requests and documentation.
• Tasks such as legacy code migration and conversion of code from one programming language to another can be automated.
• As the context window of these models is being pushed to 100k tokens and more, these tools may soon be able to understand the entire codebase and support continuous refactoring to improve code maintainability.
• Using natural language for log analysis, monitoring and database query will make it easier for developers to analyze and understand complex problems quickly.

2. Reimagine Software Development (Model-as-a-Backend)

Generative AI and LLMs open up new ways of building software in a refreshing departure from the traditional approach. Instead of building a traditional backend service and database, custom models can act as the backend. We can load all the application data into the model’s context and use natural language to post and retrieve data instead of RESTful API or GraphQL. With this approach, teams only need to build a frontend mobile or web app and it can communicate with model-backend using natural language.

3. Enhanced UX to drive great customer outcomes

In today’s competitive landscape, businesses strive to elevate their end-user experience to new heights. Large Language Models (LLM) can play a pivotal role in achieving this by leveraging their capabilities in areas such as transformation, summarization, inference and expansion. By incorporating LLMs, businesses can revolutionize how users interact with their platforms and services.

Users can use natural language when searching for data, eliminating the need for cumbersome dropdown selections. Dynamic reports can be generated effortlessly, eliminating the need for code changes. Furthermore, businesses can leverage LLM-powered chat-based internal portals and customer support applications to provide prompt and accurate responses to end-customer queries.

4. Advanced Enterprise Models: Leveraging Specialized Training on Private Data

Companies can build their own machine learning models trained on private data, and integrate them with LLMs like GPT to provide capabilities that are currently impossible with custom software.

Consider a global investment firm that has accumulated vast amounts of financial data, including historical market trends, company reports, and economic indicators. They can build a sophisticated enterprise model that has been trained on their proprietary data, incorporating knowledge specific to the financial domain.

By integrating this specialized model with a large language model like GPT, the firm could enhance its investment decision-making process. The system can provide in-depth insights into potential risks, opportunities, and optimal portfolio allocations based on the firm’s proprietary data and the contextual understanding obtained from the large language model. These models can generate detailed narratives written in a human-like manner, explaining the rationale behind investment recommendations and providing understandable explanations for complex market dynamics.

Frameworks like LangChain are getting popular for building applications by chaining multiple LLMs and integrating them with other internal systems.

5. Transforming Automation in Software Engineering

Large Language Models have the potential to revolutionize automation in the field of software engineering. Instead of relying on traditional script-based automation for tasks, autonomous agents such as Auto-GPT enable the automation of complex tasks using LLMs through natural language interactions. These agents can accomplish assigned goals autonomously. They are capable of making decisions independently and can work towards achieving various objectives. The best part is, they do not require constant human intervention or prompts.

The journey has just begun — what’s to watch out for?

While generative AI tools have many advantages, there are also concerns and limitations associated with their use:

• One of the most significant concerns is the possibility of sensitive data or code being transferred out of a company’s network.
• There is a possibility that the code or data submitted to these tools might be stored and used to improve or train the model, raising potential intellectual property concerns.
• Licensing issues can also arise if the generative AI tool is trained with copyrighted or open-source licenses that are restrictive, complicating the use of these tools. Microsoft, GitHub and OpenAI are being sued in a class action lawsuit, further highlighting the potential legal issues associated with generative AI tools.
• Outdated training data of these models will result in code suggestions with deprecated parameters and outdated library versions which in turn will result in security issues.
• AI tools at the current level don’t excel at high-level design and architecture. Using these tools without proper expertise might result in complicated solutions and architecture.
• Prompting is a skill, and unfortunately, not everyone is good at it. Garbage in, garbage out. The use of wrong prompts might result in poor technical design and more technical debt.
• Additionally, generative AI tools may make up facts based on patterns (LLM hallucination) which may not be entirely accurate or appropriate. This can lead to errors or unintended consequences.
• As with any new technology, it is essential to have a company policy in place when using generative AI tools. A company policy can ensure that the use of generative AI tools is both legal and ethical. There is an excellent post that highlights some important policies such as risk rating, reporting, labelling, record-keeping, employee training program, monitoring and auditing.

In conclusion, Generative AI and Large Language Models are significantly reshaping the software industry, offering immense potential for productivity gains and innovative practices. This marks a promising new era in software development, one that requires careful navigation to unlock its full potential.