How to Accelerate Development Velocity in the GenAI Era? Build a GenOS.

Because generative AI can follow instructions and reason through challenges, it enables developers to create solutions that automatically produce responses without having to explicitly code for them.

Intuit has already developed a variety of generative AI applications, including Intuit Assist, which is designed to guide small business owners in QuickBooks, create launch announcements in Mailchimp for marketers, help Credit Karma members across the credit spectrum make smart money decisions, and transform tax preparation in TurboTax. Behind the scenes, we’re also using generative AI to power our internal developer platforms to enable our software engineers and data workers to be more effective and efficient.

And this is just the beginning. We have many more projects in the pipeline to transform both employee and customer experiences.

The accelerating pace at which we are rolling out these solutions is possible because of a growing suite of tools that make up our GenOS, a proprietary operating system for developing and implementing generative AI-powered solutions. We built GenOS to solve an enormous challenge: making generative AI broadly available for all product teams to develop solutions that integrate the technology safely and responsibly into applications on our platform. This is just the latest in a series of investments Intuit has been making in platform and AI expertise with the goal of Democratizing AI.

Generative AI unlocks a wealth of opportunities

The first component of GenOS, GenStudio, was launched during our Spring Global Engineering Days, a twice-yearly event where we encourage technologists to pursue passion projects. GenStudio is the development sandbox where our engineers can responsibly access, experiment and build with third-party and proprietary large language models (LLMs). The response to its rollout was enormous: thousands of engineers took the opportunity to pursue hundreds of projects focused on solving customer problems with generative AI.

Developing generative AI at scale presents major challenges

To scale the power of generative AI application development at Intuit, we needed to solve three key challenges:

1. Responsible Development — generative AI and LLMs are constantly evolving. As they evolve, we must continuously consider potential risks to our customers and our users and identify strategies to mitigate them. We also account for security, legal, compliance and privacy requirements related to handling user and employee data on a single system across the company, aligned to our responsible AI and data governance practices.
2. The limitations of LLM’s domain knowledge — commercial and open source LLMs typically cannot solve Intuit’s customers’ most pressing business and financial needs off the shelf. There are several techniques to solve these problems in the industry including retrieval augmented generation (aka RAG), grounding LLMs with context on domain-specific data, and integrating with existing capabilities.
3. Support for rapid experimentation — generative AI technologies and customer expectations are changing at breakneck speed. To keep up, we need to provide our teams with the ability to experiment with a large number of potential solutions quickly so they can identify the best possible ways to leverage generative AI in their applications.

To address these three key challenges, we built a proprietary generative AI operating system (GenOS) that provides a paved path for accelerating GenAI development at Intuit. In the next section we will examine how the principal components of GenOS solve these challenges.

Principal components of GenOS

GenOS integrates a suite of components, each of which provides critical functionality for developing and deploying experiences that leverage generative AI at scale. One can compare it to a typical Operating System where application developers get access to an extensible framework for user interactions and rendering, a runtime which manages the resources to run applications, various services such as planner akin to a scheduler, short term memory and long term storage, of course the LLMs themselves and a pluggable ecosystem of domain specific capabilities similar to device drivers. It also has role based and use case based access controls to ensure the right level of access for various system components. It provides out of the box observability, governance and cost attribution so use case teams do not have to build these on their own.

GenStudio: the developer’s sandbox

GenStudio provides a developer’s sandbox for prompt engineering, evaluation and optimization. It provides a framework for teams to evaluate and optimize their prompts and their agents, allowing them to measure relevance, factualness and accuracy. Also, integrated controls test and detect potential issues while helping teams iterate their prompts quickly. GenStudio also provides a virtual playground in which technologists can make cost and performance tradeoffs while they develop their solutions.

GenUX: A library of user interface tools

GenUX provides a system of reusable libraries, widgets and components specifically designed for generative AI-based applications at Intuit, known as Intuit Assist. It extends the current UI engineering capabilities by adding user/view context, action management, conversation management and dynamic renderers for the proprietary rendering player.

GenRuntime: Where the rubber meets the road

GenRuntime hosts the generative AI capabilities available for developing new product experiences. It includes a set of sophisticated components that include GenOrchestrator, LLMs, our agents and tools ecosystem, knowledge retrieval, memory and various registries and policies. It’s worth looking at these components in greater detail.

GenOrchestrator: The brain of GenOS

GenOrchestrator orchestrates the various functions required to serve requests to GenOS leveraging generative AI. It includes the following primary components:

• Planner, which creates a plan to serve incoming requests by first understanding the intent of the request and then identifying the agents and tools required to execute it.
• Executor, which actually executes the plan via agents and tools.
• Memory and Knowledge Retrieval, which augments the planning and execution process, providing relevant context to ensure factual correctness.

Agents & Tools: A source of domain-specific capabilities

Since off-the-shelf commercial or open-source LLMs do not have Intuit’s domain knowledge, GenOS provides a plugin mechanism via agents and tools to ground them with Intuit-specific data. Agents and tools are registered via a plugin registry that manages their lifecycle.

Registries and Policies: Guardrails for security and privacy

GenOS registries and policies help enforce applicable guardrails so that authorization, data handling, legal, security, privacy and compliance requirements can be addressed. To solve for the wide range of applications developed using GenOS, we built an extensible and configurable framework to embed applicable controls eliminating redundant work, accelerating development and ensuring consistency with Intuit’s responsible AI practices.

Financial Large Language Models: Custom LLMs

Our strategy is to use the best LLMs to meet the needs of our customers. This includes our own financial Intuit LLMs, which are custom trained on Intuit’s domain-specific data, and specialize in solving tax, accounting, marketing, cash flow, and personal finance challenges. These custom LLMs help overcome the limitations of off-the-shelf commercial or open-source LLMs, providing increased ability for our teams to manage a variety of issues, including accuracy, cost and latency issues. We also use a variety of best-in-class commercial and open source LLMs from industry-leading providers, all in keeping with our responsible AI and data governance principles and practices.

Responsible AI: A framework for ethical deployment

Our responsible AI principles guide how we operate and scale our AI-driven expert platform responsibly. Along with our Responsible AI review processes, we embed controls throughout the GenOS architecture to not only address compliance, privacy and security concerns, but also to deliver delightful and personalized customer experiences.

We spent a considerable amount of time with the Legal, Privacy, Security, Authorization and Access Management, and Compliance partners to define the policies and practices we needed in place before we allowed use-case teams onboard to GenOS.

The Power of the operating system

For a young and evolving system, GenOS has been remarkably successful. 30+ Intuit Assist use cases powered by GenOS are already being tested and developed with customers, with several more in the pipeline. It has been an exciting journey with dozens of organizations coming together in a unified mission of building and delivering applications on GenOS across Intuit’s product portfolio. A lot of teams outside the core mission made significant contributions to the development of GenOS as well. Today, GenOS powers Intuit Assist experiences across our platform and products. GenStudio with its wide range of capabilities allows non tech employees to leverage the power of GenAI such as creating content or searching a knowledge base.

As we wrapped up our Fall Global Engineering Days recently, six months from where it all started with GenStudio, generative AI continues to remain the central theme with hundreds more ideas to test and iterate with customers. None of this would have happened this quickly and successfully without GenOS and key organizational strengths that paved the way for it.

Despite all these accomplishments, this is still just the beginning for GenOS. We continue to invest in automation and self-serve onboarding, rapid experimentation, and providing higher level abstractions such as for personalization. GenOS continues to enhance its capabilities by partnering with various special interest groups focusing on work streams such as Reusable Agents, Evaluation, Testing, and more. Its extensibility allows use case teams to bring in their own customizations to UI, Agents& Tools as well as guardrails.

As we have learned, the best way to accelerate generative AI capabilities across products is to make it available as a full-blown operating system for developers to build on. This ensures that product teams stay focused on generative AI applications and at the same time benefit from the collective knowledge of the organization as we continually evolve GenOS.