Eye on design — 10 keys to AI success

By Rajaram “Raj” Venkataramani, Senior Director

Analytics solutions are supported by a set of well-known principles: knowing your business audience, a seamless data supply chain, consistent dashboards and intuitive visualizations. The same cannot be said for AI. With the right set of principles, however, organizations can create enterprise AI systems that effectively scale, with minimum cost and risk.

Here’s a list of key design principles for AI solutions, based on lessons learned from our many successful AI implementations. Using these guidelines, business analysts, data scientists, data engineers and AI/machine learning (ML) engineers can create top-level AI solutions and enterprise AI services.

1. Design to experiment, learn and deploy quickly

Enterprises use AI algorithms such as deep neural networks, random forests, support vector machines and linear and logistic regression. Because these have been designed, developed and tested over time, we expect them to work. However, because of differences in domain-specific datasets, AI algorithms that work in one context may not yield acceptable results in another.

To avoid this development trap, businesses should use an experimentation sandbox to apply AI algorithms to domain-specific data and feature sets. If these experiments do not deliver high-quality results, they can be terminated with minimal loss of cost and time. This kind of experimentation environment exploits cloud environments, minimizes development time and more closely mimics real life. For example, it can replicate production-like processing nodes in the development phase to create deployable AI solutions and provide the ability to chain models to run on server clusters.

2. Design for scale and resilience

The real business impact of AI happens when it is done at scale, which means three things:

• As businesses move through the AI journey, the positive impact of their solution is verified at many touchpoints along the way.
• Enterprise-wide processes are specifically instrumented to enable the creation of intelligent customer journeys.
• Computing resources are available to train, test, deploy and execute resource-intensive algorithms capable of consuming large amounts of data.

Achieving these goals requires a well-designed AI solution that is able to scale both vertically, by adding more computing capacity, as well as horizontally, by using distributed computing nodes. It should be possible to add and remove processing nodes, including GPUs on-demand. The scalable AI design should be partitioned around the limits of the networks, data stores and operational processes to build in resilience, ensuring availability, performance and security.

3. Design for evolution

All successful AI systems change over time as they adapt to new data in the ecosystem. The efficacy of business solutions depends on the management of this change. In AI systems, these changes typically originate from two places:

• The user community identifies new requirements.
• Shifts occur in the characteristics of the input data.

Both changes alter the problem being solved by the AI system. The AI models need to be designed to tune their performance and reconfigure in response to new stimuli.

4. Design for the hybrid multi-cloud

When you play a video game, you use the console the game was designed on because it’s the optimal platform for the game. Enterprises are adopting a multi-cloud strategy for precisely the same reason — each cloud platform offers core strengths and gaps. In some cases, open-source, third-party products or startup solutions provide the best option for specific capabilities.

During or even after a cloud migration, it is likely that some systems will continue to run on-premise. So, a pragmatic AI solution design will support this hybrid approach. Additionally, the solution should include working in tandem with third-party frameworks, libraries and services to address the gaps and drive innovation. Though this approach introduces significant complexities around development and deployment architectures, network topologies and security patterns, it also offers opportunities to unearth and improve resilience, reduce total cost and accelerate time to market.

5. Design for operations control

Traditional deterministic logic systems are designed for visibility into system processes. They make it possible to know the impact of an application or a network crash on a stakeholder, and there are clearly defined processes for automated or manual system recovery.

With the inherent non-deterministic nature of AI systems, however, a model in production may not produce the desired outcome. This can even go undetected if, for example, a fraudulent transaction is not marked as fraudulent or vice versa. In both cases, this may or may not be caused by an error in the model. The model may be working as designed, but the environment may have changed without the model adapting effectively to it.

The traditional approach of reacting to issues in production systems based on a designated severity level is not sufficient. New controls are required to monitor regressive models, and re-configure, re-train and re-deploy as necessary to keep up with the changes. In a normal operations scenario, there needs to be a way to trace and explain lineage and insight-gathering processes.

Optimal AI systems are designed to deliver information and controls directly to the operations team. They also provide regulatory compliance checks and organizational controls in real time.

6. Build to mitigate risks while delivering on business goals

As with any system, every AI design decision should be justified by and traceable to a business requirement or constraint. In AI systems, however, there are inherent risks due to the probabilistic nature of system behavior, as opposed to deterministic rule-based systems. As a result, risk mitigation should be considered by design and be well-articulated.

For example, a fraud-detection model may be tuned to be highly accurate and could incorrectly mark a transaction as fraudulent, whereas a human operations expert would carry out other checks to validate the decision.

Any action that’s taken needs to be made only after considering the legal, regulatory, customer experience, customer loyalty and corporate reputation ramifications. Therefore, an AI system must have multiple models that look at situations through different lenses and arrive at decisions and actions that would minimize risk. AI governance should be set up to ensure that the human stakeholders who are responsible for system operations are informed quickly of any risk.

7. Design AI DevOps to support democratization

AI systems tend to be created and managed with collaborative teams. The players can include product managers, business analysts, social scientists, data scientists, AI and ML engineers, data analysts, data engineers and cloud engineers. Due to the diverse backgrounds of the team members, the AI DevOps platform should encourage effective communication and easy collaboration.

Also, when applying AI is everyone’s agenda, a cultural shift will need to happen to accommodate change at scale in the enterprise. Compared with traditional software, AI systems tend to go through a high number of lifecycle iterations. Even while in production, models regress and may require adaptation to changes in data and feature sets. Therefore, effective AI DevOps subsystems need to support a democratized, team environment.

8. Design for polyglot and multi-modal development

Not only are data volumes growing rapidly — so are data types. There are different types of structured and unstructured data in text, voice and images, all of which require different data stores and streaming technologies to support them. The AI system needs to support multi-modal development, including relational, graph, document, columnar and other kinds of data.

Both traditional and new languages are evolving to build algorithms that operate on these data structures. These languages aim to solve diverse classes of problems that appeal to different subject matter experts. This includes support for polyglot development, such as Python, R, Java and C++. Project managers need to anticipate and plan on this extra layer of complexity in terms of teams, tools and lifecycles.

9. Design for AI governance

In traditional systems, the points of failure are well-known around applications, data and infrastructure. Due to the deterministic nature of these systems, their behavior is easy to classify as successful or erroneous based on which response can be designed. AI systems, however, are modeled around human nature and don’t often classify behavior in binary terms. They need to be capable of monitoring sophisticated responses to alerts, complex events and exceptions.

AI governance includes traceability and “explainability” by design to ensure business goals are met and ethical and regulatory compliance issues are addressed. Designing with governance control to business, technology and operations ensures that the resulting system is compliant.

10. Design for security

All these drivers of AI system design impose new challenges in terms of handling data securely while at the same time providing greater agility and flexibility. The experimentation sandbox and multimodal environment will require additional data safeguards.

This means that while designing for a hybrid multi-cloud, data access for human agents and AI systems must be controlled. Also, certain AI algorithms are only available on the cloud, while others should only be run on-premise. AI systems need a key focus on security to keep the data within its allowed boundaries and provide access only to the people and systems that require it.

Supporting an AI-Driven Future

A well-designed AI system evolves with the business and supports the requirements of different stakeholders. It allows innovation and experimentation and scales to demand in a resilient, compliant and secure environment. To quote Andre Ng, “AI is the new electricity” but to harness it safely, enterprises should invest in robust engineering solutions.