Photo by Photos Hobby on Unsplash

Operationalizing AI with Databricks, Microsoft and Slalom

Krishanu Nandy
Slalom Technology
Published in
4 min readOct 14, 2020

--

In Through the Looking Glass — Lewis Carroll’s sequel to Alice’s Adventures in Wonderland — the Red Queen tells Alice,

“…here we must run as fast as we can, just to stay in the same place”.

As someone who works in technology, the “Red Queen Effect” is all too real with some technologies barely lasting a few years before being replaced by something newer and shinier. This challenge is particularly true for companies that are attempting to take data-driven intelligence and machine learning (ML) into production and establish artificial intelligence (AI) as a core differentiator to their competitors.

The race towards developing AI capabilities isn’t unfounded since it is arguably the most disruptive capability to drive not just business transformation, but impact people’s lives. At Slalom, we have worked together with hundreds of organizations to address challenges such as process automation, generation of rapid insights, augmenting human decision-making and making sense of complex patterns. With ever-changing patterns of human behavior and consumption, businesses will need to be more and more intentional about their investments in AI as more and more companies enter the space.

The truth, however, is that AI is hard and the challenges numerous. With increasing digitization, more and more of what we as consumers do is available as raw data to companies such as yours which provide us with goods and services. The rise of IoT (Internet of Things) devices not only hints at a deluge of data but also the interconnectedness of everything we do. However, the seemingly simple act of collecting and organizing all this data requires the expertise of data engineers, data architects and cloud engineers. Garnering insights from the organized data subsequently leverages the skillsets of data scientists who may serve up their intelligence to internal teams, like marketing and FP&A, or use it to improve customer experience. In the latter case, collaboration with product managers and front-end engineers who design the interfaces that customers interact with is critical.

Given the breadth of personas required to successfully leverage data, the importance of people in this equation is clear — data, tools, and technology are nothing without people.

Ensuring that team members are empowered and encouraged to experiment and innovate, with the knowledge not all initiatives will be successful, requires a cultural shift. To this end, at Slalom we have developed a Modern Culture of Data (MCoD) framework to help you unlock the potential of your organization and realize investments in AI. The first step is strategic alignment of your business goals. Defining an overarching vision for your organization, which is sponsored by executive leadership, is key for business units to unify with a common sense of purpose. To achieve this vision, people in your organization need access to trustworthy data via flexible and user-friendly systems that appeal to both technical and non-technical audiences. The end goal is one where data and analytics are embedded into every decision your organization makes.

The five pillars that enable a Modern Culture of Data

To realize such a culture of experimentation and data-driven decision making, Slalom has partnered with Microsoft and Databricks to build a delivery framework using the best that Azure has to offer. By combining our collective expertise across industries, data engineering, data science and DevOps, we’ve put together an acceleration path capable of sprinting you through the machine learning enablement process.

Databricks, a company founded by the creators of Apache Spark, is a first party service on the Azure cloud which allows data engineers, data scientists and data analysts, to use cutting edge tools tightly integrated with the effectively infinite scalability, reliability and security of the cloud. Teams that use Databricks can access data and use open-source coding tools on a common platform, bringing IT and business silos together, resulting in reduced time to deployment. The flexibility of the platform means that technically savvy users are able to maximize time spent in their area of expertise while simultaneously being able to collaborate with and learn from users in other teams. With the backing of the Azure cloud, your team has the choice to leverage a multitude of existing Azure AI services without having to recreate every component from scratch. Azure also provides a plethora of options for quickly and reliably deploying to production the AI solutions your teams develop.

If your company is relatively new to AI, the challenge may seem overwhelming. The sheer number of problems you could focus on and the number of ways you could approach solving these problems is immensely difficult to navigate.

At Slalom, we recommend you think big but start small — lay a solid foundation driven by a data strategy and focus on one problem that demonstrates impact and builds momentum.

It’s easy to get caught up in buzzwords but more often than not it is the simple solutions that are most impactful for a company getting started with AI. Whatever your situation, as long as you have a use case to deliver value via AI and ML, our solution can accelerate the process without reinventing the wheel.

--

--

Krishanu Nandy
Slalom Technology

Most recently I’ve been fascinated by the intersection of data, technology, business and people.