Introducing LangChain toolkits for Azure Cognitive Services

Building a multimodal application powered by GPT

In my previous article, I introduced LangChain agents as applications powered by LLMs and integrated with a set of tools like search engines, databases, websites, and so on. Within an agent, the LLM is the reasoning engine that, based on the user input, is able to plan and execute a set of actions that are needed to fulfill the request.

LangChain offers a set of tools to integrate your agents with external services like Bing Search, local File Systems, YoutTube Search, and so on. Some integrations however, like CSV, Pandas Dataframe, and Gmail, need a particular set of tools to properly function. Those sets of tools are called toolkits, and today we are going to explore Azure Cognitive Services toolkit to extend LLMs with multimodal capabilities.

Azure Cognitive Services

Azure Cognitive Services are cloud-based AI services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. They are available through REST APIs and client library SDKs in popular development languages.

Azure Cognitive Services can be categorized into five main areas: