Pawan Kumar ShuklaRAG-Chat with your relational database from Azure Prompt Flow with LLMMost of the organisations want to use LLM based capability to provide accessibility of data for other departments /groups as per role…4 min read·Jan 15, 2024--1--1
Pawan Kumar ShuklaRAG with your Custom Data : Azure Prompt flow + Azure AI Search(Azure Cognitive Search)RAG(Retrieval-augmented generation), use case of Vector DB.5 min read·Dec 4, 2023----
Pawan Kumar ShuklaHarnessing the Power of Large Language Models with Azure ML PromptFlow : Part 1 (Introduction)In today’s world, Artificial Intelligence (AI) is revolutionizing various industries by enabling intelligent automation and data-driven…4 min read·Nov 18, 2023----
Pawan Kumar ShuklaAzure Databricks workflow for ingest the csv with which have header, body and footer in…In the Data Engineering work it is not always you will get a flat format csv file. Sometime you will get the csv files with Header, Body…2 min read·Sep 30, 2023----
Pawan Kumar ShuklaTips: Use _metadata instead of input_file_name() function in Databricks runtime 10.5 and aboveWe can use the _metadata, a hidden column name for DBR 10.5 and above.1 min read·Sep 30, 2023----
Pawan Kumar ShuklaHow to reduce the storage cost for delta tables?As now these days industry is moving towards more on Data Products and almost every organisation is going to build their own data lake…2 min read·Sep 30, 2023----
Pawan Kumar ShuklaHow to update existing delta tables mount point ?Recently we got the feedback to rename the existing mount point with new mount point. And 100 delta tables were mounted on that old mount…1 min read·Jul 25, 2023----
Pawan Kumar ShuklaHow to implement complex operation with Spark Structured streaming using foreachBatch?1 min read·May 28, 2023--1--1
Pawan Kumar ShuklaSpark Structured Streaming : Read and process data(XML and json)from Azure Event hubFor ingest the event driven data(real time ) in bronze layer(Medallion- architecture…2 min read·May 21, 2023----
Pawan Kumar ShuklaExtract the data from Databricks delta table with help of Azure Data factory Copy activity with SQL…There is many ways to generate the csv file from Databricks delta lake, I am going to explain easy, cost efficient and with no…3 min read·May 19, 2023--1--1