Debu SinhaScaling Entity Resolution in the Modern Data Ecosystem using BERT, MLflow and DatabricksEntity Resolution (ER) is a crucial task in data processing that involves identifying and linking records across one or more datasets that…Apr 2Apr 2
Debu SinhaExploring Advanced Custom Transformers in Apache Spark for Enhanced Machine Learning Workflows on…In the dynamic field of machine learning, the ability to craft efficient and robust pipelines is crucial, especially when dealing with…Mar 18Mar 18
Debu SinhaOptimizing ML Model Serving with Pre-logged NLTK Downloads in MLflow on DatabricksI encountered a significant challenge while collaborating with a client on a Natural Language Processing (NLP) project. The machine…Nov 18, 2023Nov 18, 2023
Debu SinhaScaling your single node Machine Learning Model training using Pandas Function API on Databricks…IntroductionMay 2, 20231May 2, 20231
Debu SinhaUsing and scaling SHAP on Databricks for turbocharging your model explainability. (Part 1)I have been working in ML and AI for the past 10+ years. As a Specialist Solutions Architect at Databricks, I get opportunities to work…Apr 5, 20231Apr 5, 20231
Debu SinhaUnderstanding Loss Functions in MLThis blog is part of my 30 days of ML series, where I will cover the fundamental concepts of ML while going into detail on technical topics…Dec 30, 2022Dec 30, 2022
Debu SinhaUnderstanding Recurrent Neural NetworksRecurrent neural networks (RNNs) are a type of neural network that are designed to process sequential data. This means they are…Dec 30, 2022Dec 30, 2022
Debu SinhaAll you need to know about writing custom UDF using Python in Apache Spark 3.0.If you want to work with Apache Spark and Python to perform custom transformations on your big dataset in a distributed fashion, you will…Mar 13, 2022Mar 13, 2022
Debu SinhaDeploying your first ML model as a REST endpoint using Flask and Bootstrap.As a Senior Solutions Architect at Databricks outside of my client interactions, I regularly give talks and write articles about ML, MLOps…Jan 24, 2022Jan 24, 2022
Debu SinhaCheatsheet on understanding ZOrder and OPTIMIZE for your Delta tables.IntroductionNov 15, 20212Nov 15, 20212