Rupesh GuptaUnlocking the Power of Azure Databricks: Bridging the Gap Between Private ADLS and OneLakeWhen it comes to harnessing the full potential of Azure storage, particularly in scenarios involving private endpoints, one often…Sep 5, 2023Sep 5, 2023
Rupesh GuptaAuthentication in Azure pipelines using Azure Key Vault secretsSecurity is one of the main factor in your project/organization/life. So risks are increased drastically if the credentials are…Sep 20, 2022Sep 20, 2022
Rupesh GuptaRunning Azure Machine Learning Pipeline from Azure Data FactoryRecently, I came across a problem for machine learning pipelines. With the collaboration of data scientist, I was able to solve it. We use…Sep 3, 2022Sep 3, 2022
Rupesh GuptaMonitoring Azure Data Factory PipelineAs a Data/Ops Engineer, we have to take care of running pipelines. There are so many ways to handle them but I have handled it using alerts…Aug 27, 20221Aug 27, 20221
Rupesh GuptaContinuous Integration and Continuous Deployment of Azure Databricks NotebooksThere are many ways to build and deploy your databricks notebooks across the environments but my main aim will be to show what many of us…Aug 21, 2022Aug 21, 2022
Rupesh GuptainDevOps.devAzure Data Factory: CICDAll of us know how development and operations are improved after DevOps. Instead of sharing theory on DevOps, I will explain how we were in…Aug 14, 2022Aug 14, 2022
Rupesh GuptaDocker based Postgres DB — docker-compose.yamlTechnology is growing too fast. Most of times, we are so excited to learn all the hot technologies out there in the world. Almost…Feb 15, 2022Feb 15, 2022