Monitoring Your Business is Easier Than You Think

Learn how to set up webhooks in Databricks to receive monitoring alerts in Microsoft Teams

Photo by Austin Distel on Unsplash

Tools like Databricks allow you to send alerts directly to email by specifying email addresses. What do you do if you want to send alerts for applications like Microsoft Teams or Slack? For example, when a query result meets a set threshold value, you might want to notify users in Microsoft Teams. This requires setting up webhooks and the webhook URL.

In this article, we’ll explain how to send an alert to Microsoft Teams from Databricks.

How are webhooks different from APIs?

APIs serve as an interface to enable communication between two applications. Users send a request to an API and get a response in a two-way communication paradigm.

Webhooks are automated calls that enable syncing and communication between two applications. Webhooks are like mini APIs that perform one-way communication. They receive calls through event-driven HTTP posts when something changes or new data are available in the publishing application.

A step-by-step guide to using alerts with Databricks and Microsoft Teams

Photo by Ed Hardie on Unsplash

Step 1: Enable webhooks

First, enable webhooks in Microsoft Teams:

1. Right click on the Teams channel you want to send automated messages.
2. Choose Connectors → Configure ‘Incoming Webhook.’
3. Copy the URL provided and save it (you will use it in next steps to send information to Teams).

Microsoft Teams

Step 2: Create an alert destination

Next, you’ll create an alert destination in Databricks.

1. In your Databricks workspace, navigate to the ‘SQL’ persona from the left bar. Click on the ‘Settings’ gear icon → SQL Admin Console → Alert Destinations.
2. Add a New Alert Destination. Select Microsoft Teams and click ‘Create’.
   Other common destinations are email, Slack, and other collaboration tools.

Databricks SQL console

Step 3: Send webhooks to your destination

Finally, open a Databricks notebook and the code provided below to send your webhooks to Microsoft Teams.

1. Install the library ‘pymsteams.’

Notebook commands

Code:
%sh
pip install pymsteams

2. Import ‘pymsteams’ and create the below User Defined Function (UDF).
Note: You will need the URL you have saved from Microsoft Teams.

Notebook commands

Code:
import pymsteams
## Please make sure library pymsteams is installed in the cluster.
#Use the URL you have got from Microsoft teams when configuring webhooks
url=’https://-------------------------------------'

msg=’Message sent via Databricks — ‘
def sendMessageToTeams(webHookUrl: str,msg: str):
try:
# escaping underscores to avoid alerts in italics.
msg = msg.replace(‘_’, ‘\_’)
teams_msg = pymsteams.connectorcard(webHookUrl)
teams_msg.text(f’{msg}’)
teams_msg.send()
except Exception as e:
print(f’failed to send alert: {str(e)}’)

3. Call the function based on your alert conditions and requirements.

Notebook commands

Code:
sendmessageflag=False
df=spark.sql(“select sum(`Units Sold`) AS UnitsSold from test_sales_records”)
r=int(df.select(‘UnitsSold’).collect()[0][0])
if r>24999955:
sendmessageflag=True
msg=msg+’Units Sold exceeded’
if sendmessageflag:
sendMessageToTeams(url, msg)

Conclusion

Webhooks enable lightweight communication and can be used to send alerts and notifications from one application to another. As this article shows, it’s easy to setup webhooks in Databricks to a number of different applications, such as Microsoft Teams.