Azure Databricks log runtime errors to Application insights

Balamurugan Balakreshnan
Jul 13 · 2 min read

Using open census library to push error logs to Azure monitor

Prerequisite

  • Azure account
  • Azure Databricks
  • Azure Storage

Steps

  • Create a Databricks cluster
  • install library
opencensus-ext-azure
  • Create a notebook

Code

  • Choose python as language
import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler
logger = logging.getLogger(__name__)# TODO: replace the all-zero GUID with your instrumentation key.
logger.addHandler(AzureLogHandler(
connection_string='InstrumentationKey=xxxxx-xxxxxx-xxxxxx-xxxxxxx')
)
  • now log some sample logs
logger.warning("Sample from open census test 01")
logger.error("Sample from open census test 02")
  • Now lets log an exception
from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace.samplers import ProbabilitySampler
from opencensus.trace.tracer import Tracer
properties = {'custom_dimensions': {'key_1': 'value_1', 'key_2': 'value_2'}}# Use properties in exception logs
try:
result = 1 / 0 # generate a ZeroDivisionError
except Exception:
logger.exception('Captured an exception.', extra=properties)
  • Log into Application insights and go to logs

Originally published at https://github.com.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Balamurugan Balakreshnan

Written by

https://balakreshnan.github.io/

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com