Building Snowflake Connector for ServiceNow

In March we announced that the native Snowflake Connector for ServiceNow was available in public preview. The connector provides instant access to up-to-date ServiceNow data without the need to manually integrate against API endpoints. Natively built inside the Data Cloud using Snowflake’s Native Application Framework, the connector leverages built-in security and reliability capabilities. In this article, we’d like to walk you through the main building blocks of the application and the benefits of using the framework.

Direct connection

The app communicates with your ServiceNow instance directly from Snowflake. There’s no 3rd party services between. The data is retrieved from ServiceNow using Table API and is stored in the Snowflake table.

For sending external request the connector uses external functions with response and request translators

The connectivity is done using external functions together with response translators. Both are built-in functionalities of Snowflake and can be used by anyone. The external function provides you with automatic retries for known retryable errors.

Security

Snowflake values very high privacy and security of your data. So by processing and storing the data in Snowflake we were able to take advantage of the robust security features like RBAC. Specifically for connector purposes, we developed a new concept of secrets that allow you to access your ServiceNow credentials in a secure manner. The connector supports both basic and OAuth authentication.

Data governance

As mentioned previously the data from ServiceNow lands directly in Snowflake. Ingested data is decoupled from the connector and thanks to that you can safely uninstall the connector without losing the data. Connector also generates runtime logs. The log data is stored safely in the event table that lives in the customer account. You as customer can query the logs. However, the data is never automatically shared with provider. It is possible for troubleshooting purposes but requires your additional consent.

Connector application ingests data from ServiceNow but the data is stored in a database outside the app.

Alerts

In addition to telemetry logs the connector uses Snowflake alerting capabilities to send users emails about encountered errors or pending upgrades. Alerts are based on notification integration and system function for sending emails. Both are open to all developers and can be used in your application.

Scalability and performance

By building the connector as a native application we’re able to take advantage of Snowflake’s robust built-in and reliable task system and the concept of virtual warehouses. By running many tasks in parallel we can parallelize the ingestion process thus increasing the speed of data load.

By using multiple tasks the connector can ingest data for multiple tables at the same time.

Cost control

Speaking of virtual warehouses. Building the application on Snowflake means that full control over its consumption is in customers hands. The application requires a warehouse and it is customers who control the warehouse size which correlates with the credit consumption.

What is more, the connector takes advantage of the auto-suspend feature of virtual warehouses. This means that the credits are consumed only when data is being ingested and the amount of burned credits is proportional to volume of ingested data.

Summary

The Snowflake connector for ServiceNow takes advantage of many existing Snowflake features. What’s important is that all those features are already available to our customers and can be used to build secure, scalable, and performant applications.

Try out the Snowflake Connector for ServiceNow by installing it from Snowflake Marketplace. It’s available with no additional licensing cost and uses Snowflake credits for compute and storage. You can also explore the product documentation here and follow the handy quick start tutorial.