Let’s talk about events
I often talk to to customers with use cases that revolve around reacting to certain things happening in their Box instance. Eg. when a file is uploaded in a certain folder, when a collaboration is added to certain sensitive content, when a document is signed or even when a file is previewed they want to take further action. This can be anything from notifying another system to start or continue a workflow, to looking up and applying metadata in Box from a different source. Obviously the Box first-party applications already support user notifications for these scenarios so in this article I will outline some approaches to the backend automation side of this.
The automation of these scenarios normally present as one of two different approaches: Push and pull. Let’s checkout how each of these can work with Box and the pro and cons for each.
Pull approaches
Imagine a scenario where you need to notify another system when a file arrives in a specific folder in Box. This could be an important customer file that you need to act on straight away or perhaps as part of an integration to a CRM or case management system. A pull approach could be to poll the folder at a set frequency and check if any new documents had arrived since the last time you polled. This may seem relatively easy and straightforward and works well for small and well defined folder structures, but it doesn’t scale well…what if you had to poll thousands of folders or folders where access is restricted but you still need to react to certain updates. The time and API consumption would be prohibitive for this approach. Instead Box offers an event stream that allows you to listen for eg. file upload events and take action where appropriate as outlined in the below diagram
There are a couple of different event streams in the Box API. A user event stream will provide all events for a given user, where the enterprise event stream will provide all events from the whole Box enterprise.
User Events are used by tools like Box Drive to ensure that a user’s Drive is always up to date with what’s happened in Box
The enterprise event stream API offers a variety of configuration options. You can stream the logs in near real time or use historical lookups (the events for the last year are available from API) and limit by the type of event you want to poll.
Admin Events give you the same events as you’d get from running a historical report, and have some considerations with event timeliness vs ordering where Admin Streaming Events are great for near real-time lookups and reactionary use cases such as those tied to content security.
For example if you want to poll for all file upload events you can use this API and it will return a JSON array of events that you can process. Of course our Box SDKs also have functions for handling the events from these streams.
$ curl -i -X GET "https://api.box.com/2.0/events?" \
"stream_type=admin_logs" \
"&event_type=UPLOAD" \
-H "Authorization: Bearer ACCESS_TOKEN"Considerations for events
- As Box logs all events in your enterprise you can end up with a lot of events to sift through so limiting the event types to those that are required for the use case is important.
- For custom applications, the polling interval will impact the number of API calls you make so be aware of this. Eg. if you make five calls to the event stream every five minutes, you would use over 10 mill API calls annually so there is a cost factor to be considered.
Push approaches
Box also supports a push approach which means Box sends a notification when the event happens rather than you having to pull the event directly. The most common pattern for the push approach is the webhook pattern which is what we will focus on here. Webhooks are very much an industry standard in the SaaS world for driving event based architectures and for further technical deep dive on Box Webhooks I would recommend this excellent article.
Essentially a webhook is an event trigger that sits on Box platform. It can trigger based on a number of events and will send an event payload to pre-defined https endpoint that you control. The endpoint has to be publicly available as the event payload will come from the Box infrastructure so going this route does require that you can setup a public endpoint like a serverless function or similar.
For webhook security Box implements a signature verification system that allows you confirm the request is coming from Box. In terms of serverless architecture AWS Lambda, Azure Functions, GCP Cloud Functions or Vercel all offer a platform to configure and deploy the required functionality and from there interact with or notify other systems.
Some use cases require business logic to respond to the webhook by making further API calls to Box. For example to update metadata, you might need to move content or collaborate users. This is perfectly possible, but it is important to note that the webhook payload does NOT carry any authentication information such as an API token. To execute Box API calls based on the webhook event you would still need to authenticate using one of Box’s server side authentication mechanisms — JWT or Client Credentials Grant.
There is an exception to the above regarding the authentication as part of the payload. A ‘special’ type of Webhook that does carry authentication information called a ‘Box Skill’. This can only be configured to fire on uploads, moves, or copies of documents and is specifically designed to allow the retrieval of external data and applying this as metadata to the uploaded document without having to authenticate to Box separately. Most common use cases for this is external database lookup to provide metadata or by using AI to analyze and extract rich insights from the document. For more info see this excellent article where Alex Novotny shows how to add an AI generated summary to a document on upload.
The Box Sample Code catalog has some more good examples as well if you filter by Webhooks.
If you are interested in low code/no code solutions, cloud based integrators such as Zapier and Workato also make use of webhooks for a lot of their integrations with Box.
Consideration for webhooks
- If there is going to be heavy load on the webhook make sure the underlying infrastructure can manage it. Box will retry any webhook that doesn’t respond with a 2xx http return code within 10 seconds. For webhooks where heavy loads are expected a queue implementation would be worth considering.
- If the user who created the webhook is deleted from Box the webhook is also deleted so for production webhooks we recommend creating using a AutomationUser/Service Account to create the webhook (Service accounts tends to stay longer than developers)
Common use cases for both approaches
Below you can see a list of common use cases where developers can use events to drive business logic across different systems.
- Notification to external system when a customer or user has uploaded files in a shared folder, previewed files, signed document etc. I have worked with customers who need to update records in ERP or case management systems based off certain events in Box and a webhook approach is great for event based architectures like this.
- Kick off downstream process when a partner/vendor uploads for example a spreadsheet of data into a shared Box folder. The data might have to be consumed by a reporting or BI platform and this can all be automated using events or webhooks.
- Read metadata from external system and apply to Box folder or file on creation or after certain lifecycle events. For example an incoming invoice can be processed by an AI engine to extract important metadata, that is written to the file in Box and can be used to further drive downstream workflows.
I hope this article has given some ideas on how to react to Box events from a developer perspective and given you enough ideas and further information on how to progress. As always you can engage with our API and Box development experts in our brand new developer forum!
Happy coding.
Resources
Events guide
Webhooks guide
