A new way to list large numbers of users

August 2016 — this post originally featured on our previous blog at developers.intercom.io

Based on feedback from customers, we have released a new way to list users via the API. While the list user functionality is suitable for most use cases, it does not work well when you have a large number of users.

You can now use the scroll option to be able to more easily iterate over your users. Instead of having to page through your user list you can do the following:

Example Scroll Request

curl -u <APP_ID>:<API_KEY> -H 'Accept: application/json' 'https://api.intercom.io/users/scroll' | python -m json.tool > scroll.file
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   140    0   140    0     0    302      0 --:--:-- --:--:-- --:--:--   302
cat scroll.file | grep -w email |wc -l
100

Assuming you have more than 100 users, your initial request will contain a max of 100 users. By default the per page limit was 50 so now with the scroll API you get 100 returned each time. It will also contain a “scroll_param” that you can use to fetch the next page:

Get next scroll page

curl -u <APP_ID>:<API_KEY> -H 'Accept: application/json' 'https://api.intercom.io/users/scroll?scroll_param=244faea3-466d-421a-8d51-eb288b2525a7' | python -m json.tool > scroll_param.file
cat scroll_param.file | grep -w email | wc -l
82

You can see that now there are only 82 users returned, so it looks like I have reached the end of my user list. When this happens you should receive an empty user list on your next scroll query:

Last scroll page

curl -u <APP_ID>:<API_KEY> -H 'Accept: application/json' 'https://api.intercom.io/users/scroll?scroll_param=139e9131-4147-4f1e-a64e-f41edd71e6bc' | python -m json.tool > scroll_param.file

The empty user list shows that you have reached the end of your list.

Your scroll parameter will timeout

Each app can only have 1 scroll open at a time. You’ll get an error message if you try to have more than one open per app.

Plus, if the scroll isn’t used for 1 minute, it expires and calls with that scroll param will fail.

Although low-level curl calls are lovely …

We love curl as much as you do, but know you don’t necessarily want to be working with low-level curl calls every day. If you do, no problem — but if not, you can put this together in a handy script using our Ruby SDK.

As of yet we have not added any specific changes to the SDK for the scroll feature but you can still use it if, for example, you work through it in the following steps:

Ruby script

require 'intercom'
intercom = Intercom::Client.new(app_id: ENV['MY_APP_ID'], api_key: ENV['MY_API_KEY']);
#Uss an array to store users as we iterate through each set 
users = []
#Get the first scroll wihtout any params
scroll_list  = intercom.get("/users/scroll", "");
users += scroll_list['users'];
#If users array is empty then stop getting next scroll
until scroll_list['users'].empty? do
  scroll_list  = intercom.get("/users/scroll/", scroll_param:scroll_list['scroll_param']);
  #Build up the array and start again (if there are more users)
  users += scroll_list['users'];
end

Interested in building on the Intercom platform? Check out our Developer Hub to get started.