HubSpot API: Load Companies With a Lot of Custom Fields

To get iterate through all the companies in a HubSpot portal with all of the information (i.e. include all properties for each company), one has to list every property in get all companies API call.

Unfortunately, when a portal has a lot of properties (~400, depending on the property names), then such API call crashes with HTTP error code 414. This error code means that HTTP GET request has exceeded size limit (typically, between 2kB and 8kB, depending on the Web server).

At the moment, HubSpot API doesn’t have a good solution for this issue. Ideally API should allow one of the following:

1. Make this API call as a POST request. POST requests are not limited
   in size the same way as GET requests and hence developers will be able to list all the properties at once
2. Add a special request parameter to return full set of properties. I.e. remove the necessity to list all the properties
3. Follow the pattern similar to contacts API. For contacts there is an
   API endpoint to get batch of contacts by id. This will allow the following workaround: make a GET API call to get list of companies including only minimal set of properties (in fact, only id of every
   company is needed). Right after that, make a second API call to get a batch
   of companies using ids just returned by the first call. This way it
   will take only 2 API calls to get a batch of companies with full set of properties

Until any of the previous suggestions are implemented by HubSpot API team, one is left with the following suboptimal workaround:

Workaround 1

Make a GET API call to load batch of companies. Iterate over loaded companies and make a GET API call to load each company by Id.

This will use an additional API call for each company: for a portal with 100,000 companies — 100,000 additional API calls (counted against one’s API usage).

Workaround 2

1. Split all properties into groups of 200 properties.
2. Load all companies with first group 200 properties
3. Load all companies with second group of 200 properties and combine them with first list of companies by id.
4. Continue until all companies are loaded for each group of properties.

This workaround is easier on the API limits, but comes with an additional complexity of storing ‘partially’ loaded companies and combining them.