Three ways to push data to ArcGIS Online.
Master your update workflows
Working with GIS means being constantly updating data. When your data is online and exposed to hundreds or thousands of users, the process to push new data is important. Today, we’re going to cover the options you have to push new data, manually or automatically. Be prepared for some good news to anyone using ArcGIS Enterprise 10.9 and above.
1. Overwrite a feature layer collection.
When you overwrite a feature layer collection, you replace the entire content of the service. That includes all the layers and the data they contain.
This is a fantastic tool because:
- It is supported consistently on ArcGIS Online and ArcGIS Enterprise. No matter which technology you use, this tool will be available to you.
- You don’t need to delete the old data prior to pushing your new data. ArcGIS will do that for you.
- All the layers in your feature layer collection are updated at once. This is ideal when you’re doing bulk operations.
There is one drawback that makes me favor other options when I am updating my data. It requires you to keep the database associated with the feature service on ArcGIS Online. The official help listing that limitation can be found here.
There is also a restriction in terms of how the data must have been published. In order to access that capability, the data must have been published from ArcGIS Online by uploading a file (CSV, File Geodatabase). If the data has been published from Pro, this option won’t be available.
This capability can also be used in automated processes. It is exposed by the Python API for ArcGIS.
2. Append to a feature layer collection
When you need to add data to an existing layer in a feature layer collection, then append your best option.
This tool is ideal when you need to push new data from a weekly generated data source for example. It is also better suited than overwrite when the amount of data to push is relatively small. Finally, it lets you update existing features in place.
This tool has many advantages:
- It lets you push only the data you need to add or update.
- It gives you control of your update cycle layer by layer.
- Since ArcGIS Enterprise 10.9, it is available both in ArcGIS Enterprise and ArcGIS Online.
This is usually my go-to when automating update processes. Since it has fewer constraints than the overwrite tool, I use it extensively. If you need to replace an entire layer, you can always truncate it before running the append.
The major disadvantage of that tool for me comes from an automation standpoint. The Append tool has been added to ArcGIS Enterprise recently. But the Python API still does not let you use that functionality. This is for version 2.0 and below: if you call the append method on ArcGIS Enterprise, then the API will block you. It will return an error message that says appending data is only possible with ArcGIS Online.
3. Use the editing endpoint of a layer.
This capability is an inheritance from the past. It is still needed so that web clients can communicate with feature services. However, when pushing new data, there are very few cases where you should use that.
The main use would be to update data on ArcGIS Enterprise prior to 10.9 automatically. There would be situations where the overwrite tool is not an option, such as when missing the source file or dealing with a big hosted feature layer)
The main benefit is that it gives fine grained-control to developers. For everyone else, it is a nightmare to use outside of a traditional web app. That’s about it. The Python API for ArcGIS exposes that capability.
It has a lot of downsides:
- It is slow. Compared to the other methods, this is by far less efficient.
- When working with really big polygons, such as the Canadian territory of Yukon, there might be problems. When the geometries become bigger than 10 MB, the edit endpoints tend to fail systematically.
- It forces you to make your feature layer editable.
The append tool has been a game-changer for me. Since it became available on ArcGIS Enterprise at version 10.9, I would like to be able to call it from the Python API regardless of the platform. This capability is the one I rely on the most in my workflows. What about you? If that’s not the one you want to use, you now have plenty of options.
