Recently I had the need to pull data from ArcGIS Online into a Mapbox GL JS web map. While it would be possible to use the WMS publishing feature in AGOL to display this layer, what would be ideal is to have access to the raw data. The data also needs updated on a regular basis, so manually downloading a copy from AGOL is not an option.
Another method to view this data would be to query the feature service directly from AGOL and return GeoJSON using something similar to the query below.
While this would work for some layers, a few exceed the default
maxRecordsCount in AGOL. To get around this limitation I followed the advice from this post. The basic method described is to batch download the features from each layer until all of the features have been extracted. I turned this method into a NodeJS tool that does just that. Give it an AGOL Feature Service url and it will find all the layers and extract all the features, saving them as GeoJSON. To find all the available layers, first the tool queries the service URL. The tool then creates an array of layers to download.
This tool agol-cache is available on npm and GitHub. It has not been thoroughly tested, as it does exactly what I need and nothing more. For example, while it would be possible to extract domains and values, this tool simply extracts the raw layer properties.
A sample query using
The tool works with ArcGIS Online Feature Services and has not been tested with Map Services or services published from ArcGIS Server.