Support bulk appends when the ArcGIS Python API becomes available

Related products: FME Form

Support bulk appends when the ArcGIS Python API is available, with and without truncate.


See the question Writing LARGE datasets to ArcGIS Online for backstory and other information


I think the Python API is a pure python module you can just install:

https://developers.arcgis.com/python/guide/install-and-set-up/

 


Support this method in both Portal and ArcGIS Online.


@carsonlam when developing this functionality, there needs to be either 1) A search performed prior to writing the fgdb to AGOL that searches for file geodatabase items with the file geodatabase name and if found deletes it, or 2) Use a unique name for the file geodatabase/zip (I appended the Job ID, which may not work for desktop users).

The biggest issue that I've come across with my script so far is that if the python script doesn't complete, the file geodatabase item could be orphaned, and won't upload the item since the item already exists. This should be tested in QA/QC!

Also another thing regarding this idea that I have found is that the traditional insert via applyEdits works faster in some instances (when writing a small number of records to AGOL), so that should still be the default and maybe this could work with a checkbox type thing? Or maybe even a new "bulk" writer.


For those following this idea, I was able to get it implemented (in a hack fashion). Check out the question for the fmw workspace.


Can this be implemented in the ArcGIS Online Connector?


Append is available via REST now so it can be a writer mode:

 

https://developers.arcgis.com/rest/services-reference/enterprise/append-feature-service-layer-.htm

 

Example usage:

 

https://community.esri.com/t5/arcgis-data-interoperability-blog/building-a-data-driven-organization-part-9-load/ba-p/1125830

 

Deletes aren't supported so they will have to be done via applyEdits but deletes are quite performant.