True Looper

Related products: FME Form

I'd like to see a Creator that can take in a list, SQL Statement, etc... That will grab the first item and FULLY run through the whole workspace and then go back to the beginning , grab next and do again. I have massive amounts of shape files on an S3 bucket that I need to grab one at a time and process. The server wouldn't need a huge harddrive as I can delete before getting next item. Seems like a simple thing to do, but has been very tough to do. I still can't do it effectively.

The classic way to do that is with a controller workspace and a worker workspace. The worker workspace processes the data and has as a parameter the file to process. The controller workspace contains a Directory and File Pathnames reader and a workspaceRunner. This triggers the worker space once for every feature read in by the pathnames reader.

 

 

Alternatively, you can use batch deploy.