span8
span4
span8
span4
I really don't know if this is something that FME can do, but it has done a lot of things so far so just seeing if this might be possible!
I have a directory of 'D:\data' which has many folders containing data from map layers (D:\data\parks, D:\data\buildings, D:\data\rivers etc).
What I am trying to do is automate the deletion of the contents of these sub-folders (which include both files and more folders), except any folders within 'data' that start with the name 'raster' (for instance I don't want to delete D:\data\raster_aerial2016 and all of its contents).
Is this something that FME Desktop can do? I don't have experience in python / batch files and FME has solved many problems so just wondering if there are any transformers that can help me out?
Nice question. Of course - and I know I'm stating the obvious - you'll be sure to test your chosen technique (including mine) before letting it loose on your real data. I'd hate to be the cause of your entire filesystem going up in smoke!
My thought is that you could use the "Directory and File Pathnames" reader to read a list of files, use a Tester to drop all those that contain the word 'raster' and then use the "File/Copy" writer to move the remaining files to the /dev/null equivalent for your operating system.
Attached is a workspace that shows the technique. It searches for txt files in C:\FMEOutput and moves it to "nul" in another folder. Windows doesn't allow a file called "nul", so it is deleted.
Hope this helps.
I find the SystemCaller transformer referenced above by jeroenstiers to be great for file system operations: if you can do it in a CMD window, you can do it in FME!
Hi,
Yes, copying the required files to another folder like E:\Data to E:\Data1 with filecopy writer and delete the folder E:\Data in normal windows is the best option.
Pratap
Hi @cartochris,
I created a small example of workspace you can use for this functionality. It does make use of a small python-script that lists all folders in the parent DATA-folder. It loops through this list and to check the name of every folder. If the folder doesn't start with 'raster', a feature is created (with the url of the folder) and send to a SystemCaller that will use the path and the RMDIR "PATH_TO_FOLDER" \S \Q command to remove it and all files / folders inside it automatically.
You might want to remove the \Q in the statement above so you will have to confirm before removing a folder.
I don't think I have to specify how dangerous this code can be if the parent-folder is pointing to a drive... So take watch out with the use!
The PythonCode:
import fme import fmeobjects import glob class RemoveData(object): def __init__(self): self.parentUrl = 'D:/DATA/' def input(self,feature): # Don't do anything for every feature entering pass def close(self): # List all folders in the parentfolder (self.parentUrl) folders = glob.glob('{}{}'.format(self.parentUrl, '*') ) # Loop throught all returned folders for cFolder in folders: # Check if the name starts with 'raster' if cFolder.split('\\')[1][0:6] == 'raster': # Skip this folder continue # Create feature, add attribute, and send it to workspace newFeature = fmeobjects.FMEFeature() # Convert the slashes to windows-notation newFeature.setAttribute('url', '"{}"'.format(cFolder.replace('\\\\', '\\').replace('/', '\\'))) self.pyoutput(newFeature)
example-removingallsubfolders.fmw
Hopefully this helps you!
Jeroen
Hi @cartochris, as far as I know, there is no way to remove files unless you write a script. However, I think you can copy or move only files which are saved under the "D:\data\raster_*" folder to a backup folder in the disk system using the Directory and File Pathnames Reader and File Copy Writer. Once you created the backup, it's easy to remove remnant files manually.
Thanks for your reply @takashi. My raster folders are very large in size so wouldn't like to move them ideally, but this is no problem I think I might be able to do what I need using the robocopy command in Windows.
I don't think moving files in the same disk drive takes so long time, since it is equivalent to renaming. However, I didn't know that moving a file to 'null' works to delete it. It would be a good tip as @Mark2AtSafe suggested.
Anyway, be careful not to delete necessary files. I always create a backup for safe before cleaning up a folder, even if it could take a long time.
How to write a file for each distinct value? 3 Answers
Send messages to terminal when batch processing from command line 1 Answer
Batch deploy disabled 1 Answer
Batch deploy issue 3 Answers
© 2019 Safe Software Inc | Legal