Solved

read only the newest file from a directory

  • 2 March 2018
  • 6 replies
  • 40 views
vki
Rank
Badge +6
  • vki Contributor
  • Contributor
  • 53 replies

In a directory I have some textfiles and their names contain a date (xy... 2017-12-13.txt). Each month there's a new file. I want to read only the newest file, but I don't want to change the name in the reader each month. So my idea is to read them all (or at least one record to get the fme_basename), compare the dates and then read only the newest file.

So I tried various approaches (first extracting the date out of the filename, then use Aggregator, Sorter, VariableSetter...) but I don't manage to compare the dates of the features as I can't see a way to set the initial value...

Any ideas?

icon

Best answer by taojunabc 3 March 2018, 01:12

View original

6 replies

debbiatsafe
Rank
Userlevel 3
Badge +17

Hi @vki

You use a Directory and Path reader to read only .txt files from your directory. This reader outputs a feature for each file matching your criteria along with feature attributes such as path_filename (file name) and path_windows/paths_unix (file path), etc.

Assuming your file naming is consistent (name YYYY-MM-DD.txt), you can create a new date attribute by using the substring extractor function of the AttributeManager text editor to extract the date part of file name without the file extension (eg. 2017-12-13). Use the StringReplacer to replacer non-numeric characters (dashes in your case) and then use a Tester to verify the attribute values are numbers only. Attach a Sorter to sort by descending numeric values and use the Sampler to get the first feature which should contain the file path of the newest file. Use the path_windows attribute as the input for dataset parameter with a FeatureReader or WorkspaceRunner.

Hope this helps,

Debbi

T
Badge +6

If your text files have a fix format (xy... 2017-12-13.txt), You can use [Direcotry and File Pathnames] reader to read file's infomation(path_rootname:file name,path_windows:file path etc.) . Then creat a new attribute (@Right(@Value(path_rootname),10)) to get date part. then sort it by Alphabetic,Descending. Get top 1 record by sampler, You should get the newest file name(path_rootname).

getlastdatefile.fmw

vki
Rank
Badge +6

Thanks a lot to both of you. I didn't know the Directory and Path-Reader! Now it works like a charm!

Vera

fmelizard
Rank
Userlevel 4
Badge +13

Thanks a lot to both of you. I didn't know the Directory and Path-Reader! Now it works like a charm!

Vera

Glad it works. In case others come across this and do not have their files named as nicely as you did, it is possible to ask the PATH reader to give back a bunch of metadata on each file, including its last modified date `path_modified_date' -- just set the PATH reader to "Retrieve File Properties" in its parameters when you create it.

 

 

 

K

Rather than going to the trouble of parsing out the date from the filename, wouldn't it be easier to rely on the path_created_date, or path_modified_date from the Directory and Path reader? You will have to check the option to Retrieve file properties to 'Yes', but that would be an easier way to determine the latest file.

billybob
Rank
Badge +6

Does anyone know how to use this same or similar technique to find the latest file from a One Drive folder. I'm able to select a file from the One Drive folder, but I haven't figured out how to "search" through the list of files for the latest one. This might not be the correct approach, but if you know of one I'd greatly appreciate any insight.

Thanks,

Frank

Reply


Community Stats

28,753
Posts
109,619
Replies
36,587
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.