• EN
Question

Batch converting Vector Product Format libraries

  • 25 September 2014
  • 7 replies
  • 4 views
M
Badge
I am quite new to FME so forgive me if the solution to my problem is obvious.

 

I have 29 libraries of DNC data in Vector Product Format that I need to convert. It seems to me that this shouldn't be too complicated, but the problem is that each library has different feature types, for example "gen18a\\grt" is specific to library 18.

 

My initial tries have been with a powershell script (something that I'm also new at) that looks like this: 

 

 

Start-Process 'fme.exe'  -ArgumentList '"C:\\Users\\mbu\\Documents\\My FME Workspaces\\DNC_conversion.fmw"',

 

'--SourceDataset_VPF_DB C:\\Data\\DNC\\DNCDX012_025_Japan_North_Pacific\\DNC12\\DHT',

 

'--FEATURE_TYPES "<All>"',

 

'--FANOUT_DIRECTORY C:\\Data\\DNC_converted\\M6M\\DNC12_test'

 

 

Obviously the FEATURE_TYPES line does not work, but it illustrates what I'm trying to do. The full list of feature types needs to be read and passed to the writer for each library.

 

As I said, I'm an FME beginner, so if there's a better, completely different way of doing this I'm open to suggestions.

 

 

Thanks!

 

 

 

7 replies

takashi
Rank
Userlevel 3
Badge +17
Hi,

 

 

How did you set the reader feature type properties?

 

Perhaps the solution is similar to the answer to this question.

 

Generic (Any Format) Reader and Schema (https://safecommunity.force.com/CommunityAnswers?id=906a0000000d3qZAAQ)

 

 

Takashi

 

 

M
Badge
Thanks for the reply!

 

The reader feature type properties is set identical to the screenshot in your other answer. Unfortunately I can't see how to apply it to my own problem. For my (perhaps misguided) solution to work I need to pass a list of feature types to the writer in command line. If this seems like a dumb way to do thing, please say so :) Trying to learn both FME and powershell at the same time has a somewhat steep curve!
takashi
Rank
Userlevel 3
Badge +17
hmm.. I think all the souce datasets will be read by the reader feature type if you run the workspace with FME Workbench.

 

 

Could you please explain what the "full list of feature types" is, and why you need the list? Maybe a hint to solve the problem is there.
M
Badge
Well, my powershell example only converts one DNC library (DNC12), and it only works if I manually insert all the feature types in that particular library into FEATURE_TYPES. The list of feature types for each library is very long AND library specific.

 

What I want to do is "for-each" through all the 29 DNC libraries and convert each one using all the feature types. This is where the need to get the full list if feature types comes in.

 

I have seen that people with a similiar problem are suggested to use Schema reader, but I don't quite understand how it would be different than using the VPF format reader that I'm using now.

 

 

I've been under the impression that something like this needs to be done using command-line, but if it can be done completely in the Workbench, then I'm all ears for that as well!

 

 

takashi
Rank
Userlevel 3
Badge +17
Since you checked "Merge Feature Type" of the reader feature type, it will read every feature from the specified dataset, regardless of feature types.

 

If the workspace works fine as expected for converting single dataset when you run it with Workbench, I don't think you need to specify feature types explicitly in the script.

 

In other words, the "--FEATURE_TYPES" line could be removed.

 

Did you try it?
M
Badge
Yes, I've tried removing that line and it seems logical that it should work. Indeed, from the log file it looks like everything is read as it should, with the message "Reader `VPF_DB_1' of type `VPF_DB' provided schema for types: <huge list of all feature types>"

 

The writer is then loaded, and everything suddenly stops. No errors, just end of log file. The only thing that is output to the fanout directory is an empty folder called "0"
takashi
Rank
Userlevel 3
Badge +17
Such a situation may occur if the attribute which was set to the "Attribute to Fanout on" parameter does not exist in the actual data. Every feature that will be written by the writer must have the "Attribute to Fanout on" attribute when using Dataset Fanout option.

Reply


Community Stats

28,802
Posts
109,791
Replies
36,632
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.