Solved

How to expose all attributes in generic way? Any "magic" parameter in AttributeExposer or Python Script?

  • 25 August 2016
  • 7 replies
  • 370 views

Badge

Hi

I need to expose all hidden attributes in generic way (different attributes are created from different dataset).

1. With AttributeExposer I can select specified attributes. How to expose them dynamically? Is there any "magic" way to do that? '*' or something?

2. How to determine by Python Script that attribute is hidden or not?

icon

Best answer by david_r 25 August 2016, 11:43

View original

7 replies

Userlevel 5

Hi, a quite similar question was answered recently, have a look here: https://knowledge.safe.com/questions/32483/dynamic-attributes-in-custom-transformer.html

Let us know if you think your case doesn't apply.

Badge

Hi, a quite similar question was answered recently, have a look here: https://knowledge.safe.com/questions/32483/dynamic-attributes-in-custom-transformer.html

Let us know if you think your case doesn't apply.

Workaround with AttributeExploder is sufficient.

 

 

BTW

 

When I use Dynamic(schema) Writer with "Schema from first feature" parameter, I usually hope to get objects with all attributes.

 

When attributes are hidden - Automatic(schema) Writer do not provide for ones, so I thought that need to Expose attributes before e.g. using Python API

 

 

Badge +9

Hi @david_r, I am looking to solve this exact same issue but cannot figure out what the suggested solution is. In my case I want to read some external data (Excel files) with a FeatureReader. The schema can be different between files so I'm using the Generic port. The basic structure of the Excel files will be the same: they will start with a column that contains an ID, but the header of the column might be different (hence why I cannot just statically specify that value). When I look at the data that is read, I DO see the attributes in Inspector, but without exposing them, I cannot do anything in the workspace. For example I manually exposed Asset ID# but not the highlighted ones, yet they show up in Inspector:

...but not in the workspace:

The suggested solution above mentions the AttributeExploder transformer, but how would that help? I already have the schema coming out of the FeatureReader but cannot figure out how to feed that dynamically into an attribute exposer (doesn't seem possible to read actual attributes dynamically and converts everything to a string). Any explanation/clarification/solution would be greatly appreciated!

Userlevel 5

Hi @david_r, I am looking to solve this exact same issue but cannot figure out what the suggested solution is. In my case I want to read some external data (Excel files) with a FeatureReader. The schema can be different between files so I'm using the Generic port. The basic structure of the Excel files will be the same: they will start with a column that contains an ID, but the header of the column might be different (hence why I cannot just statically specify that value). When I look at the data that is read, I DO see the attributes in Inspector, but without exposing them, I cannot do anything in the workspace. For example I manually exposed Asset ID# but not the highlighted ones, yet they show up in Inspector:

...but not in the workspace:

The suggested solution above mentions the AttributeExploder transformer, but how would that help? I already have the schema coming out of the FeatureReader but cannot figure out how to feed that dynamically into an attribute exposer (doesn't seem possible to read actual attributes dynamically and converts everything to a string). Any explanation/clarification/solution would be greatly appreciated!

It's important to distinguish between attributes that are known at design-time (what's visible and configurable in FME Workbench) and attributes that are known at run-time (while fme.exe is actually executing the workspace and processing your data).

 

Exposing an attribute only makes sense at design-time. When the workspace is executed (run-time) all attributes are equally available to all the transformers etc, whether they've been previously exposed or not.

 

There is no way to "expose dynamic attributes", in the sense that it contradicts itself:

 

  • An exposed attribute is known when designing the workspace, and therefore it's not dynamic.
  • A dynamic attribute is unknown when designing the workspace, and it's not possible to expose an attribute unless you know the name of said attribute.
I hope this makes sense.

 

 

Or to put it in another way: if your workspace doesn't know in advance which attributes it will receive, how can it decide what to do with those unknown attributes?

 

 

More information on dynamic workflows here:

 

https://knowledge.safe.com/articles/1050/dynamic-workflow-tutorial-introduction.html
Badge +9

Thanks for that explanation @david_r. Honestly I have struggled to understand Dynamic workflows (I haven't needed them really so that makes it even harder to fathom!) but now I'm starting to understand a little more. At least I know I cannot be chasing a solution that is not actually possible. So for my current workflow I ended up adding a user parameter that takes a string to define the header text by the user at run-time and it uses that parameter to expose the attribute. That's as "dynamic" as I can make it I suppose :)

Userlevel 5

Thanks for that explanation @david_r. Honestly I have struggled to understand Dynamic workflows (I haven't needed them really so that makes it even harder to fathom!) but now I'm starting to understand a little more. At least I know I cannot be chasing a solution that is not actually possible. So for my current workflow I ended up adding a user parameter that takes a string to define the header text by the user at run-time and it uses that parameter to expose the attribute. That's as "dynamic" as I can make it I suppose :)

I totally understand, dynamic workflows aren't immediately intuitive and it usually takes a couple of tries to understand what's going on.

 

Sounds like you found a pretty good workaround there.
Badge +9

Wanted to add a quick follow-up to my question above. After reading & understanding the following post by @BrianAtSafe, I was able to do the following to solve my problem, avoiding the above mentioned workaround (no need to use a user parameter to expose the attribute):

I used an AttributeKeeper to only keep the attributes in the list I needed (just the first one). We only need to merge this data with the first feature, so in that case use a sampler (1st N items, 1 item) to merge on that one only. When doing this, I found it necessary to use a counter before merging, then sort by the count attribute to ensure the first feature has the attribute list. But since I needed to run a tester to filter out unwanted features that resulted from an AttributeExploder (specific to my workflow), I just merged all features and avoided the Counter, Sampler and Sorter. Hope this helps somebody!

Reply