Sorter options for handling null, missing, empty string, non-numeric

Related products: FME Form

Picking up on questions such as
https://knowledge.safe.com/questions/4786/null-sorter.html

and comparing the FME Sorter to Excel, which sorts empty cells to the bottom (for a numeric sort)...

I'd like to suggest adding options to the Sorter or creating a "PedanticSorter" transformer with options allowing the user to sort certain types of values to the end, including nulls, empty strings, non-numeric etc. There could be a drop down box with "sort to end" and "sort to beginning" for each value type. For non-numeric this would apply when you choose to sort numerically but some of the values are not numeric.

A further option could be a "rejected" output port. Then the drop down list for each value type would include "Output via rejected port" as a third option.

"Sort to top" and "sort to bottom" would be better wording for the choices + "output via rejected port".

 

The default option for each value type would be whatever Sorter does now (usually sort to top), thus maintaining backwards compatibility if the transformer was upgraded old Workspaces.

 


So the next question is how do we sort these attribute types among themselves if two or more if them exist in the data? For example the attribute includes null AND empty string values.

I would say null comes before empty string if sorting to the top but what about when sorted to the bottom?

Is a hard coded rule acceptable or does it need to be user configurable?


Hopefully this hub transformer does what you need:


https://hub.safe.com/transformers/controllednullsorter


Thanks @Mark2AtSafe. I haven't had chance to try this yet but as soon as I do I'll feed back 🙂

Another option I have used as a work around uses a combination of NullAttributerMapper-->Sorter-->NullAttributerMapper.

 

Begin with the first NullValueMapper by mapping your NULL, Missing or Empty value to a value you know will be at the top or bottom of your sort (i.e. ZZZZ for a an alphabetic ascending sort). This will replace the value to be used for the Sorter. After your sort load a new NullAttributeMapper and set all of your ZZZZ back to NULL, Missing or Empty. The sorting will remain unchanged after this final mapping.

 

A bit clunky but it works!


Now called the AdvancedSorter I believe.


I actually like the way you think! What I did was simple. I used a tester to look for null, missing, or empty based on specific value that was causing problems. I then create an attribute called sort priority, if it is null I set priority to 1, if it is not null I set priority to 0. Then I apply that to the sorter before sorting that specific value. Thanks for pointing me in a solid direction!