Hello,
I have a problem with the calcul of a sum (expression evaluator, statistic calculator..).
The result is not correct. I make this calcul : 10000+10000+1+100+0.1+0.01 and the result is 20101.109999999997 .
I have this problem in fme 2018 and 2017 but in 2015 the result is correct .
Have you got any idea to solve my problem? Thanks a lot.
Best answer by david_r
View original
It's due to floating point precision, more info here: https://floating-point-gui.de/
You can use the AttributeRounder to, let's say 5 decimals, which will return 20101.11.
Why it works differently in FME 2015 I cannot really say. Perhaps someone from Safe has a theory...
+17
It's due to floating point precision, more info here: https://floating-point-gui.de/
You can use the AttributeRounder to, let's say 5 decimals, which will return 20101.11.
Why it works differently in FME 2015 I cannot really say. Perhaps someone from Safe has a theory...
Thank you very much @david_r for your answer. I didn't know the floating point precision. Now yes ! Thank you too for the proposition of used the attributerounder with 5 decimals (I had chosen 2 decimals).
Thank you too @takashi indeed the result seem good only in the table view.Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
