Started By
Message

re: True/False Excel Statement Issue

Posted on 11/13/16 at 8:26 am to
Posted by LSUtigerME
Walker, LA
Member since Oct 2012
3809 posts
Posted on 11/13/16 at 8:26 am to
I'm not sure what exactly he's trying to do though.

Round may work, but is also subject to errors if the rounded digit differs and rounds one way or the other.

TRUNC could work as well.

Typically, when comparing values you would use a tolerance value and either ratio or subtract the values and compare against a tolerance.
Posted by Spock's Eyebrow
Member since May 2012
12300 posts
Posted on 11/13/16 at 10:28 am to
quote:

Round may work, but is also subject to errors if the rounded digit differs and rounds one way or the other.

TRUNC could work as well.


ROUND is normally the right choice when dealing with currency. TRUNC is not, because it exposes you to just what you said, the direction in which the rounded digit errs. For example, suppose you are computing x and y in different ways, and FP rounding errors in the least significant bit resulted in the following for some hypothetical format with 8 decimal digits of precision:

x = 1.2500001
y = 1.2499999

Rounding to two places gives you 1.25 for both, while truncating to two places gives you 1.25 and 1.24. Excel has actually got 15 decimal digits, so the errors are that much farther out.

quote:

Typically, when comparing values you would use a tolerance value and either ratio or subtract the values and compare against a tolerance.


Yeah, I talked about that in my first post as the more general approach to the problem.
first pageprev pagePage 1 of 1Next pagelast page
refresh

Back to top
logoFollow TigerDroppings for LSU Football News
Follow us on Twitter, Facebook and Instagram to get the latest updates on LSU Football and Recruiting.

FacebookTwitterInstagram