- My Forums
- Tiger Rant
- LSU Recruiting
- SEC Rant
- Saints Talk
- Pelicans Talk
- More Sports Board
- Fantasy Sports
- Golf Board
- Soccer Board
- O-T Lounge
- Tech Board
- Home/Garden Board
- Outdoor Board
- Health/Fitness Board
- Movie/TV Board
- Book Board
- Music Board
- Political Talk
- Money Talk
- Fark Board
- Gaming Board
- Travel Board
- Food/Drink Board
- Ticket Exchange
- TD Help Board
Customize My Forums- View All Forums
- Show Left Links
- Topic Sort Options
- Trending Topics
- Recent Topics
- Active Topics
Started By
Message
re: True/False Excel Statement Issue
Posted on 11/13/16 at 10:28 am to LSUtigerME
Posted on 11/13/16 at 10:28 am to LSUtigerME
quote:
Round may work, but is also subject to errors if the rounded digit differs and rounds one way or the other.
TRUNC could work as well.
ROUND is normally the right choice when dealing with currency. TRUNC is not, because it exposes you to just what you said, the direction in which the rounded digit errs. For example, suppose you are computing x and y in different ways, and FP rounding errors in the least significant bit resulted in the following for some hypothetical format with 8 decimal digits of precision:
x = 1.2500001
y = 1.2499999
Rounding to two places gives you 1.25 for both, while truncating to two places gives you 1.25 and 1.24. Excel has actually got 15 decimal digits, so the errors are that much farther out.
quote:
Typically, when comparing values you would use a tolerance value and either ratio or subtract the values and compare against a tolerance.
Yeah, I talked about that in my first post as the more general approach to the problem.
Popular
Back to top
Follow TigerDroppings for LSU Football News