- My Forums
- Tiger Rant
- LSU Recruiting
- SEC Rant
- Saints Talk
- Pelicans Talk
- More Sports Board
- Fantasy Sports
- Golf Board
- Soccer Board
- O-T Lounge
- Tech Board
- Home/Garden Board
- Outdoor Board
- Health/Fitness Board
- Movie/TV Board
- Book Board
- Music Board
- Political Talk
- Money Talk
- Fark Board
- Gaming Board
- Travel Board
- Food/Drink Board
- Ticket Exchange
- TD Help Board
Customize My Forums- View All Forums
- Show Left Links
- Topic Sort Options
- Trending Topics
- Recent Topics
- Active Topics
Started By
Message
re: True/False Excel Statement Issue
Posted on 11/11/16 at 5:36 pm to TexasTiger90
Posted on 11/11/16 at 5:36 pm to TexasTiger90
quote:quote:
Excel uses floating point which is inexact. For example, 0.1 has no exact representation in binary notation, and it introduces errors into calculations.
This makes total sense now that I think about it. When my budgets are done for a project, I try to get every subtotal of Divisions 1-16 to be a whole dollar, but even when all of the subtotals read "$X.00", the bottom grand total will read $1,500,000.01. It annoys the shite out of me
That explanation is inaccurate. Machine and round off error can and do occur but it will not be at the 100ths level. The things you're seeing at x.00 and getting a x.01 in the sum is a result of showing 2 decimal places in the individual cells but you're formula is giving you numbers in the 3rd decimal places that add up to give you the .01.
You can fix this by adding the Round() function in you formula. Assuming you want two decimal points, the new formula should be should be Round(original formula, 2). That should give you exactly what you see when those cells are summed.
Posted on 11/11/16 at 9:14 pm to mdomingue
quote:
That explanation is inaccurate. Machine and round off error can and do occur but it will not be at the 100ths level.
I believe my explanation was correct WRT to the message I replied to.
However, it's not really correct to talk about "the 100ths level". As I said, Excel's FP numbers have 15 decimal digits of precision, and the errors I was talking about are limited typically to the least significant digit or two. It might be hundredths, millionths, or whatever, depending on where the decimal point is in a given number. These small errors can and do cause comparisons to fail for numbers computed in different ways.
quote:quote:
This makes total sense now that I think about it. When my budgets are done for a project, I try to get every subtotal of Divisions 1-16 to be a whole dollar, but even when all of the subtotals read "$X.00", the bottom grand total will read $1,500,000.01. It annoys the shite out of me
The things you're seeing at x.00 and getting a x.01 in the sum is a result of showing 2 decimal places in the individual cells but you're formula is giving you numbers in the 3rd decimal places that add up to give you the .01.
That seems to be a different issue than I was responding to, and I agree that cumulative small errors aren't likely to make 1,500,000 equal to 1,500,000.01. That's due to errors in the 10th significant digit, not the 15th or so.
Popular
Back to top
Follow TigerDroppings for LSU Football News