Math issue

[Borg]

Doing a simple subtraction is giving me a weird answer.
This is the line of code and the if condition is met.
IF ex = gx THEN ra = ABS(gy - ey): PRINT "1"; ra; gy; ey: GOTO 1150
The PRINT statement is only for debugging.
The variable values are gy 13.7926, ey 13.4951
doing the math manually ra = .2975
The answer I get from the program is ra = .2974997

What am I doing wrong here?

[Pete]

Well a computer doesn't "count" in base 10, it's a binary system and this leads to a lot of rounding errors and other problems with seemingly "simple" math functions.

You could try PRINT USING https://www.qb64.org/wiki/PRINT_USING

... or convert the variables to strings and then use VAL() to subtract them.

Code: QB64: [Select]

1. gy = 13.7926
2. ey = 13.4951
3. PRINT gy - ey
4. PRINT USING ".####"; gy - ey
5.  
6. gy$ = LTRIM$(STR$(gy))
7. ey$ = LTRIM$(STR$(ey))
8.  
9. PRINT VAL(gy$) - VAL(ey$)
10.  

Pete

[Bert22306]

My first thought was to dimension the three variables as DOUBLE or _FLOAT. That shows very clearly that the problem is simply round-off errors. Long string of nines, when re-dimensioned. PRINT USING would solve that problem.

[SMcNeill]

It’s as Pete indicitated: basic floating point math.

In decimal, what is 1/3? 

0.3333333333333333333333333333333333333..... 

We can’t truly represent 1/3 in decimal form as it’s a never ending fraction.  The best we can do is approximate the value.

For a computer, numbers are represented by binary values (power of 2), instead of decimal values (power of 10). Using powers of 2, it’s impossible to actually represent something as simple as 1/10...

2 ^ 4 = 16
2 ^ 3 = 8
2 ^ 2 = 4
2 ^ 1 = 2
2 ^ 0 = 1
2 ^ -1 = 1/2
2 ^ -2 = 1/4
2 ^ -3 = 1/8
2 ^ -4 = 1/16

We can’t represent 1/10th in binary, just as we can’t represent 1/3rd in decimal format.  It’s just a basic flaw of the math itself.


So how do we get such precision without errors??  In something like financial matters/programs, we track tenths and hundredths, but HOW do those programs work??

They track INTEGER values, not SINGLE.  Instead of $10.03 + $2.19, they add/subtract the value as 1003 cent + 219 cent.

Your only choices are:

1) Convert to integer values, so you’re not dealing with single precision math.

OR

2) Allow for variance from rounding errors.  Instead of a statement like IF a = b THEN...  use a statement like IF ABS(a - b) < 0.001 THEN....

By using “IF ABS(a - b) < 0.001 THEN”, the values don’t have to be EXACT; they only need to be within a specified threshold. 


You can also convert a value like .2974997 by rounding it to a lower level of precision, with a simple statement like: ra = INT(ra * 10000 + 0.5) / 100000    Then you can round the 7 digit value down to a 4 digit value, which may give you the results you’re looking for.


At the end of the day, all you can do is either:

1) Use integer values to avoid rounding errors.

OR

2) Write your program to account for the natural precision errors which WILL occur with single precision math.

[Borg]

Thanks for helping.
Maybe I should back up a bit. This bit of code is part of a program that converts HPGL or .plt text files into G-Code text files. The PRINT statement is just for debugging. The values are converted to strings and written to a text file.  I do something similar to the PRINT USING to arrive at the proper number of decimal places but it truncates and does not round. A CNC machine will not even try to execute a command that is off by .0001". 

If I write a separate program that does nothing but execute that bit of code, it gives me the correct answer. 
 
Again, thanks for trying to help me.

[Borg]

You have just given me an idea. I get the data in integer format and then convert to FPD by dividing by 10,000. I will try leaving the numbers as integers for the math, then converting to FPD.
Thanks for the idea.

[Borg]

Worked like a charm. You guys are geniuses.

[Pete]

Some of what Steve covered is what I needed to do for my office software back in 1990. I was so surprised when I started programming that even something this simple didn't work on a computer...

Code: QB64: [Select]

1. FOR i = 1 TO 10
2.     x = x + .1
3.     PRINT i; x
4. NEXT
5.  

Anyway, I did some work-a-rounds for two decimal numbers needed to work with currency. I just multiplied by 100 and then used string conversion to return the decimal. I recall some +.005 rounding was needed, too, but it did the trick for my ledger routine.

So this is something a bit different, but my fun is in trying out new stuff. Because it's new, anyone who can find any holes in it is welcome to point them out. Basically this routine changes decimal values to strings, adds them as integers, and then replaces the decimal in the sum. I included the raw computer addition, which returns a close but incorrect sum, for comparison.

Code: QB64: [Select]

1. a(1) = .00456
2. a(2) = 25.001
3. a(3) = 15
4. a(4) = 12.
5. a(5) = .3
6. a(6) = 100.0
7. a(7) = 92.023567
8. a(8) = 0.1000009
9.  
10. FOR i = 1 TO 8
11.     total = total + a(i)
12. NEXT
13. PRINT "This total is incorrect: "; total
14. PRINT
15.  
16. DIM sum AS _INTEGER64
17. FOR i = 1 TO 8
18.     x$ = LTRIM$(STR$(a(i)))
19.     IF INSTR(x$, ".") <> 0 THEN j = LEN(x$) - INSTR(x$, ".")
20.     IF j > place THEN place = j
21. NEXT
22.  
23. FOR i = 1 TO 8
24.     j = 0
25.     x$ = LTRIM$(STR$(a(i)))
26.     IF INSTR(x$, ".") <> 0 THEN
27.         j = INSTR(x$, ".")
28.         x$ = MID$(x$, 1, j - 1) + MID$(x$, j + 1)
29.     ELSE
30.         j = LEN(x$) + 1
31.     END IF
32.     x$ = x$ + STRING$(place - (LEN(x$) - (j - 1)), "0")
33.     LOCATE , 15 - LEN(x$): PRINT x$, a(i)
34.     sum = sum + VAL(x$)
35. NEXT
36. sum$ = LTRIM$(STR$(sum))
37. IF place > 0 THEN sum$ = MID$(sum$, 1, LEN(sum$) - place) + "." + MID$(sum$, LEN(sum$) - place)
38. PRINT: PRINT "This total is correct: "; sum$
39. END
40.  

Pete

[Borg]

Why is it that I am just now learning about the problems inherent in floating point math and qb64? I wrote many engineering programs back in the 90's using QB45 with  a lot of math and the problem never surfaced. Maybe it's because it wasn't being checked by a really persnickity CNC machine.  They have no mercy in that regard.  Some of the stuff I wrote involved hydraulics, simple beam loading, various steam calculations and more. I still have the source code but it wont run on qb64 unless I go through and change all of the LOCATE statements as well as all the printer output and probably more. In those days Windows still supported DOS based EXE's. I think that ended with XP.

It's fun getting back into it now with qb64.

If anyone is interested I can post up some of the source code on that stuff. Too busy right now.

Again, thanks to you all for helping out.

[_vince]

Why is it that I am just now learning about the problems inherent in floating point math and qb64?

This very topic gets brought up about several times a month for pretty much the entire history of QB-related internet forums

[SMcNeill]

The problem isn’t just in QB64.  The issue is what’s known as “periodical digits”.  Any (irreducible) fraction where the denominator has a prime factor that does not occur in the base requires an infinite number of digits that repeat periodically after a certain point, and this can already happen for very simple fractions. For example, in decimal 1/4, 3/5 and 8/20 are finite, because 2 and 5 are the prime factors of 10. But 1/3 is not finite, nor is 2/3 or 1/7 or 5/6, because 3 and 7 are not factors of 10. Fractions with a prime factor of 5 in the denominator can be finite in base 10, but not in base 2 - the biggest source of confusion for most novice users of floating-point numbers.