Problem with divide

[MLambert]

Hi,

This routine seems to work wonderfully on windows 7 .. but windows 10 it fails every now and then

        GFCT# = VAL(GFCT$)
        GFOT# = VAL(GFOT$)

        IF GFCT# = 0 OR (GFCT# + GFOT#) = 0 THEN
            calc3# = 0.00

        ELSE

            calc# = (GFCT# / (GFCT# + GFOT#)) * 100
            Z = INSTR(1, STR$(calc#), ".") ' find the decimal point
            IF Z = 0 THEN ' no decimal point means all whole figures
                calc3# = calc#
            ELSE
                calc1# = VAL(MID$(STR$(calc# + 0.005), INSTR(STR$(calc# + 0.005), "."), 3)) ' extract and round up the decimals
                calc2# = VAL(MID$(STR$(calc#), 1, (Z - 1))) ' extract the whole number
                calc3# = calc2# + calc1# ' combine the whole number and decimal
                IF Z = 0 THEN calc3# = calc#
            END IF
        END IF

        GFCTP$ = STR$(calc3#) '

RESULTS
"GFCT$","000025","GFOT$","000003","GFCT#",25,"GFOT#",3,"calc#",89.2857142857143,"calc1#",.29,"calc2#",89,"calc3#",89.29000000000001,"GFCTP$"," 89.29000000000001"

Why do I get the 89.29000000000001 ??  .. on windows 7 it comes out as  89.29 .. along with 1000s of similar calculations. Just every now and then I get a huge number

Thanks,

Mike

[SMcNeill]

It's simply floating point math, doing its normal thing.

Here's a simple thought exercise for you: write out the binary values of 2.

2 ^ 0 = 1
2 ^ 1 = 2
2 ^ 2 = 4
2 ^ 3 = 8
And so on....

Now, if I tell you to write out the binary value of 10, you can do that quite easily:

1010 = one 8 + one 2 = 10

Now, using binary NEGATIVE powers, you can write out fractional values:

2 ^ -1 = 1/2
2 ^ -2 = 1/4
2 ^ -3 = 1/8

Now, give me the binary value of 1/10...

IF you can do that, then you can do perfect single precision math, without ever encountering any sort of rounding issues.


BUT...


Base 10 math can't return a terminated value for 1/3.  (It's 0.3333333333333333333333333...keep typing...333333333333333333....keep on typing....33333333333333333....type those 3333s forever and ever....333333333333.....we still don't have 1/10; keep typing....3333333333333.... Never going get there!!)

By the same token, you can't  perfectly calculate a terminated value for 1/10 with base 2 binary.

And, since computers work in binary.....

Floating point math will always generate various degrees of precision differences.  It's just base math.

[MLambert]

Thank you for the help ... but I think you may have missed my point. Why does it work perfectly for 1000s of calculations but not for the example ?

Thanks,

Mike

[MLambert]

I may not have made myself clear.

I receive the answer of 89.2857142857143 .. which rounded becomes 89.29 ... why am I getting the answer  89.29000000000001 ??

This only happens 17 times in 400,000 records.

Mike

[bplus]

Why? because junk creeps into floating point math.

[MLambert]

what in the hell are you talking about ??

[MLambert]

I am stunned with your response. I ask a question and I am told not to bitch about it ??

[EricE]

I wrote some test code.

Code: QB64: [Select]

1. DIM mem_x AS _MEM
2. DIM mem_y AS _MEM
3. DIM int64x AS _INTEGER64
4. DIM int64y AS _INTEGER64
5. x# = 89.29000000000001
6. y# = 89.29
7.  
8. IF x# = y# THEN
9.     PRINT "EQUAL"
10. ELSE
11.     PRINT "NOT EQUAL"
12. END IF
13.  
14. mem_x = _MEM(x#)
15. mem_y = _MEM(y#)
16.  
17. _MEMGET mem_x, mem_x.OFFSET, int64x
18. _MEMGET mem_y, mem_y.OFFSET, int64y
19.  
20. PRINT x#, HEX$(int64x)
21. PRINT y#, HEX$(int64y)
22.  
23. ' Always free memory blocks
24. _MEMFREE mem_x
25. _MEMFREE mem_y
26.  

QB64's eight byte DOUBLE variables cannot exactly represent 89.29 to two decimal places.

[bplus]

I am stunned with your response. I ask a question and I am told not to bitch about it ??

You are asking the same question over and over, well it worked in Windows 7 why doesn't it work with Windows 10?

Floating point math changes with software and hardware it can't be perfect with division problems.

How come you aren't asking how to fix the display of the number?

BTW did you see Luke's explanation of QB64 rounding using a "Bankers" method? If it's odd it's rounded up, even rounded down (or vice versa) so that rounding errors balance each other out when you add up large sets of numbers.

[bplus]

Oh I see you are trying to round the number yourself.

They changed the compiler for QB64 for v1.3  and for 64 bit from 32, unless you are still using older version of QB64 what worked before might work different now.
 
For 2 decimal round try: number2digit = int((number + .005) * 100 ) / 100

Code: QB64: [Select]

1. number = 89.285
2. number2decimal = INT((number + .005) * 100) / 100
3. PRINT number, number2decimal
4.  

Oh hey check this, two different answers (Windows 10-64 QB64 v1.4):

Code: QB64: [Select]

1. number = 89.285
2. number2decimal = INT((number + .00499) * 100) / 100
3. PRINT number, number2decimal
4. number2decimal = INT((number + .004999) * 100) / 100
5. PRINT number, number2decimal
6.  
7.  

[MLambert]

Thank you everyone for the help

Mike