_UNSIGNED _INTEGER64 bug?

[SMcNeill]

This is expected behavior, and it’s really simple to understand, if you look at binary numbers.

1 is 00000001 in binary.
255 is 11111111 in binary.

Let’s do the math and add those:

 00000001
+11111111
—————————
100000000

Now, if we’re storing the answer in an _UNSIGNED BYTE, we take 8 bits for it.  The last 8 bits are 00000000.

1 + 256 = 0 (If we’re using bytes to hold the answer.)

The process is just that simple.

[RhoSigma]

There's definetly something wrong with _UNSIGNED _INTEGER64, I already noticed it 2 years back here https://www.qb64.org/forum/index.php?topic=1084.msg102794#msg102794

I did also stumble over it when making the &B additions to the last two realease versions, but didn't change anything on it, as the problem seems to be scattered across several functions and not easy to spot entirely.

In some situations, and only with _INTEGER64, the _UNSIGNED condition seemes to be simply ignored and so it will only allow the signed range, loosing half of the possible unsigned values.

[SMcNeill]

There's definetly something wrong with _UNSIGNED _INTEGER64, I already noticed it 2 years back here https://www.qb64.org/forum/index.php?topic=1084.msg102794#msg102794

I did also stumble over it when making the &B additions to the last two realease versions, but didn't change anything on it, as the problem seems to be scattered across several functions and not easy to spot entirely.

In some situations, and only with _INTEGER64, the _UNSIGNED condition seemes to be simply ignored and so it will only allow the signed range, loosing half of the possible unsigned values.

There's also the issue of how things behave depending on which set of math registers are used -- the x64 ones or the x86 set.  You might always try swapping over to the x86 set and see if it gives you the results you're expecting on your 64-bit machine.

Code: QB64: [Select]

1. DECLARE CUSTOMTYPE LIBRARY ".\fpp"
2.     SUB set_dpfpu 'to toggle to double precision floating point math
3.     SUB set_qbfpu 'to toggle back to what most folks will see with QB64 64-bit default math
4. END DECLARE
5. DIM a AS _INTEGER64
6. DIM b AS _INTEGER64
7. DIM z AS _INTEGER64
8. z = 9223372036854775807 ' This is the biggest allowed number of this type.
9. '                         (Only here for reference.)
10.  
11. a = (2 ^ 55) - 1 '          This number should be odd.
12. b = 36028797018963967 '   The number above should equal this one.
13. PRINT a
14. PRINT b
15. PRINT "-------------------------"
16.  
17. set_dpfpu
18. a = (2 ^ 55) - 1 '          This number should be odd.
19. b = 36028797018963967 '   The number above should equal this one.
20. PRINT a
21. PRINT b
22. PRINT "-------------------------"
23.  
24. set_qbfpu 'Just to showcase that this is what we get in "normal" QB64-mode math, on a windows 64-bit machine.
25. a = (2 ^ 55) - 1 '          This number should be odd.
26. b = 36028797018963967 '   The number above should equal this one.
27. PRINT a
28. PRINT b
29.  

  [ You are not allowed to view this attachment ]  

As you can see from the image above, the answer we get is different solely based on which math units we're running it on.

And honestly, I don't understand this...  I can see why floating point math is affected the way it is, but INTEGER math?  This one was a total shock to me, when STx first showcased code which highlighted the issue.  Now, there might be some simple trick like making that a 2~&& ^ 55~&&, or such, but at the end of the day, just changing from normal precision to double precision can make a large difference, as shown above.

[luke]

And honestly, I don't understand this...  I can see why floating point math is affected the way it is, but INTEGER math?  This one was a total shock to me, when STx first showcased code which highlighted the issue.  Now, there might be some simple trick like making that a 2~&& ^ 55~&&, or such, but at the end of the day, just changing from normal precision to double precision can make a large difference, as shown above.

This should be fixed in the latest development build.

What's going on here? There's multiple factors at play. First that that pow() is inherently a 'floaty" function. It always returns a floating point number. In QB64 we use the version that returns a long double (113 bits of precision), which should be enough to hold even a 64 bit integer perfectly. Because pow() is floaty, it means the subtraction is actually a floating-point operation. The resulting floating-point number is then rounded to fit into a 64 bit integer variable.

The x87 FPU can be in several rounding modes. The two of interest are double mode (53 bits of precision) and extended mode (64 bits of precision). This effectively controls how many bits of precision are available in the result of any floating point operation. If the FPU were in double mode, 2 ^ 55 is beyond the range (note it needs 55 bits, we only have 53) in which you can represent each integer (with floating point the available numbers become less dense as you get bigger), so subtracting 1 has no effect on the value. In extended mode all is well because we have the extra bits.

QB64 used what appeared to be a Windows-specific function _controlfp to set the FPU mode. From a brief read of the documentation it appears the extended mode is not supported on x86_64, which doesn't make much sense but is consistent with this working in 32-bit QB64 but not 64-bit QB64. I have changed it to use the fldcw instruction directly.

Side note: Linux by default sets extended mode, which is why this error wasn't visible there. Apparently BSD uses double mode by default, so I guess this is a win for BSD users too?

[SMcNeill]

I have changed it to use the fldcw instruction directly.

This should fix a lot of the precision issues which people like jack have highlighted several times in his work.  In Windows, folks were seeing two different sets of results between the QB64-32bit version and the QB64-64bit version.  It’s nice to see that we’re going back to producing consistent programs again.  ;)

[jack]

This should fix a lot of the precision issues which people like jack have highlighted several times in his work.

I was hoping for that, but the problem with _Float is still there