Text Only | Text with Attachments

QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: zaadstra on September 30, 2021, 10:13:42 am

Title: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on September 30, 2021, 10:13:42 am
Hi,

Writing a little epoch calculator, I ran into an issue which looked like rounding.  However, the suspected part was a muliplication of huge numbers.

Inspired by this topic https://www.qb64.org/forum/index.php?topic=4210.0 (https://www.qb64.org/forum/index.php?topic=4210.0) "Possible error in INTEGER64 calculations",  I changed a variable in the suspected calculation to _INTEGER64 and got it working.
When noticing that I still was in QB64-32 I switched to QB64-64 and recompiled. And there the bug was back. I got it woriking by DIMming all variables to _INTEGER64.

Code: QB64: [Select]
  2.  
  3. '  2019:02:05 17:50:30
  4. tm$ = "1978:09:30 15:00:00"
  5. 'tm$ = "1970:01:02 00:00:00"
  6.  
  7. 'DIM tm_sec AS _INTEGER64
  8. 'DIM tm_min AS _INTEGER64
  9. 'DIM tm_hour AS _INTEGER64
  10. 'DIM tm_day AS _INTEGER64
  11. 'DIM tm_month AS _INTEGER64
  12. DIM tm_year AS _INTEGER64
  13. 'DIM tm_yday AS _INTEGER64
  14.  
  15. tm_sec = VAL(MID$(tm$, 18, 2))
  16. tm_min = VAL(MID$(tm$, 15, 2))
  17. tm_hour = VAL(MID$(tm$, 12, 2))
  18. tm_day = VAL(MID$(tm$, 9, 2))
  19. tm_month = VAL(MID$(tm$, 6, 2))
  20. tm_year = VAL(MID$(tm$, 1, 4)) - 1900
  21.  
  22. tm_yday = INT(275 * tm_month / 9) - (INT((tm_month + 9) / 12) * (1 + INT((tm_year - 4 * INT(tm_year / 4) + 2) / 3))) + tm_day - 30
  23.  
  24. tm_epoch&& = tm_sec + tm_min * 60 + tm_hour * 3600 + (tm_yday - 1) * 86400 + (tm_year - 70) * 31536000 + INT((tm_year - 69) / 4) * 86400 - INT((tm_year - 1) / 100) * 86400 + INT((tm_year + 299) / 400) * 86400
  25.  
  26. PRINT tm$
  27. PRINT tm_yday ' check: https://www.esrl.noaa.gov/gmd/grad/neubrew/Calendar.jsp?view=DOY&year=2021&col=4
  28. PRINT tm_epoch&& ' check: https://www.epochconverter.com/
  29. 'PRINT
  30. 'PRINT (tm_year - 70) * 31536000&& ' this is the bastard
  31. 'PRINT INT((tm_year - 69) / 4) * 86400
  32. 'PRINT INT((tm_year - 1) / 100)
  33. 'PRINT INT((tm_year + 299) / 400) * 86400

Output QB64-32:
Code: Text: [Select]
  1. 1978:09:30 15:00:00
  2.  273
  3.  276015600

Output QB64-64:
Code: Text: [Select]
  1. 1978:09:30 15:00:00
  2.  273
  3.  276015616

Removing the REMarks on all DIM lines makes QB64-64 print the same correct result as QB64-32.
When all DIM's are REMmed and only the output var tm_epoch is _INTEGER64, both compilers calculate the result wrong by 16.
Both QB64 32 and 64 bit are version 1.5.

Is this some magic, or reversed restrictictions making the smaller bit compiler run more accurate?
What are the common rules and best practices here?
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: bplus on September 30, 2021, 10:16:00 am
How can you use _Integer64 with only 32 bit version?
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on September 30, 2021, 10:32:28 am
Quote from: bplus on September 30, 2021, 10:16:00 am
How can you use _Integer64 with only 32 bit version?
Good question, just ask the complier.  It did not complain!  Maybe it works less efficient but it just works.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on October 01, 2021, 04:24:01 pm
But .... any serious answer?
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: bplus on October 01, 2021, 04:37:48 pm
Do any of the sub calculations take the number of digits over say 13 digits (conservative, maybe 14 digits for some numbers) that's when rounding errors for an _Integer64 number start screwing up a calculation). Rounding is a place where 32 bit compiler may differ with 64 bit.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: Pete on October 01, 2021, 04:46:08 pm
My bet would be if you ran a 32-bit system alongside a 64-bit system, you'd start seeing rounding differences between the two. Maybe we need an _INTEGER32 keyword. Ah screw it. Let's look to the future and create an _INTEGER128 keyword, instead.

Pete

If you live in the past, we've already seen your future.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 01, 2021, 05:12:37 pm
What’s odd about a 32-bit system having 64-bit integers for math?  Doesn’t it also have 64-bit DOUBLE and 256-bit _FLOATs?

There’s no 64-bit OFFSETs in a 32-bit OS, but there’s no problem with data structures being larger than that.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: jack on October 01, 2021, 05:42:20 pm
don't variables default to single if not specified?
all your variables except tm$ and tm_year are whatever the default type is, not good practice in my opinion
btw, the correct result should be 276015600 so the 64-bit version as posted is wrong
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: jack on October 01, 2021, 06:01:28 pm
correction, epoch is defined as integer64
but having all the other variable as single is a bad idea, if you put a DefInt T at the top then the 64-bit version reports the correct value
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on October 02, 2021, 04:17:08 am
Indeed 276015600 is correct so the 64 bit is wrong.  As the code is the same I'm surprised.  If 64 bit as correct and 32 wasn't, you could have blamed the bits....
As I'm using the 64 bit version most of the times, I'm worried there is a little quirk in that version.

Usually I'm using single so no need for a Dim.  Only when very large numbers (or fractions) are expected.
I can't imagine where rounding occurs when only multiplying integer numbers.

It's also possible that calculations with different data type variables behave odd, but I've never seen rules about that.
It may be that the 64 bit version is a bit more strict in mixing variable types?
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 02, 2021, 05:11:08 am
There’s a lot of possible reasons why math can run into issues on various architecture.  Let me illustrate a very simple one:

Pretend you’re working on a system that uses 8-bit math registers.  Now, you want to find the center point between two numbers on a line.  (Point 100, Point 200, and Centerpoint is??)

A = 100
B = 200
C = (A + B) / 2

You run the above and it gives you an impossible answer like 23…

Now that’s obviously flipping wrong!!

So what happened??

Intermediary register overflow!!

You’re using 8-bit math registers, remember??  They store values from 0 to 255, and your answer is 150, so they should be fine.  Right?

WRONG!!

Your PC is processing things one step at a time, so it has to store the results in a temporary register to continue working on your multi-step math problem.

In this case, it does A + B first and stores the result…. 100 + 200 = 300…. INTERNAL OVERFLOW ERROR!!!  Before you can even get to divide by 2, you’ve already performed an overflow calculation!

Now, what’s the solution here??

Change your math formula: C = A + (B - A) / 2

B - A is 200 - 100,..  valid for 8-bit math registers.
100 / 2 = 50… valid
100 + 50 = 150… valid

YAYYY!!  We can calculate the answer without any overflow!!


Now, I *assume* you’re using the stable version of QB64.  V 1.5, I think it is?

In it:
QB64’s 32-bit version defaults to using x86 Floating point math registers.
QB64’s 64-bit version defaults to using 64-bit SSE math registers.

X86 FPU math is 80-bit precision.
64 bit SSE math is 64-bits precision.

And THAT explains the issue you’re seeing with the difference in the two systems.  Without changing your variable type, it defaults to SINGLE.

In 32-bit QB64, you have 80 bits for your math precision with floating point values.
In 64-bit QB64, you have 64 bits for your math precision with floating point values.

64-bit SSE registers won’t hold the value you’re calculating, so you get rounding issues.

Solution??

As you’ve found yourself — define your variables as INTEGER types, and the problem goes away.

And that’s basically the whole mystery behind the “glitch” explained.  It’s not a QB64 issue itself; it’s a basic difference in the amount of precision your PCs architecture provides.

(And the reason why I assume you’re using the stable V 1.5?  The Development build should now have both 32-bit and 64-bit versions defaulting to 80-bit FPU math.  It’s slower on 64-bit systems, basically doing your math in the CPU instead of the dedicated SSE math processors, but it *should* keep results consistent between 32-bit and 64-bit versions, and prevent issues like these.)
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: luke on October 02, 2021, 06:28:16 am
Or to distill your issue down to a few lines:
Code: QB64: [Select]
  1. a = 23554800
  2. b = 252460800
  3. m&& = a + b
  4. Print m&&
  5.  

Since a & b are both Single, the addition is done to Single precision. The correct answer is 276015600, but that number can't be represented as a Single: the nearest possible values are 276015584 and 276015616. The FPU rounds up*, and finally the value is cast to an Integer64 for storage in m&&. That all makes sense, and fully explains why we're getting the wrong result.

The question, then, is why does QB64-32 get the answer right?

* Technically it rounds to nearest, preferring the value with an even mantissa to break ties
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 02, 2021, 06:37:47 am
Quote from: luke on October 02, 2021, 06:28:16 am

The question, then, is why does QB64-32 get the answer right?

80-bit FPU math precision vs 64-bit SSE math precision.  Those extra 16-bits are enough to store the value without rounding.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on October 02, 2021, 07:26:46 am
Thanks for all the insights!

Indeed I'm using the v1.5 release.

Now let's see.... that big number in bytes (by google):
276 015 600 =
0b10000011100111010100111110000

That is 'only' 29 bits so easily should fit in all scenario's!

Looking back in the old code,  I suspected this part "(tm_year - 70) * 31536000" which gave as result (separated printed out):
 2.52288E+08
The scientific notation did raise my eyebrows so I started changing the prog in the first place.

Checking that out, 1978 * 31536000 = 62.378.208.000, which is
0b111010000110000001111110101100000000

Now we're talking of 36 bits.  Bigger than 32 bits but it still fits in 64 bit FP or 80 bit SSE!

Looking at the storage for a Single var, which is 4 bytes which is 32 bits.  Sure that doesn't fit.

Here's the part where I probably went wrong.  I assumed that the calculation registers are big enough for 'everything' and the var to be stored in limits you to good or overflowed storage.  The _Integer64 would be fine.
@SMcNeill 's  explanation of 80bit FP for x86 and 64bit for x64 calculations confirms this idea.  But the dark magic under the hood thinks different ;-)

So is the compiiler breaking up calculatings, and does it execute calcultations with Single var's in a smaller register?  Hence the rounding as @luke illustrates?  Just trying to understand, to be able to prevent these nasty coding errors.

Some tests:

Code: QB64: [Select]
  2. DIM tm_year AS _INTEGER64
  3. tm_year = 1978
  4. a = (tm_year - 70) * 31536000
  5. PRINT a ' prints scientific number
  6. PRINT (tm_year - 70) * 31536000 ' correct (only if DIM is present)
  7. a&& = (tm_year - 70) * 31536000
  8. PRINT a&& ' correct
  9. b&& = a
  10. PRINT b&& ' prints rounded a
  11.  

Output:

Code: Text: [Select]
  1.  6.017069E+10
  2.  60170688000
  3.  60170688000
  4.  60170686464

Note: Hopefully the new version gets a switch to force use of the SSE instructions, because "We have a need, a need for speed!" :-)
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 02, 2021, 08:06:36 am
A quick example of how to change floating point precision in QB64. This swaps between quick math (which uses hardware math processors) to extended precision math (which uses software processing). Note that the default qbfpu is quite a bit faster, and for most folks this should be more than sufficient for your needs, as it tracks precision down to about the 15th decimal point. IF, however, you absolutely have to have greater levels of precision, you can now swap over to extended precision and have about 20 decimal points worth of precision, at a significant reduction of speed.

And, if you need more than 20 decimal points of precision, you're just shit out of luck. Find a math library for that, or else write a string math handling routine -- your CPU isn't equipped to handle anything more than this, natively.


FPU_Precision.h
Code: [Select]
void set_dpfpu() { unsigned int mode = 0x37F; asm ("fldcw %0" : : "m" (*&mode));}
void set_qbfpu() { unsigned int mode = 0x27F; asm ("fldcw %0" : : "m" (*&mode));}


QB64 Code:

Code: QB64: [Select]
  1. ' FPU_Precision.h needs to be in QB64 folder
  4. DECLARE CUSTOMTYPE LIBRARY ".\FPU_Precision"
  5.     SUB set_dpfpu 'to toggle to double precision floating point math
  6.     SUB set_qbfpu 'to toggle back to what most folks will see with QB64 64-bit default math
  8.  
  10.  
  11.  
  12. 'Let's print our results without screwing with anything first.
  13. x = 5##
  14. y = x / 9##
  15. PRINT USING "QB64 division       #.####################"; y
  16.  
  17.  
  18. 'Set the double precision math
  19. set_dpfpu
  20. x = 5##
  21. y = x / 9##
  22. PRINT USING "QB64 division       #.####################"; y
  23.  
  24. 'Set the QB64 precision math
  25. set_qbfpu
  26. x = 5##
  27. y = x / 9##
  28. PRINT USING "QB64 division       #.####################"; y
  29.  

With the above header in your QB64 directory, you can use DECLARE LIBRARY to toggle between FPU and SSE processing as neded.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 02, 2021, 08:10:28 am
And if you’re curious WHY your value overflows, check out this post here: https://www.qb64.org/forum/index.php?topic=4057.msg133955#msg133955


Quote
Looking at the storage for a Single var, which is 4 bytes which is 32 bits.  Sure that doesn't fit.

Remember — with a floating point number, you’re not just storing the integer value itself, but also the sign, decimals, and exponent values…. Check the link above and see if it helps you break down the internal storage process a bit better.

(1 bit is for sign, 8 bits for exponent, 23 bits left for the rest for a SINGLE value.)

https://en.wikipedia.org/wiki/Single-precision_floating-point_format
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: luke on October 02, 2021, 10:00:58 am
Quote from: zaadstra on October 02, 2021, 07:26:46 am
Here's the part where I probably went wrong.  I assumed that the calculation registers are big enough for 'everything' and the var to be stored in limits you to good or overflowed storage.  The _Integer64 would be fine.
@SMcNeill 's  explanation of 80bit FP for x86 and 64bit for x64 calculations confirms this idea.  But the dark magic under the hood thinks different ;-)

So is the compiiler breaking up calculatings, and does it execute calcultations with Single var's in a smaller register?  Hence the rounding as @luke illustrates?  Just trying to understand, to be able to prevent these nasty coding errors.

I should emphasise that the results you're seeing with QB64-64 are the "correct" results, in that it's the expected value - so that won't be changing in the future unless someone can convince me otherwise. The intermediate mode of the FPU that Steve mentioned doesn't really come into play, since I'm running the calculations on a fixed development copy (technically a Linux version which never had the bug, but the point stands).

PRINT's decision to use scientific notation isn't really reflective of anything other than a formatting rule, so you can't read into it too much either way.

The rule for the arithmetic operations (+-*/) is basically that the value of lesser type is converted to the greater type, and the result is a value of the greater type. Here "lesser" and "greater" refer to an ordering of the types, which basically runs, from least to greatest: _byte, integer, long, _integer64, single, double, _float. One exception is that the / (division) operator always converts its arguments and returns a floating-point number, for obvious reasons. So in this small program:
Code: QB64: [Select]
  3. Print a - b
  4.  
The result of the subtraction is of Single type.

It also turns out that because of the nature of floating point numbers, there are some numbers that can't be represented. For a Single, you have 24 bits (technically 23 + 1 implicit bit) that work as a regular value, so you can store every integer up to 16777216 (2^24). But beyond that you can only store some integers:
Code: QB64: [Select]
  1. b = 16777216
  2. Print Using "########"; b - 1
  3. Print Using "########"; b
  4. Print Using "########"; b + 1
  5. Print Using "########"; b + 2
(Notice because b is a Single, the addition operation gives a Single result.) The number 16777217 simply can't be represented, so the result is rounded to a representable number. In this particular case we could fix the issue by making 1 a Double, which causes b to get converted to a Double and the result to be a Double:
Code: QB64: [Select]
  1.  = 16777216
  2. Print Using "########"; b + 1#

Now, it turns out that 31536000 can be stored in a Single... but the result of (tm_year - 70) * 31536000 can't. We can see this using the Print Using trick again:
Code: QB64: [Select]
  1. tm_year = 1978
  2. a = (tm_year - 70) * 31536000
  3. Print Using " ###########"; a
Note that if you just do a straight-up "1908 * 31536000" you'll get a nonsense answer because the constants are of type Long and the result is bigger than will fit in a Long; using a Single or _Integer64 variable causes the type of the intermediate expressions to be greater.

It's coming into midnight (or 1am if you count the impending daylight saving change) and I'm not quite sure what the message of the post is anymore. Anyway, if you want to do maths with big numbers and can't accept some rounding, best to make everything an _Integer64 and it should all work out.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on October 02, 2021, 10:02:41 am
Thanks for the examples, @SMcNeill.

To clarify my 'note' remark, this was about the new development version where you wrote the cpu FP would be the default, hence calculating slower. In that case I would rather select the (now default) SSE calcultations, when speed is needed.

Another question, why would you use the _FLOAT marks for a literal value 5## ?  How does this help?

P.S. the progam in your link threw me an Illegal function in line 72 (at toggling the first OR the last red bit).
Am I correct that the explained examples only show 31 bits (including sign bit?)  I would expect 32 bits.  Now at least I see that Single var has a lot less bits than 32 :-)
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on October 02, 2021, 10:28:28 am
Thanks @luke  for the late night answer.

I think that solves all mysteries, the order of variable types.

Looking at the QB64 data types table I always think that a (big) integer var is always bigger (greater) than a single.  But it is not (it is lesser).
Finding this was the message of this post if you ask me, I've learned a lot.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 03, 2021, 12:25:17 am
Quote from: zaadstra on October 02, 2021, 10:02:41 am
Am I correct that the explained examples only show 31 bits (including sign bit?)  I would expect 32 bits.  Now at least I see that Single var has a lot less bits than 32 :-)

A SINGLE stores values as:
1-bit for sign.
8-bits for exponent (+127)
23-bits for the mantissa.

So, let’s look at a value for a number such as …. Umm..  13, just for example.

13 in binary is: 1101, in standard format.

In binary, scientific notation, 13 is represented as 1.101E1  (I think… it’s midnight here and I’m doing the conversion in my head.  Let’s pretend that’s right, if it’s not.)

Now that we know the formalized format of the binary value of 13, we can represent it as a SINGLE.

The first bit is 0, as 1 represents the minus sign.
The next 8-bits are 11111111, as they represent the binary value of 128 (our exponent of 1, plus 127)
The last 23 bits are our mantissa — the digits right of the decimal point.  101 (and 20 more 0’s)

13 decimal = 1101 in binary = 1.101E1 in formalized format.

0 - 11111111 - 10100000000000000000000000 would represent the 32-bits for how it’s stored in memory as a SINGLE value.

1 bit Sign +8 bit Exponent (plus 127) + 23 bit Mantissa (what’s right of the decimal point) = 32 bit SINGLE representation.
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: zaadstra on October 03, 2021, 03:50:29 am
I gues I should have been more specific.  I counted the bits from your examples.

Code: Text: [Select]
  1. And, the value for 3 would be:
  2. 0 --- 10000000 --- 1000000000000000000000
  3.  
  4. 0100000001000000000000000000000  --> 31 bits
  5.  
  6.  
  7. 0 - 11111111 - 10100000000000000000000000    (example above)
  8.  
  9. 01111111110100000000000000000000000  --> 35 bits

So you see I really worked out the examples!
How it works now is very clear,  thanks for that.  It has been a great learning weekend! :-)
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 03, 2021, 06:02:20 am
When dealing with 20+ zeros in a row, it’s hard to count them all.  LOL! 

Ignore any extra bits beyond 32.  :P   You’ll learn in time, I’m good with the complicated stuff, but can’t count past 1, 3, 2, without messing up.

The concept is there, even if my 0s aren’t.  😂😂
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: Dimster on October 03, 2021, 08:08:47 am
Gawd there is so much that goes into this.. I'm not that strong in math. I do understand if I want the variable value to be 7 digits I want to declare that variable as a single, a double will give me 15 digits. So I guess my point is I'm surprised that it's just not that simple in the BASIC language. Why all that convolution and scientific notation when a layman/woman is just looking for a 7 digit answer. Now I appreciate where the math formula used to value the single variable is producing a number greater than 7 digits, there are rules of how to display the accuracy back to 7 digits using scientific notation but why not just truncate to the first 7 digits and call it a day. After all the programmer is just expecting a 7 digit value. Don't get me wrong, I sometimes do want more accuracy to a calculation but even if I'm trying to calculate pi I don't think I will ever be absolutely comfortable with the answer produced by BASIC math limitations. 
Title: Re: Calculation difference between QB64 64 and 32 bit versions
Post by: SMcNeill on October 03, 2021, 08:46:29 am
A whole lot of it, @Dimster, is depending on how deep one wants to go into the rabbit hole for computing.   Take 1 + 2 = 3…. That’s about as simple as it gets.

But how does the computer do it with switches that can only be ON (1) or OFF (0)?

It converts to binary format behind the scenes:

1 = 00000001
2 = 00000010
———————-
      00000011

Then it basically adds those ONs (1s) and OFFs (0s) to produce the 00000011, as above.

And. In binary, 00000011 is the value of 3, in decimal.

If all you need to know is what 1 + 2 is, you can stop digging there and just use your calculator app and get 3 as the answer.  It’s only if you’re wanting to know *exactly* what’s going on under the hood, that you need to break stuff down to binary addition and conversion.

Same way with working with decimal values.  If you just need to store to 7 decimal points, knowing how to DIM variable AS SINGLE is good enough for that. 

The vast majority of programmers never really HAVE to know *exactly* what format syntax is used to store those values internally, but I do wish more would take interest in learning at least enough about the subject to have a basic understanding of why 1/10 is a non-precise value.

All the discussion above is for those who want to delve more into internal data structures, than BASIC itself.  If it’s not something you need for your personal projects, nor something you’re particularly interested in, then I’d just skim it, keep it in mind as a “it’s a little more complex than I first imagined” thought, and then I’d move on and not worry about it.


Consider it like playing a video file.  I don’t need to know every detail about how an MP4 video is formatted, stored, synced, and captioned in multiple languages.  I just need to know enough to double click on them and let my PC play them!  I’ll leave it up to the guys who get paid to work on them, to sort out HOW the encoding and all works.  It’s just one of those many “rabbit holes” of computer knowledge that I’ve never had to delve very deeply into yet.  😉
Text Only | Text with Attachments