Author Topic: LNG Arithmetic Bug?  (Read 2614 times)

0 Members and 1 Guest are viewing this topic.

Offline XRaySpeX

  • Newbie
  • Posts: 8
LNG Arithmetic Bug?
« on: February 28, 2018, 06:37:40 pm »
Can anyone tell me why the enclosed simple, but large, DEFLNG arithemetic does not give the correct answer? Is it a bug or can I just not calculate?

It happens with other large values but I cannot sus out a pattern except that Scale is a power of 2 & from my experience Floating Point errors usually occur at powers of 2.

I have replaced IF (OCCUR >= YPOINTS * Scale) by IF (OCCUR / Scale >= YPOINTS) & it works correctly.

This was happening with QB64 v0.980, so I came back here today to get the latest version, but it's still happening.
« Last Edit: February 28, 2018, 06:48:59 pm by XRaySpeX »

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
Re: LNG Arithmetic Bug?
« Reply #1 on: February 28, 2018, 07:00:26 pm »
Looks like the multiplication exceeds limits of LONG type

see:
Code: QB64: [Select]
  1. DEFLNG A-Z
  2.  
  3. OCCUR = 1649329259: Scale = 16777216: YPOINTS = 350
  4. Compare1 = OCCUR - YPOINTS * Scale
  5. Compare2 = OCCUR >= YPOINTS * Scale
  6.  
  7. PRINT "OCCUR="; OCCUR; "Scale="; Scale; "YPOINTS="; YPOINTS
  8. PRINT "Why is OCCUR - YPOINTS * Scale ="; Compare1; " & not "; -4222696341; "?"
  9. PRINT "Why is OCCUR >= YPOINTS * Scale = "; Compare2; "(TRUE) & not"; FALSE; "(FALSE)"; "?"
  10. PRINT "In Numbers: "; 1649329259 - 16777216 * 350
  11. PRINT "By replacing IF (OCCUR>=YPOINTS*Scale) by IF (OCCUR/Scale>=YPOINTS), it works OK"
  12.  
  13. FOR i = 1 TO YPOINTS STEP 10
  14.     PRINT i * Scale
  15. PRINT "press any to continue..."
  16.  
  17. OCCUR = 1649329259: Scale = 16777216: YPOINTS = 350
  18. FOR i = 1 TO YPOINTS STEP 10
  19.     PRINT i * Scale
  20.  
« Last Edit: February 28, 2018, 07:08:27 pm by bplus »

Offline XRaySpeX

  • Newbie
  • Posts: 8
Re: LNG Arithmetic Bug?
« Reply #2 on: February 28, 2018, 07:25:28 pm »
Duh! I should have seen that. It's so obvious.

Thanks, @bplus.

Offline XRaySpeX

  • Newbie
  • Posts: 8
Re: LNG Arithmetic Bug?
« Reply #3 on: March 05, 2018, 09:22:41 pm »
Thanks, @bplus, you've given me an idea.

By switching from LONG to INTEGER64 I can extend the scope of my primes program from 2.1 E9 to 9.2 E18 :).

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
Re: LNG Arithmetic Bug?
« Reply #4 on: March 06, 2018, 12:10:15 am »
Primes, yum!