Is this a _FLOAT or VAL() bug?

[Pete]

Code: QB64: [Select]

1. WIDTH 80, 43
2. _SCREENMOVE 0, 0
3. s$ = "1"
4. FOR i = 1 TO 20
5.     s$ = "0" + s$
6.     x$ = "." + s$
7.     x## = VAL(x$)
8.     PRINT " "; x$, LEN(x$) - 1
9.     PRINT x##
10. NEXT

  [ You are not allowed to view this attachment ]  

OK, two things. Why does it put a trailing zero at .000000000010? It should be tn zeros, and a one. Next, why does it skip 1D-20? Everything after that is one digit off.

Single gives correct results. Double gives the same incorrect results as _FLOAT.

Pete

[SMcNeill]

If I took a guess, I’d say it’s probably a rounding issue at 1D-20.

[jack]

the las two entries look wrong to me, here's what I get with FB using double

Code: QB64: [Select]

1. .............
2.  .0000000000000000001        19
3. 1e-19
4.  .00000000000000000001       20
5. 9.999999999999999e-21
6.  .000000000000000000001      21
7. 9.999999999999999e-22
8.  

here using 80-bit

Code: QB64: [Select]

1. .......................
2.  .0000000000000001           16
3. 1e-16
4.  .00000000000000001          17
5. 1e-17
6.  .000000000000000001         18
7. 1e-18
8.  .0000000000000000001        19
9. 1e-19
10.  .00000000000000000001       20
11. 1e-20
12.  .000000000000000000001      21
13. 1e-21
14.  

[Pete]

The FB results are more the rounding errors I would expect. Not being off by a full power of 10. What did to print for #11? That's the one I showed that had a trailing zero that shouldn't be present.

Pete

[jack]

for 11 I get (double)

Code: QB64: [Select]

1.  .00000000001  11
2. 9.999999999999999e-12
3.  

[Pete]

Thanks. I'm wondering what QB rendered for double. double and _FLOAT are the same. Single is accurate. Maybe QB had the off by power of 10 rounding error for d-20, too.

Pete

[jack]

I thought that this variation would give the right output, but no.

Code: QB64: [Select]

1. WIDTH 80, 43
2. _SCREENMOVE 0, 0
3. s$ = "1"
4. FOR i = 1 TO 20
5.     s$ = "0" + s$
6.     x$ = "." + s$ + "##"
7.     x## = VAL(x$)
8.     PRINT " "; x$, LEN(x$) - 3
9.     PRINT x##
10. NEXT
11.  

however, pcbasic, which is a gwbasic emulator gives

Code: QB64: [Select]

1.  .00000000001#               11
2.  .00000000001
3.  .000000000001#              12
4.  .000000000001
5.  .0000000000001#             13
6.  .0000000000001
7.  .00000000000001#            14
8.  .00000000000001
9.  .000000000000001#           15
10.  .000000000000001
11.  .0000000000000001#          16
12.  .0000000000000001
13.  .00000000000000001#         17
14. 1D-17
15.  .000000000000000001#        18
16. 1D-18
17.  .0000000000000000001#       19
18. 1D-19
19.  .00000000000000000001#      20
20. 1D-20
21.  .000000000000000000001#     21
22. 1D-21
23.  

[Qwerkey]

It does behave properly if you start with

s$ = "11" or s$ = "101"

How interesting.  Computers, eh?

[Qwerkey]

Also, try it with:

Code: QB64: [Select]

1. WIDTH 80, 43
2. _SCREENMOVE 0, 0
3. s$ = "1"
4. FOR i = 1 TO 20
5.     s$ = "0" + s$
6.     x$ = "." + s$
7.     x## = VAL(x$)
8.     PRINT " "; x$, LEN(x$) - 1
9.     PRINT x##, 1.1 * x##
10. NEXT

[jack]

the following code

Code: QB64: [Select]

1. $CONSOLE:ONLY
2. _DEST _CONSOLE
3. DIM ldx AS _FLOAT
4. ldx = 5## / 9##
5. PRINT ldx
6. PRINT USING "##.##################^^^"; ldx
7. PRINT USING "##.##################^^^^"; ldx
8.  

output

Code: QB64: [Select]

1.  .5555555555555556
2. 0.555555555555555556^^^
3. 0.555555555555555556F+00
4.  

[jack]

tested on Windows

Code: QB64: [Select]

1. $CONSOLE:ONLY
2. _DEST _CONSOLE
3. DIM ldx AS _FLOAT ', x AS _FLOAT
4. PRINT "variable arithmetic on _FLOAT have DOUBLE precision on QB64 64-bit, OK on QB64 32-bit"
5. FOR x## = 1## TO 8##
6.     ldx = x## / 9##
7.     PRINT ldx
8.     PRINT USING "##.##################^^^^"; ldx
9. NEXT
10. PRINT "you can assign a _FLOAT constant expression OK"
11. ldx = 8## / 9##
12. PRINT USING "##.##################^^^^"; ldx
13.  

QB64 32-bit

variable arithmetic on _FLOAT have DOUBLE precision on QB64 64-bit, OK on QB64 32-bit
 .1111111111111111
 0.111111111111111111F+00
 .2222222222222222
 0.222222222222222222F+00
 .3333333333333333
 0.333333333333333333F+00
 .4444444444444444
 0.444444444444444444F+00
 .5555555555555556
 0.555555555555555556F+00
 .6666666666666666
 0.666666666666666667F+00
 .7777777777777778
 0.777777777777777778F+00
 .8888888888888888
 0.888888888888888889F+00
you can assign a _FLOAT constant expression OK
 0.888888888888888889F+00

QB64 64-bit

variable arithmetic on _FLOAT have DOUBLE precision on QB64 64-bit, OK on QB64 32-bit
 .1111111111111111
 0.111111111111111105F+00
 .2222222222222222
 0.222222222222222210F+00
 .3333333333333333
 0.333333333333333315F+00
 .4444444444444444
 0.444444444444444420F+00
 .5555555555555556
 0.555555555555555580F+00
 .6666666666666666
 0.666666666666666630F+00
 .7777777777777778
 0.777777777777777790F+00
 .8888888888888888
 0.888888888888888840F+00
you can assign a _FLOAT constant expression OK
 0.888888888888888889F+00

[Jack002]

The top post here by Pete,
first, I see 1D-20 near the bottom where you see 1D-21. I looked at the recip, got what I expect, then I did a recip of the recip. I didn't always get the original value. Seems there is a bug.

Code: QB64: [Select]

1. WIDTH 80, 43
2. _SCREENMOVE 0, 0
3. s$ = "1"
4. FOR i = 1 TO 20
5.     s$ = "0" + s$
6.     x$ = "." + s$
7.     x## = VAL(x$)
8.     PRINT x$, LEN(x$) - 1
9.     PRINT x##, 1 / x##, 1 / (1 / x##)
10. NEXT
11.  
12.  
13.  

2nd observation, the log base 10 is NOT the exponent every time. It should be

Code: QB64: [Select]

1. WIDTH 80, 43
2. _SCREENMOVE 0, 0
3.  
4. CALL look(".00000000000000000001")
5. CALL look(".000000000000000000001")
6. CALL look(".0000000000000000000001")
7.  
8.  
9. SUB look (i$)
10.     x## = VAL(i$)
11.     PRINT i$, LEN(i$) - 1
12.     PRINT x##, LOG(x##) / LOG(10.0)
13. END SUB
14.