QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: Petr on July 01, 2018, 03:08:19 pm

Title: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 01, 2018, 03:08:19 pm

Code: QB64: [Select]

1. REDIM P(0, 0) AS INTEGER
2. a& = _NEWIMAGE(800, 600, 256)
3. SCREEN a&
4. text$ = "Qb64"
5. TextToArray P(), text$
6.  
7.  
8. MaxX = UBOUND(p, 1): MaxY = UBOUND(p, 2)
9. MinX = LBOUND(p, 1): MinY = LBOUND(p, 2)
10. PRINT
11. PRINT "new array values:"; MaxX, MaxY, MinX, MinY 'array IS resized, but contains nonsense. the expected output are two same inscriptions
12.  
13. FOR y = MinY TO MaxY
14.     PRINT
15.     FOR x = MinX TO MaxX
16.         IF P(x, y) THEN COLOR 15 ELSE COLOR 8
17.         PRINT P(x, y);
18. NEXT x, y
19.  
20.  
21.  
22. SUB TextToArray (array( x , y) AS INTEGER, text AS STRING)
23.     v& = _NEWIMAGE(LEN(text$) * 8, 16, 256)
24.     _DEST v&
25.     _PRINTSTRING (0, 0), text$, v&
26.     _DEST 0
27.     _SOURCE v&
28.     FOR y = 0 TO 15
29.         PRINT
30.         FOR x = 0 TO 8 * LEN(text$) - 1
31.             REDIM _PRESERVE array(x, y) AS INTEGER
32.             IF POINT(x, y) THEN array(x, y) = 1
33.             '  _DEST 0
34.             IF array(x, y) THEN COLOR 15 ELSE COLOR 8
35.             PRINT array(x, y);
36.     NEXT x, y
37.     _DEST 0: _SOURCE 0
38. END SUB
39.  
40.  

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: FellippeHeitor on July 01, 2018, 03:15:26 pm

It's a known bug, Petr. No solution yet.

https://github.com/Galleondragon/qb64/issues/27

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 01, 2018, 03:16:59 pm

I see. And will it work if I type it using TYPE?

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 01, 2018, 03:26:54 pm

So solved, with type it works:

Code: QB64: [Select]

1. TYPE P
2.     X AS INTEGER
3.     Y AS INTEGER
4.     V AS INTEGER
5. END TYPE
6. REDIM P(0) AS P
7. a& = _NEWIMAGE(800, 600, 256)
8. SCREEN a&
9. text$ = "Qb64"
10. TextToArray P(), text$
11.  
12.  
13. 'MaxX = UBOUND(p, 1): MaxY = UBOUND(p, 2)
14. 'MinX = LBOUND(p, 1): MinY = LBOUND(p, 2)
15. 'PRINT
16. 'PRINT "new array values:"; MaxX, MaxY, MinX, MinY 'array IS resized, but contains nonsense. the expected output are two same inscriptions
17.  
18. i = 0
19. FOR y = 0 TO P(0).Y - 1
20.     PRINT
21.     FOR x = 0 TO P(0).X - 1
22.         IF P(i).V THEN COLOR 15 ELSE COLOR 8
23.         PRINT P(i).V;
24.         i = i + 1
25. NEXT x, y
26.  
27.  
28.  
29. SUB TextToArray (array() AS P, text AS STRING)
30.     v& = _NEWIMAGE(LEN(text$) * 8, 16, 256)
31.     _DEST v&
32.     _PRINTSTRING (0, 0), text$, v&
33.     _DEST 0
34.     _SOURCE v&
35.     FOR y = 0 TO 15
36.         PRINT
37.         FOR x = 0 TO 8 * LEN(text$) - 1
38.             REDIM _PRESERVE array(i) AS P
39.             IF POINT(x, y) THEN array(i).V = 1
40.             '  _DEST 0
41.             IF array(i).V THEN COLOR 15 ELSE COLOR 8
42.             PRINT array(i).V;
43.             i = i + 1
44.     NEXT x, y
45.     array(0).X = x
46.     array(0).Y = y
47.     _DEST 0: _SOURCE 0
48. END SUB
49.  
50.  

Thank for help, Fellippe.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: FellippeHeitor on July 01, 2018, 03:30:03 pm

My pleasure.

BTW, if you want a map of all builtin characters, I took them from QB64's internals in this sample, so you don't need to use temp images or POINT:

https://github.com/FellippeHeitor/Snippets/blob/master/largeFont16.bas

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 01, 2018, 03:37:04 pm

Thank you, I'll look at it right away.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 01, 2018, 03:45:16 pm

Yeah, that's exactly what I need. And it will be even faster. Great. Thanks!

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: SMcNeill on July 01, 2018, 08:47:41 pm

What's going on here is nothing more than what you'd normally expect to encounter, with this code.  It's not a bug; it's just the way memory basically works.

Let's see if I can explain what's going on under the hood for you...

Let's take an array of 2x4 dimensions. 

DIM a(1 TO 2, 1 TO 4)

Now, let's fill that array with data from 1 to 8...

FOR x = 1 TO 2
    FOR y = 1 to 4
        c = c + 1
        a(x,y) = c
NEXT y, x

Now, since memory is stored in a single block of uninterrupted data, the way this would look in memory is "12345678"

If we were to graph it to X/Y coordinates, the way it'd look would be like this:
15
26
37
48

X is across, Y is down, and it's stored in memory in a TOP to BOTTOM, LEFT TO RIGHT (Y,X) format.

Now, if we add another column of data, (REDIM _PRESERVE a(1 TO 3, 1 TO 4)),  what we'd see would be something similar to this in memory "123456780000"

Graphing this to X/Y coordinates, what we'd have would be something which looks like the following:
150
260
370
480

Notice that in this case X1, Y1 is STILL 1.   X1, y4 is STILL 4.   Our data remains the same and unchanging, with new data placed at the end of it.  _PRESERVE has worked perfectly as anybody would imagine it should, in this case.

Now, let's take that original 2x4 array and instead of adding an column, let's add a row:   REDIM _PRESERVE a(1 TO 2, 1 TO 5).   

As a memory block, we'd STILL preserve the old data "12345678" and we'd add space for the new data "00", which would sit in memory as "1234567800".

Now, the issue here is how that preserved memory would now map to our array.  Remember, we index(read) it UP TO DOWN, LEFT TO RIGHT, so what it'd look like would be:
16
27
38
40
50

X1, Y1 is still 1.  X4, Y4 is still 4....   The thing is, X1, Y5 is NOW 5..   X2, Y1 is no longer 5; it now points to 6...

It's at this point that people start to scratch their heads and say, "What the heck happened to my data??  It's all scrambled!

Except it's NOT.  It's still preserved as that same block of 1-8, with two new 0s added to the end of it.   The thing that we've changed is our INDEX system which we use to refer to that data.   

IF somebody needs to increase rows instead of columns, REDIM _PRESERVE *isn't* going to be the tool for the job.  All it basically does is keep the old array exactly as it was, and then add (or remove) space to the end of it.

And, "1234056780" is a much different dataset than "1234567800" is....

If your data starts as "12345678" in memory, ALL REDIM _PRESERVE will ever do is either add blank spaces to the end of that data (such as "12345678000000000000"), or else it'll remove a few spaces from memory (such as "1234"). 

It's NOT going to shuffle memory back and forth and insert gaps as necessary for you to keep the indexes the same -- and thank goodness it doesn't!!  Can you imagine the hit to performance EVERY program would suffer, if that was the default behavior for _PRESERVE??   It'd make the process so slow that it'd generally be worthless for many common needs.

If you need to REDIM multidimensional arrays, and keep the indexes exactly as they are, you'll have to write your own routine to do that.  Or -- if at all possible -- structure your data so you can add columns to the end of your data, instead of adding rows.   (Change Y, but leave X alone in an array dimmed in an (X,Y) format.)

*********************
*********************

And that, is basically the answer to your, "Can you please show me where the error is? I do not know why wrong values come."

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: SMcNeill on July 01, 2018, 09:05:41 pm

From the link (https://github.com/Galleondragon/qb64/issues/27) which tries to call it a bug report, let me use it as an example to show you what's going on here:

Code: QB64: [Select]

1. REDIM TEST(1 TO 2, 1 TO 3) AS INTEGER
2. TEST(1, 1) = 100
3. TEST(1, 2) = 200
4. TEST(1, 3) = 300
5. TEST(2, 1) = 400
6. TEST(2, 2) = 500
7. TEST(2, 3) = 600
8.  
9.  
10. DIM m AS _MEM
11. m = _MEM(TEST())
12. PRINT "ORIGINAL, in memory:"
13. FOR i = 0 TO 5
14.     PRINT _MEMGET(m, m.OFFSET + 2 * i, INTEGER),
15. NEXT
16. PRINT
17. PRINT "AS X/Y:"
18. FOR x = 1 TO 2
19.     FOR y = 1 TO 3
20.         PRINT TEST(x, y),
21.     NEXT
22.     PRINT
23. NEXT
24.  
25.  
26. PRINT
27. PRINT
28.  
29.  
30. REDIM _PRESERVE TEST(1 TO 3, 1 TO 3) AS INTEGER
31. m = _MEM(TEST())
32.  
33.  
34. PRINT "REDIMMED, in memory:"
35. FOR i = 0 TO 8
36.     PRINT _MEMGET(m, m.OFFSET + 2 * i, INTEGER),
37. NEXT
38. PRINT
39. PRINT "AS X/Y:"
40. FOR x = 1 TO 3
41.     FOR y = 1 TO 3
42.         PRINT TEST(x, y),
43.     NEXT
44.     PRINT
45. NEXT
46.  
47. END
48.  

Where the problem really presents itself is in the user's representation of data.

Listed as X/Y, we see a list of:
TEST(1, 1) = 100
TEST(1, 2) = 200
TEST(1, 3) = 300
TEST(2, 1) = 400
TEST(2, 2) = 500
TEST(2, 3) = 600

BUT, it's stored in memory in Y/X format, which would be better represented as the following list:
TEST(1, 1) = 100
TEST(2, 1) = 400
TEST(1, 2) = 200
TEST(2, 2) = 500
TEST(1, 3) = 300
TEST(2, 3) = 600

And, when we REDIM _PRESERVE, we simply add space to the end of that data in memory, or else we remove space.  In this case, we add space to get:  "100,400,200,500,300,600,0,0,0"

And that maps out with our new  indexes to become:
100,500,0
400,300,0
200,600,0


It's not actually a bug.  It's all working as intended, even if the user in this case expects it to do something much more complicated for them, instead.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 02, 2018, 10:25:40 am

Hi Steve. Thank you for exhausting answer. I do not understand why the _PRESERVE field is not memorized as well as the field over REDIM (x, y), but it is written differently. Well, I avoid to PRESERVE for the multidimensional field, I wanted to make the effort easier and not complicate. I finally solved it completely without PRESERVE.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: SMcNeill on July 02, 2018, 02:10:01 pm

Quote from: Petr on July 02, 2018, 10:25:40 am

I do not understand why the _PRESERVE field is not memorized as well as the field over REDIM (x, y), but it is written differently.

The reason WHY is for performance's sake. 

Imagine a DIM array(1 TO 2, 1 TO 4), with the data 1-8 stored in it, as I mentioned above.  In memory, it'd be stored as basically "1,2,3,4,5,6,7,8" (though without commas, and in Binary format).

Now, REDIM _PRESERVE array(1 TO 3, 1 TO 4)...

The way it is now, _PRESERVE preserves the data as it currently exists by basically following these steps:

1) Create a new memblock large enough to store the new array.  (3 * 4 * element size, which varies based on data type, with default SINGLE being 4 bytes, in this case.)
2) Copy the data from the old array memblock directly to the new array memblock.
3) Free the old array.
4) Make array() point to the new memblock instead of the old one.

0)So "12345678" is our original 2* 4 data.
1)We create a "000000000000" new chunk of data in memory to hold 12 elements.
2)We _PRESERVE the old data and copy it over to become "123456780000".
3)Free the old memory
4)and change pointers.

Simple, fairly fast, and straight forward.

To make _PRESERVE work across multiple dimensions, think of what it'd have to do:
1) Create a new array of proper size, as before. "000000000000"
2) Calculate how large each chunk of "preserved memory" must be, and move it to the proper place inside the new array....

This step means we move "12" first (one old dimension), so we have "120000000000" in the new array.

Then we calculate the proper position for the next position and move the next chunk of substainable data (the "34"), so we have "120340000000".

Continue to calculate and copy....
"120340560000"
"120340560780"

3)Free old array from memory.
4)Swap pointers.

As you can see, there's quite a bit more involved to preserve the data so it's "120340560780", instead of just "123456780000"...

Now, imagine the time this REDIM _PRESERVE process would take, if we REDIM _PRESERVE array(10000,10000,10000,10000).....

How many calculations would that be?  How many copies of chunks of memory into their new, proper positions?

I wouldn't even want to take a guess at how large a performance hit such a routine would cause for our programs!

I really don't see how any routine could calculate those "gaps" in the old data and position it "properly" in the new data, without the performance being so abysmal as to render it worthless, for all intents and purposes.

If written into the language itself, QB64 _PRESERVE would have to be flexible enough to check and move data properly for ALL cases.  It'd need to work for array(a,b,c,d,e,f,g,h,i,j), just as much as it would for a simple array(x,y), and the overhead would be astoundingly terrible.

Personally, I feel it's much better to leave it as it is, and let the user roll their own solution.  Then, they can search for a method which will give them the results they need (as you did above), without adversely destroying the performance they'd expect.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: TempodiBasic on July 09, 2018, 08:20:51 pm

Hi Petr
Hi Fellippe
Hi Steve

1.
yes I remember still this issue that I have talked with Steve in the other forum. www.[abandoned, outdated and now likely malicious qb64 dot net website - don’t go there]....
What do you think about the original website?

2.
I have understood the memory structure that brings to these results using _PRESERVE...but I remember also how  this new knowledge let me seeing _PRESERVE as a poor useful option....at first glance, but the issue of performance talked by Steve are very important.
The reality is that memory RAM is managed as a single line along that the data are put by ram manager.
However we must agree that at this time of development of QB64 _PRESERVE is a good option/feature for monodimensional arrays on which it works very well!
2.1 so if it is useful for his goals,  the coder can use a set of monodimensional array to play with _PRESERVE  feature of REDIM
2.2 so if it is useful for his goals,  the coder can use a monodimensionale UDT array like has done Petr to play with _PRESERVE feature of REDIM

3.
about

Quote

To make _PRESERVE work across multiple dimensions, think of what it'd have to do:
1) Create a new array of proper size, as before. "000000000000"
2) Calculate how large each chunk of "preserved memory" must be, and move it to the proper place inside the new array....

This step means we move "12" first (one old dimension), so we have "120000000000" in the new array.

Then we calculate the proper position for the next position and move the next chunk of substainable data (the "34"), so we have "120340000000".

Continue to calculate and copy....
"120340560000"
"120340560780"

3)Free old array from memory.
4)Swap pointers.

IMHO I think that this algorithm can become more efficient with some modifications using mathematical models...
but this my sentence lasts a sentence because I don't know how parser of QB64 is written/works, and how it manages REDIM statements.


4.
about

Quote

Personally, I feel it's much better to leave it as it is, and let the user roll their own solution.  Then, they can search for a method which will give them the results they need (as you did above), without adversely destroying the performance they'd expect.

Well if I agree here at the same time I have found a good reason to prefer FOR to _MEM in copying array issue.
To read more please go to the other thread  FOR vs _MEM

Thanks to read, to talk and to feedback.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: Petr on July 10, 2018, 12:36:29 pm

Hi TempoDiBasic, I've seen your arrays experiments using DIM and _MEM but I have not had time to try it out myself. I did a lot of different things. As soon as more time is available, I like to look at it closely.

Title: Re: Can you please show me where the error is? I do not know why wrong values come
Post by: TempodiBasic on July 20, 2018, 01:13:50 pm

Hi Petr
here other further experiments with array multidimensional and REDIM _PRESERVE
going forward understandig  RAM managment in array in QB64

please run these two examples...
the first experiment uses Steve's example to see how data are put in the ram

Code: QB64: [Select]

1. REDIM TEST(1 TO 2, 1 TO 3) AS INTEGER
2. TEST(1, 1) = 100
3. TEST(1, 2) = 200
4. TEST(1, 3) = 300
5. TEST(2, 1) = 400
6. TEST(2, 2) = 500
7. TEST(2, 3) = 600
8.  
9.  
10. DIM m AS _MEM
11. m = _MEM(TEST())
12. PRINT "ORIGINAL, in memory:"
13. FOR i = 0 TO 5
14.     PRINT _MEMGET(m, m.OFFSET + 2 * i, INTEGER); "-";
15. NEXT
16. PRINT
17. PRINT "AS X/Y"
18. FOR x = 1 TO 2
19.     FOR y = 1 TO 3
20.         PRINT TEST(x, y); "  -  ";
21.     NEXT
22.     PRINT "//";
23. NEXT
24. PRINT
25. PRINT "AS Y/X"
26. FOR y = 1 TO 3
27.     FOR x = 1 TO 2
28.         PRINT TEST(x, y); "  -  ";
29.     NEXT
30.     PRINT "//";
31. NEXT
32.  
33.  
34. PRINT
35. PRINT
36.  
37.  
38. REDIM _PRESERVE TEST(1 TO 3, 1 TO 3) AS INTEGER
39. m = _MEM(TEST())
40.  
41.  
42. PRINT "REDIMMED, in memory:"
43. FOR i = 0 TO 8
44.     PRINT _MEMGET(m, m.OFFSET + 2 * i, INTEGER); "_";
45. NEXT
46. PRINT
47. PRINT "AS X/Y"
48. FOR x = 1 TO 3
49.     FOR y = 1 TO 3
50.         PRINT TEST(x, y); "  -  ";
51.     NEXT
52.     PRINT "//";
53. NEXT
54. PRINT
55. PRINT "AS Y/X"
56. FOR y = 1 TO 3
57.     FOR x = 1 TO 3
58.         PRINT TEST(x, y); "  -  ";
59.     NEXT
60.     PRINT "//";
61. NEXT
62.  
63. PRINT " It seems that outer/newer index follows ram structure of the array..."
64. END
65.  
66.  

this experiment uses a 3 dimensional array of integer

Code: QB64: [Select]

1. REDIM TEST(1 TO 2, 1 TO 3, 1 TO 2) AS INTEGER
2. TEST(1, 1, 1) = 100
3. TEST(1, 1, 2) = 200
4. TEST(1, 2, 1) = 300
5. TEST(1, 2, 2) = 400
6. TEST(1, 3, 1) = 500
7. TEST(1, 3, 2) = 600
8. TEST(2, 1, 1) = 700
9. TEST(2, 1, 2) = 800
10. TEST(2, 2, 1) = 900
11. TEST(2, 2, 2) = 901
12. TEST(2, 3, 1) = 902
13. TEST(2, 3, 2) = 903
14.  
15.  
16. DIM m AS _MEM
17. m = _MEM(TEST())
18. PRINT "ORIGINAL, in memory:"
19. FOR i = 0 TO 11
20.     PRINT _MEMGET(m, m.OFFSET + 2 * i, INTEGER); "-";
21. NEXT
22. PRINT
23. PRINT "AS X/Y/Z"
24. FOR x = 1 TO 2
25.     FOR y = 1 TO 3
26.         FOR z = 1 TO 2
27.             PRINT TEST(x, y, z); "  -  ";
28.         NEXT
29.         PRINT "//";
30.     NEXT
31.     PRINT "^^";
32. NEXT
33. PRINT
34. PRINT "AS Z/Y/X"
35. FOR z = 1 TO 2
36.     FOR y = 1 TO 3
37.         FOR x = 1 TO 2
38.             PRINT TEST(x, y, z); "  -  ";
39.         NEXT
40.         PRINT "//";
41.     NEXT
42.     PRINT "^^";
43. NEXT
44.  
45.  
46. PRINT
47. PRINT
48.  
49.  
50. REDIM _PRESERVE TEST(1 TO 3, 1 TO 3, 1 TO 3) AS INTEGER
51. m = _MEM(TEST())
52.  
53.  
54. PRINT "REDIMMED, in memory:"
55. FOR i = 0 TO 26
56.     PRINT _MEMGET(m, m.OFFSET + 2 * i, INTEGER); "_";
57. NEXT
58. PRINT
59. PRINT "AS X/Y/Z"
60. FOR x = 1 TO 3
61.     FOR y = 1 TO 3
62.         FOR z = 1 TO 3
63.             PRINT TEST(x, y, z); "  -  ";
64.         NEXT
65.         PRINT "//";
66.     NEXT
67.     PRINT "^^";
68. NEXT
69. PRINT
70. PRINT "AS Z/Y/X"
71. FOR z = 1 TO 3
72.     FOR y = 1 TO 3
73.         FOR x = 1 TO 3
74.             PRINT TEST(x, y, z); "  -  ";
75.         NEXT
76.         PRINT "//";
77.     NEXT
78.     PRINT "^^";
79. NEXT
80.  
81. PRINT " ram manager seems to follow outer/new index of a multidimensional array..."
82. END
83.  
84.  

Thank's to read, to share and to feedback
QB64 is great ! And moreover QB64's community!

Text Only | Text with Attachments