Text Only | Text with Attachments

QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: TerryRitchie on March 28, 2020, 05:00:32 pm

Title: REDIM _PRESERVE bug
Post by: TerryRitchie on March 28, 2020, 05:00:32 pm
Is there any time line on when this bug may get corrected? I really need to be able to preserve values across multi-dimensional arrays.

Code: QB64: [Select]
  1. 'REDIM _PRESERVE bug in multiple dimensioned arrays.
  2.  
  3.  
  4. REDIM Test(0, 0)
  5.  
  6.  
  7. FOR i = 0 TO 9
  8.     Test(0, i) = i
  9.     PRINT Test(0, i)
  10.     REDIM _PRESERVE Test(0, UBOUND(test, 2) + 1)
  12.  
  14. FOR i = 0 TO 9
  15.     PRINT Test(0, i)
  18. PRINT "That works. But when you REDIM the other index this happens."
  21. REDIM _PRESERVE Test(1, UBOUND(Test, 2))
  22. FOR i = 0 TO 9
  23.     PRINT Test(0, i), Test(1, i)
  26. PRINT "The values get spread across both indexes."
Title: Re: REDIM _PRESERVE bug
Post by: FellippeHeitor on March 28, 2020, 08:10:10 pm
That's an out-of-my-league issue. Luke assigned it to himself on the repo, but Luke has a life outside QB64 realm, so... no idea.
Title: Re: REDIM _PRESERVE bug
Post by: STxAxTIC on March 28, 2020, 08:16:31 pm
This is why I wish I started working on a really nice linked list library like a year ago. I have yet to develop this thought properly, but I suspect that linked lists can get us around this redim problem, and god willing, *all* problems surrounding arrays: mixing types and so on. I already use the technique on a case-by-case basis when I have a program that really demands a lot of its data - namely 3D levels and text editing to name my big two.

Anyway, see if linked lists work for you Terry. It's gonna have to be all hand-rolled but the power tradeoff is hands-down better when you need it.

EDIT: By the way your signature is hilarious!
Title: Re: REDIM _PRESERVE bug
Post by: TerryRitchie on March 28, 2020, 08:30:10 pm
Quote from: FellippeHeitor on March 28, 2020, 08:10:10 pm
That's an out-of-my-league issue. Luke assigned it to himself on the repo, but Luke has a life outside QB64 realm, so... no idea.

Ok, I do understand that life does take precedence and Luke has one too :-)  I'll start investigating the method STxAxTIC pointed out. Thank you for the reply Fellippe.

By the way Fellippe, I am absolutely astonished that you wrote something as complex as InForm without the need for dynamic multi-dimensional arrays!
Title: Re: REDIM _PRESERVE bug
Post by: TerryRitchie on March 28, 2020, 08:40:26 pm
Luke, if you happen to read this and get the issue resolved there's a month's worth of your favorite beverage on the way .. Starbucks?  A case of YooHoo? :-)))
Title: Re: REDIM _PRESERVE bug
Post by: SMcNeill on March 28, 2020, 09:14:41 pm
It's not really a bug, as it is a set limitation to how REDIM works.  I did a video explaining the whole process and how it all works behind the scenes here:


If one needs to REDIM and preserve beyond the first index, then they just need to write their own routine to handle it.

SUB CustomRedim (Original2DArray(), NewIndex1, NewIndex2)
    DIM Temp(NewIndex1, NewIndex2)
    FOR X = 0 to NewIndex1
        FOR Y = 0 TO NewIndex2
           If X <= UBOUND(Original2DArray, 2) and Y <= UBOUND(Original2DArray, 1) THEN Temp(X, Y) = Original2DArray(X, Y)
       NEXT
    NEXT
    REDIM Original2DArray(NewIndex1, NewIndex2)
    Original2DArray() = Temp()
END SUB
   
Title: Re: REDIM _PRESERVE bug
Post by: TerryRitchie on March 28, 2020, 09:21:44 pm
Bam, Steve to the rescue. I was just sitting here contemplating the best method for rolling my own REDIM and Steve shows up with the solution.
Title: Re: REDIM _PRESERVE bug
Post by: bplus on March 28, 2020, 10:25:00 pm
I hate to rain on parades but this can't be done in QB64

Code: QB64: [Select]
  1. Original2DArray() = Temp()

We went over this when I explained the need to use MEM to copy arrays.
https://www.qb64.org/forum/index.php?topic=1695.msg109301#msg109301

Apparently the illustration didn't stick.
Title: Re: REDIM _PRESERVE bug
Post by: SMcNeill on March 28, 2020, 11:08:08 pm
Quote from: bplus on March 28, 2020, 10:25:00 pm
I hate to rain on parades but this can't be done in QB64

Code: QB64: [Select]
  1. Original2DArray() = Temp()

We went over this when I explained the need to use MEM to copy arrays.
https://www.qb64.org/forum/index.php?topic=1695.msg109301#msg109301

Apparently the illustration didn't stick.

I swear, I've did exactly this before.  Did the behavior change over time?  Does it only work in some situations?  I dunno!  Right now my PC is down again (bad power supply once again; the one Best Buy installed 6 months ago was 300watts smaller than the original -- SURPRISE!!  IT DIDN'T LAST!!)  Once Uncle Sam sends me my CoronaCash Stimulus $$$, I'm going to use it to get my PC fixed and a few upgrades installed in it (SSD, new video card, a proper sized power supply), and then I'll be back and actually coding again afterwards.  Maybe then I can sort out why my brain tells me X() = Y() works, and sort out why/when it doesn't for us.

If all else fails, just _MEMCOPY from one to the other (or use a series of FOR/NEXT loops to copy the data).  I'll add digging into the issue to my to do list, for when I'm able to type on something more substantial than my iPad, or notebook's crappy keyboard.  ;)
Title: Re: REDIM _PRESERVE bug
Post by: FellippeHeitor on March 29, 2020, 12:00:34 am
In previous experiments, doing array1() = array2() in hopes of transferring their contents only resulted in the first element being copied. That wasn't recently.
Title: Re: REDIM _PRESERVE bug
Post by: OldMoses on March 29, 2020, 06:10:41 am
It didn't work for me just now. Good to know, I'll opt for _MEMCOPY if I have to do that.

Code: QB64: [Select]
  1. DIM A(10) AS INTEGER
  2. DIM B(10) AS INTEGER
  3. FOR x = 0 TO 10
  4.     A(x) = 10 - x
  6. B() = A()
  7. FOR y = 0 TO 10
  8.     PRINT A(y), B(y)
  10.  
Title: Re: REDIM _PRESERVE bug
Post by: Petr on March 29, 2020, 07:05:38 am
Why do you confuse the QB64 syntax and C syntax?

Code: QB64: [Select]
  1. DIM A(10) AS INTEGER
  2. DIM B(10) AS INTEGER
  3. FOR x = 0 TO 10
  4.     A(x) = 10 - x
  5.     B(x) = A(x)
  7.  
  8. FOR y = 0 TO 10
  9.     PRINT A(y), B(y)
  11.  
Title: Re: REDIM _PRESERVE bug
Post by: OldMoses on March 29, 2020, 07:29:29 am
Quote from: Petr on March 29, 2020, 07:05:38 am
Why do you confuse the QB64 syntax and C syntax?

Code: QB64: [Select]
  1. DIM A(10) AS INTEGER
  2. DIM B(10) AS INTEGER
  3. FOR x = 0 TO 10
  4.     A(x) = 10 - x
  5.     B(x) = A(x)
  7.  
  8. FOR y = 0 TO 10
  9.     PRINT A(y), B(y)
  11.  

Indeed, that is the way I would've normally done it before discovering MEM commands. I was just trying out the "bug", pointed out by bplus, for myself.
Title: Re: REDIM _PRESERVE bug
Post by: TempodiBasic on March 29, 2020, 08:08:01 am
Hi Guys
Hi Terry


in my little experience I have met this issue different times in coding when I've used an UDT ...

so you must remeber that REDIM PRESERVE works well only on a monodimensional array of a simple type data.
I have understood that the issue is around as the array has been packed into the memory... just like a stack so when you REDIM a multidimensional  array  your data seems spred among dimension of array, indeed they are simply stand into memory but we read the stack in different manner when  by changing the declaration of array ....
IMHO,saying that we have A(3,3,3) and we want to REDIM PRESERVE  A(4,5,2)the only solution is to write a manual SWAP routine that swaps the value from the first multidimensional array A(3,3,3) to a temporany multidimensional array B(4,5,2) and then copy this to the new dimensioned first array A(4,5,2).
As a alternative shortway you can REDIM 3 monodimensional array and then redim preserve them and at the end you store in the new dimensioned array....


you can see here some thread :

the original issue of PRESERVE  https://www.qb64.org/forum/index.php?topic=310.msg2155#msg2155 (https://www.qb64.org/forum/index.php?topic=310.msg2155#msg2155)
with the link to the thread on [abandoned, outdated and now likely malicious qb64 dot net website - don’t go there] in which Steve talks about REDIM  _PRESERVE and the managment of memory

UDT with fixed string vs variable string  https://www.qb64.org/forum/index.php?topic=1753.msg110020#msg110020 (https://www.qb64.org/forum/index.php?topic=1753.msg110020#msg110020)

another issue about _PRESERVE https://www.qb64.org/forum/index.php?topic=398.msg2766#msg2766 (https://www.qb64.org/forum/index.php?topic=398.msg2766#msg2766)

  [ This attachment cannot be displayed inline in 'Print Page' view ]  

copying array FOR vs _MEM  https://www.qb64.org/forum/index.php?topic=322.msg2156#msg2156 (https://www.qb64.org/forum/index.php?topic=322.msg2156#msg2156)

here linked list
https://www.qb64.org/forum/index.php?topic=982.msg101768#msg101768 (https://www.qb64.org/forum/index.php?topic=982.msg101768#msg101768)

here the sort routines
https://www.qb64.org/forum/index.php?topic=530.msg3830#msg3830 (https://www.qb64.org/forum/index.php?topic=530.msg3830#msg3830)

here loading and saving array to file using _MEM
https://www.qb64.org/forum/index.php?topic=387.msg2632#msg2632 (https://www.qb64.org/forum/index.php?topic=387.msg2632#msg2632)

Thanks to read
Title: Re: REDIM _PRESERVE bug
Post by: STxAxTIC on March 29, 2020, 08:21:54 am
Quote from: TempodiBasic on March 29, 2020, 08:08:01 am
here linked list
https://www.qb64.org/forum/index.php?----------------
 (https://www.qb64.org/forum/index.php?topic=982.msg101768#msg101768)

Nah. That was well-argued to not be the real thing.

My examples are rather convoluted but the simplest one must be qXed: https://www.qb64.org/forum/index.php?topic=1642.msg108603#msg108603 (https://www.qb64.org/forum/index.php?topic=1642.msg108603#msg108603)

The whole nugget is contained in the Cell UDT. If you see why I need this, and how it is generalized, you're all set. (All it does is store single characters. Simplest example I have.)

Code: QB64: [Select]
  1. TYPE Cell
  2.     Identity AS LONG
  3.     Pointer AS LONG
  4.     Lagger AS LONG
  5.     Content AS STRING * 1
Title: Re: REDIM _PRESERVE bug
Post by: FellippeHeitor on March 29, 2020, 09:16:03 am
Oh, that was a fun thread.
Title: Re: REDIM _PRESERVE bug
Post by: luke on March 29, 2020, 10:55:31 am
This is an interesting problem. Out of curiosity I've experimented with how one could rearrange the array data without the need for a temporary array (that is, after the memory region has been extended in size). I got a solution that works okay if you're expanding dimensions, but I've not been able to get anything that works for when you increase some dimensions and decrease others (I'm primarily interested in solutions for any number of dimensions, though I've been playing with 3 mainly).

That said, it's perhaps all just academic because for arrays that are large enough to matter realloc() probably is moving the data anyway. Maybe. I don't really know.

I know where in qb64.bas to modify it, it's just the way the array code is generated is an absolute pain to work with.
Title: Re: REDIM _PRESERVE bug
Post by: bplus on March 29, 2020, 11:17:52 am
Well in other Basics where the is no REDIM _PRESERVE the workaround is to over dimension the array and use your own pointers to track upper, and maybe even lower, bounds. Back in Basic's day, you didn't want to waste the space but now days it's much more available.

Title: Re: REDIM _PRESERVE bug
Post by: Dimster on March 29, 2020, 12:24:00 pm
So this could be a stupid question but...is it possible to make a Random Array? Would a Random Array be similar to a Redim _Preserve Array?
Title: Re: REDIM _PRESERVE bug
Post by: bplus on March 29, 2020, 12:34:31 pm
Quote from: Dimster on March 29, 2020, 12:24:00 pm
So this could be a stupid question but...is it possible to make a Random Array? Would a Random Array be similar to a Redim _Preserve Array?

You mean like Random access for a file?
Title: Re: REDIM _PRESERVE bug
Post by: Qwerkey on March 29, 2020, 12:48:27 pm
After watching Steve's video on REDIM _PRESERVE I came away with an irrational fear of ever using it!  If anybody can come up with the Greek for REDIM _PRESERVE and stick -phobia after it, there'd be a completely new psychological condition to be added to psychologists' lists.
Title: Re: REDIM _PRESERVE bug
Post by: SMcNeill on March 29, 2020, 12:56:16 pm
Quote from: Qwerkey on March 29, 2020, 12:48:27 pm
After watching Steve's video on REDIM _PRESERVE I came away with an irrational fear of ever using it!  If anybody can come with the Greek for REDIM _PRESERVE and stick -phobia after it, there'd be a completely new psychological condition to be added to psychologists' lists.

Represervaphobia?  Sounds like a winner to me!
Title: Re: REDIM _PRESERVE bug
Post by: bplus on March 29, 2020, 01:07:49 pm
Steve what did you say!?!? LOL

@Dimster
Do you know there are 2 kinds of arrays, Static and Dynamic are official names, and Statics are started with DIM and or STATIC and Dynamic are started with REDIM or $DYNAMIC metacommand (I never fooled with).

Actually Random access for an array sounds quite interesting, too bad the compiler has to reserve space in advance of using an array outside it's current boundary which is what REDIM _PRESERVE does. How does OS manage expanding files? filling up blocks and what, linked lists or version thereof to track them?
Title: Re: REDIM _PRESERVE bug
Post by: FellippeHeitor on March 29, 2020, 01:19:26 pm
REDIM the array and use _MEM to access its contents like it was a file.
Title: Re: REDIM _PRESERVE bug
Post by: Dimster on March 29, 2020, 01:39:09 pm
Ya bplus - like a Random File but files tend to hold data long term whereas an array is just during a program run....more short term temporary hold of a value. Arrays, are all sequential store and sequential access. Not exactly sure what a Random Array would look like/ act like but, if it did exist, then resetting the records length may easier than a Redim _Preserve of an array.

Just read your recent post re Static/Dynamic arrays. I've tried to use Subs and Functions and reading Multi Dimensional arrays backwards to build a quasi Random Dynamic Array. What a mess in coding, hard to follow the action to find the glitches. Was fun but abandoned it.
Title: Re: REDIM _PRESERVE bug
Post by: bplus on March 29, 2020, 02:39:17 pm
Quote from: Dimster on March 29, 2020, 01:39:09 pm
Ya bplus - like a Random File but files tend to hold data long term whereas an array is just during a program run....more short term temporary hold of a value. Arrays, are all sequential store and sequential access. Not exactly sure what a Random Array would look like/ act like but, if it did exist, then resetting the records length may easier than a Redim _Preserve of an array.

Just read your recent post re Static/Dynamic arrays. I've tried to use Subs and Functions and reading Multi Dimensional arrays backwards to build a quasi Random Dynamic Array. What a mess in coding, hard to follow the action to find the glitches. Was fun but abandoned it.

Arrays aren't accessed sequentially, just use index for the array "record" like get #file, recordNumber
dim records(lb to ub) as recordType
XPlace = records(recordNumber).x

Quote
I've tried to use Subs and Functions and reading Multi Dimensional arrays backwards to build a quasi Random Dynamic Array.
What? ;-)
Title: Re: REDIM _PRESERVE bug
Post by: Dimster on March 30, 2020, 02:13:01 pm
Alright, so no laughing. Here's what I was trying to do:

Imagine an Array like MainArray(a to z,1 to 12,1 to 31,1 to 50) ' I know this will exceed memory, bin there and found out, but just image it works.

So the first element is Years, 2nd is Months, 3rd is Days and 4th is 50 Events.

The program has multiple files which contain the data and MainArray is mostly used to Update the data. Now on a particular day you want to update Event 3 by 2, Event 25 by 6 and Event 18 by 4.
But the thing is those Events may be today's event or a past event, and chances are you know the event did occur but you don't have the specifics on when they occurred. The array needed a holding place in each of the year, month, and day for each of the events until the information of that event could be clarified. Redim _Preserve, then expand each element for a HOLD to place the data. If you should find a year for that data on the next run of the program, then Redim_Preseve, figure out if the year needs to be fitted in the middle of a to z or before or after a to z, and Swap the hell out of everything.

Sheeeze ...Be nice if there was just an automatic Random Access to a to z, whereby all that Redim_Preseve/Swapping is done.
Title: Re: REDIM _PRESERVE bug
Post by: bplus on March 30, 2020, 03:12:24 pm
@Dimster

I suspect you have over complicated a Database structure. YY-MM-DD is just a date field and if your events are just numbers within a range, 0 to 99 or 0 to 999... you could probably fit them all in a fixed length string that you could design a function to read or edit one of the 50 events with use of MID$
Title: Re: REDIM _PRESERVE bug
Post by: SMcNeill on March 30, 2020, 07:11:38 pm
Quote from: bplus on March 30, 2020, 03:12:24 pm
@Dimster

I suspect you have over complicated a Database structure. YY-MM-DD is just a date field and if your events are just numbers within a range, 0 to 99 or 0 to 999... you could probably fit them all in a fixed length string that you could design a function to read or edit one of the 50 events with use of MID$

Or else you need an index file to organize the data. The index might look something like:

YYYY-MM-DD-Event, Record-Number
2010-12-25-1, 1
2012-11-14-6, 2
2010-12-25-6, 3

The above would be an index pointing to the record position in the actual data file.  You wrote the data entry for your first event on Dec 25, 2012 in the first record spot.  The second event for that same date might be the 8th record.  The 6th event on that date is in the 3rd record position...

You create/write the data as needed, and the index is sortable for quick search and reference. 

If the data was of variable length, you'd do as above, but instead of Record Number, you'd track Start-Of-Data-Offset and End-Of-Data-Offset.
Title: Re: REDIM _PRESERVE bug
Post by: Dimster on March 31, 2020, 09:30:51 am
bplus - I agree combining the date fields to one YYYY-MM-DD field, stored as String, and info retrieved using Mid$ is an approach which I did consider but as I recall, there was a lot of coding for those times when I had only the month and event, or only the year and event, and sometimes just the event with lots of work to track down more specifics on the event. So I had to have the date field as only YYYY, or only MM, or only DD and the Event. Combining the date fields worked once I had all the data for a complete record. Also, the data in this array was used in calculations and stats, so it had to be converted from strings to numeric. I think I ran into some issues on the reliability of the numeric values which is why I settled on the date fields as numeric values at the outset rather than strings.

Steve - not exactly sure how the sortable index would work. The stat info needed would be say - how many times did event 3 occur on a Wednesday - or how often did event 6 and event 11 occur together in 2009. I see where the Updating would be a breeze, just add every thing to the last record and keep the array completely sequential. I was trying to find a logical place within the array to squeeze in the new data (thus the thought on Random access to the array), which made the calculations/stats more accurate if the data came to the formulas sequentially.

I did give up on that mighty array a number of years ago and only recently was trying to re-think it using some of the new, very rich QB64 syntax. I was looking at Redim _Preserve and it appears this works by making a copy of the entire array? I don't think it would say copy just a portion of an array? In the case of a my mighty array, if I wanted to make an update to say Feb 15, 2002, I could preserve all the data from the beginning of the array up to Feb 15, 2002 in one group, all the info in the array from Feb 16, 2002 in the next group - then write back the 1st group, then write my new entry for Feb 15, 2002 and then added the second group back? Redim _Preserve does all or nothing I gather???
Text Only | Text with Attachments