Text Only | Text with Attachments

QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: random1 on August 06, 2019, 07:16:58 pm

Title: Loading contents of a file into an array
Post by: random1 on August 06, 2019, 07:16:58 pm
Hi all

Need help loading the entire contents of a .txt into an array.  Below is what I am using now.
The file has 1000 entries / lines and I would like to load the entire file without having to use
line input. 

dim myarray(1 to 1000) as string

Erase myarray

L1=1
Open "file.txt" for input as #1
do while not eof(1)
line input #1, dat1$
dat1$=ltrim$(rtrim$(dat1$))
myarray(L1) = dat1$
L1=L1+1
loop
close
Title: Re: Loading contents of a file into an array
Post by: bplus on August 06, 2019, 08:26:55 pm
Hi random1

Can I call you rnd for short?

Well I see this is a trick question, binary access does not like string arrays.

So here is a trick answer:
Code: QB64: [Select]
  1.  
  2. DIM myarray(1 TO 20) AS STRING * 60 '<<<<<<<<<<< compilation error....  blah phooey!
  3. FOR i = 1 TO 20
  4.     myarray$ = myarray$ + LEFT$("This is line " + STR$(i) + SPACE$(60), 60)
  5.     'myarray(i) = LEFT$("This is line " + STR$(i) + SPACE$(60), 60)  '<<<<<<<<<<< compilation error....
  7.  
  8. OPEN "file.txt" FOR BINARY AS #1
  9. PUT #1, , myarray$
  10. 'PUT #1, , myarray() '<<<<<<<<<<< compilation error....
  12.  
  13. 'ERASE myarray
  14.  
  15. 'alternate open
  16. OPEN "file.txt" FOR BINARY AS #1
  17. l = LOF(1)
  18. myarray$ = SPACE$(l)
  19. GET #1, , myarray$
  20. 'GET #1, , myarray() '<<<<<<<<<<< compilation error....
  22. FOR i = 1 TO 20
  23.     PRINT MID$(myarray$, (i - 1) * 60 + 1, 60)
  24.     'PRINT myarray(i) '<<<<<<<<<<< compilation error....
  26.  
Title: Re: Loading contents of a file into an array
Post by: SMcNeill on August 06, 2019, 08:56:15 pm
Quote from: bplus on August 06, 2019, 08:26:55 pm
Hi random1

Can I call you rnd for short?

Well I see this is a trick question, binary access does not like string arrays.

Binary access works lovely fine with string arrays; you just need to read and parse for them:

Code: QB64: [Select]
  1. DIM array(10000) AS STRING
  2.  
  3. OPEN "temp.txt" FOR OUTPUT AS #1
  4. FOR i = 1 TO 10000
  5.     PRINT #1, "Junk Line of Input #"; i
  8.  
  9. OPEN "temp.txt" FOR BINARY AS #1 'Load the data
  10. l = LOF(1) 'all of it at once
  11. junk$ = SPACE$(l)
  12. GET #1, 1, junk$ 'The file is now in junk$
  14.  
  15.  
  16. ParseData array(), junk$ 'And we parse junk$ into our array
  17. FOR i = 1 TO 10
  18.     PRINT i, array(i) 'first 10 results
  20. PRINT ".... Skip a few..."
  21. FOR i = UBOUND(array) - 10 TO UBOUND(array)
  22.     PRINT i, array(i) 'last 10 results
  24.  
  25.  
  26. SUB ParseData (array() AS STRING, text$)
  27.     temp$ = text$
  28.     IF INSTR(text$, CHR$(13)) THEN CRLF = 2 ELSE CRLF = 1
  29.     DO
  30.         IF l >= LEN(temp$) THEN EXIT DO
  31.         l = INSTR(temp$, CHR$(10))
  32.         count = count + 1
  33.         array(count) = LEFT$(temp$, l - CRLF)
  34.         temp$ = MID$(temp$, l + 1)
  35.     LOOP
Title: Re: Loading contents of a file into an array
Post by: bplus on August 06, 2019, 08:59:06 pm
Rats SAM, you beat me :D
Code: QB64: [Select]
  1.  
  2. REDIM myarray(1 TO 20) AS STRING 'make a test array
  3. FOR i = 1 TO 20
  4.     myarray(i) = "This is line " + STR$(i)
  5.     B$ = B$ + myarray(i) + CHR$(13) + CHR$(10)
  7.  
  8. OPEN "file.txt" FOR BINARY AS #1
  9. PUT #1, , B$
  11.  
  12. ' we have a sequential file of strings to load
  13. REDIM myarray(0) AS STRING
  14.  
  15. 'alternate open
  16. OPEN "file.txt" FOR BINARY AS #1
  17. l = LOF(1)
  18. B$ = SPACE$(l)
  19. GET #1, , B$
  21.  
  22. Split1000 B$, CHR$(13) + CHR$(10), myarray()
  23. FOR i = 0 TO UBOUND(myarray)
  24.     PRINT myarray(i)
  26.  
  27. 'notes: REDIM the array(0) to be loaded before calling Split '<<<<<<<<<<<<<<<<<<<<<<< IMPORTANT!!!!
  28. SUB Split1000 (mystr AS STRING, delim AS STRING, arr() AS STRING)
  29.     ' bplus modifications of Galleon fix of Bulrush Split reply #13
  30.     ' http://xmaxw.[abandoned, outdated and now likely malicious qb64 dot net website - don’t go there]/forum/index.php?topic=1612.0
  31.     ' this sub further developed and tested here: \test\Strings\Split test.bas
  32.     DIM copy AS STRING, p AS LONG, curpos AS LONG, arrpos AS LONG, dpos AS LONG, LD AS INTEGER
  33.  
  34.     copy = mystr 'make copy since we are messing with mystr when the delimiter is a space
  35.  
  36.     'special case if delim is space, probably want to remove all excess space
  37.     IF delim = " " THEN
  38.         copy = RTRIM$(LTRIM$(copy))
  39.         p = INSTR(copy, "  ")
  40.         WHILE p > 0
  41.             copy = MID$(copy, 1, p - 1) + MID$(copy, p + 1)
  42.             p = INSTR(copy, "  ")
  43.         WEND
  44.     END IF
  45.     curpos = 1
  46.     arrpos = 0
  47.     LD = LEN(delim) 'mod
  48.     dpos = INSTR(curpos, copy, delim)
  49.     DO UNTIL dpos = 0
  50.         arr(arrpos) = MID$(copy, curpos, dpos - curpos)
  51.         arrpos = arrpos + 1
  52.         IF arrpos > UBOUND(arr) THEN REDIM _PRESERVE arr(UBOUND(arr) + 1000) AS STRING
  53.         curpos = dpos + LD
  54.         dpos = INSTR(curpos, copy, delim)
  55.     LOOP
  56.     arr(arrpos) = MID$(copy, curpos)
  57.     REDIM _PRESERVE arr(arrpos) AS STRING 'need this line? YES to get the ubound correct
  59.  
  60.  
  61.  

Quote
Binary access works lovely fine with string arrays; you just need to read and parse for them:
I still say, Binary access does not like string arrays, SAM is the one one who works "lovely fine" with string arrays.

You can't do this with strings:
Code: QB64: [Select]
  1. DIM myarray(1 TO 20) AS INTEGER
  2. FOR i = 1 TO 20
  3.     myarray(i) = i ^ 2
  5. OPEN "mySquares.dat" FOR BINARY AS #1
  6. PUT #1, , myarray()
  8. ERASE myarray
  9. OPEN "mySquares.dat" FOR BINARY AS #1
  10. GET #1, , myarray()
  12. FOR i = 1 TO 20
  13.     PRINT myarray(i)
  15.  

You can't even do it with fixed length strings.

And now Pete will say, "That's because they have been fixed."
Title: Re: Loading contents of a file into an array
Post by: random1 on August 07, 2019, 03:36:30 pm
Hi all

Thanks for all the replies, got me thinking.  Since all the strings in the files are fixed
length, why not just load the data into a string using open & binary and then process
the string much the same way I would an array.   The goal was to speed up the time
it takes to load and process over 200 files.   
The stuff posted here will not go to waste, it's going into my tool-box as i am sure I will
make use of it down the road.

Thanks again

R1 
     

   
 
 
Title: Re: Loading contents of a file into an array
Post by: SMcNeill on August 07, 2019, 04:37:01 pm
Quote from: bplus on August 06, 2019, 08:59:06 pm

I still say, Binary access does not like string arrays, SAM is the one one who works "lovely fine" with string arrays.

You can't do this with strings:
Code: QB64: [Select]
  1. DIM myarray(1 TO 20) AS INTEGER
  2. FOR i = 1 TO 20
  3.     myarray(i) = i ^ 2
  5. OPEN "mySquares.dat" FOR BINARY AS #1
  6. PUT #1, , myarray()
  8. ERASE myarray
  9. OPEN "mySquares.dat" FOR BINARY AS #1
  10. GET #1, , myarray()
  12. FOR i = 1 TO 20
  13.     PRINT myarray(i)
  15.  

You can't even do it with fixed length strings.

And now Pete will say, "That's because they have been fixed."

For random length strings, you can do it like so:
Code: QB64: [Select]
  2. DIM myarray(1 TO 20) AS STRING
  3. DIM myarray2(1 TO 20) AS STRING
  4.  
  5. OPEN "mySquares.dat" FOR BINARY AS #1
  6. FOR i = LBOUND(myarray) TO UBOUND(myarray)
  7.     junk$ = "This is just some random length junk string"
  8.     myarray(i) = MID$(junk$, INT(RND * 10), INT(RND * LEN(junk$)))
  9.     l& = LEN(myarray(i)) 'get the string size
  10.     PUT #1, , l& 'put the size value
  11.     PUT #1, , myarray(i) 'put the string
  14.  
  15. OPEN "mySquares.dat" FOR BINARY AS #1
  16. FOR i = LBOUND(myarray2) TO UBOUND(myarray2)
  17.     GET #1, , l& 'get the size
  18.     myarray2(i) = SPACE$(l&) 'set our string to recieve that size
  19.     GET #1, , myarray2(i) 'get the string
  22. FOR i = 1 TO 20
  23.     PRINT myarray(i); TAB(40); myarray2(i)
  25.  


And when it comes to fixed length string arrays, you can do them even easier: 
Code: QB64: [Select]
  2. DIM myarray(1 TO 20) AS STRING * 40
  3.  
  4. OPEN "mySquares.dat" FOR BINARY AS #1
  5. FOR i = LBOUND(myarray) TO UBOUND(myarray)
  6.     junk$ = "This is just some random length junk string"
  7.     myarray(i) = MID$(junk$, INT(RND * 10), INT(RND * LEN(junk$)))
  9. temp$ = myarray()
  10. PUT #1, , temp$
  12.  
  13. OPEN "mySquares.dat" FOR BINARY AS #1
  14. temp$ = SPACE$(LEN(myarray())) 'clear the string so we know we're getting it anew.
  15. GET #1, , temp$ 'get the whole array
  16. myarray() = temp$
  18. FOR i = 1 TO 20
  19.     PRINT i, myarray(i)

Title: Re: Loading contents of a file into an array
Post by: bplus on August 07, 2019, 06:43:31 pm
Hi Steve,

The first demo, not so impressive, you are doing things line by line maybe even slower than a sequential file and load, certainly more complex.

2nd code demo opens my eyes, you can set a string to an array and vice versa! ? ! That is cool! There you are doing things in file chunks.
Title: Re: Loading contents of a file into an array
Post by: SMcNeill on August 07, 2019, 07:00:17 pm
Quote from: bplus on August 07, 2019, 06:43:31 pm
Hi Steve,

The first demo, not so impressive, you are doing things line by line maybe even slower than a sequential file and load, certainly more complex.

Aye, but since you’re working with variable length strings, you either need to know the size or have a separator character to keep them apart.  If your string might hold control characters (Array(1) = “Hello” + CHR$(13) + “World”), you can’t count on LINE INPUT working properly for you (it’d read the above as 2 lines and not one).  When terminators aren’t possible, size has to be stored separate just so you know what’s what in the file.

“Welikecheese” — one word?  Two 6-letter words?  Three words?  With variable length data, we honestly have no way of knowing, without either recording the size or using a terminator code.  ;)
Text Only | Text with Attachments