Loading contents of a file into an array

[random1]

Hi all

Need help loading the entire contents of a .txt into an array.  Below is what I am using now.
The file has 1000 entries / lines and I would like to load the entire file without having to use
line input. 

dim myarray(1 to 1000) as string

Erase myarray

L1=1
Open "file.txt" for input as #1
do while not eof(1)
line input #1, dat1$
dat1$=ltrim$(rtrim$(dat1$))
myarray(L1) = dat1$
L1=L1+1
loop
close

[bplus]

Hi random1

Can I call you rnd for short?

Well I see this is a trick question, binary access does not like string arrays.

So here is a trick answer:

Code: QB64: [Select]

1.  
2. DIM myarray(1 TO 20) AS STRING * 60 '<<<<<<<<<<< compilation error....  blah phooey!
3. FOR i = 1 TO 20
4.     myarray$ = myarray$ + LEFT$("This is line " + STR$(i) + SPACE$(60), 60)
5.     'myarray(i) = LEFT$("This is line " + STR$(i) + SPACE$(60), 60)  '<<<<<<<<<<< compilation error....
6. NEXT
7.  
8. OPEN "file.txt" FOR BINARY AS #1
9. PUT #1, , myarray$
10. 'PUT #1, , myarray() '<<<<<<<<<<< compilation error....
11. CLOSE #1
12.  
13. 'ERASE myarray
14.  
15. 'alternate open
16. OPEN "file.txt" FOR BINARY AS #1
17. l = LOF(1)
18. myarray$ = SPACE$(l)
19. GET #1, , myarray$
20. 'GET #1, , myarray() '<<<<<<<<<<< compilation error....
21. CLOSE #1
22. FOR i = 1 TO 20
23.     PRINT MID$(myarray$, (i - 1) * 60 + 1, 60)
24.     'PRINT myarray(i) '<<<<<<<<<<< compilation error....
25. NEXT
26.  

[SMcNeill]

Hi random1

Can I call you rnd for short?

Well I see this is a trick question, binary access does not like string arrays.

Binary access works lovely fine with string arrays; you just need to read and parse for them:

Code: QB64: [Select]

1. DIM array(10000) AS STRING
2.  
3. OPEN "temp.txt" FOR OUTPUT AS #1
4. FOR i = 1 TO 10000
5.     PRINT #1, "Junk Line of Input #"; i
6. NEXT
7. CLOSE
8.  
9. OPEN "temp.txt" FOR BINARY AS #1 'Load the data
10. l = LOF(1) 'all of it at once
11. junk$ = SPACE$(l)
12. GET #1, 1, junk$ 'The file is now in junk$
13. CLOSE
14.  
15.  
16. ParseData array(), junk$ 'And we parse junk$ into our array
17. FOR i = 1 TO 10
18.     PRINT i, array(i) 'first 10 results
19. NEXT
20. PRINT ".... Skip a few..."
21. FOR i = UBOUND(array) - 10 TO UBOUND(array)
22.     PRINT i, array(i) 'last 10 results
23. NEXT
24.  
25.  
26. SUB ParseData (array() AS STRING, text$)
27.     temp$ = text$
28.     IF INSTR(text$, CHR$(13)) THEN CRLF = 2 ELSE CRLF = 1
29.     DO
30.         IF l >= LEN(temp$) THEN EXIT DO
31.         l = INSTR(temp$, CHR$(10))
32.         count = count + 1
33.         array(count) = LEFT$(temp$, l - CRLF)
34.         temp$ = MID$(temp$, l + 1)
35.     LOOP
36. END SUB

[bplus]

Rats SAM, you beat me :D

Code: QB64: [Select]

1.  
2. REDIM myarray(1 TO 20) AS STRING 'make a test array
3. FOR i = 1 TO 20
4.     myarray(i) = "This is line " + STR$(i)
5.     B$ = B$ + myarray(i) + CHR$(13) + CHR$(10)
6. NEXT
7.  
8. OPEN "file.txt" FOR BINARY AS #1
9. PUT #1, , B$
10. CLOSE #1
11.  
12. ' we have a sequential file of strings to load
13. REDIM myarray(0) AS STRING
14.  
15. 'alternate open
16. OPEN "file.txt" FOR BINARY AS #1
17. l = LOF(1)
18. B$ = SPACE$(l)
19. GET #1, , B$
20. CLOSE #1
21.  
22. Split1000 B$, CHR$(13) + CHR$(10), myarray()
23. FOR i = 0 TO UBOUND(myarray)
24.     PRINT myarray(i)
25. NEXT
26.  
27. 'notes: REDIM the array(0) to be loaded before calling Split '<<<<<<<<<<<<<<<<<<<<<<< IMPORTANT!!!!
28. SUB Split1000 (mystr AS STRING, delim AS STRING, arr() AS STRING)
29.     ' bplus modifications of Galleon fix of Bulrush Split reply #13
30.     ' http://xmaxw.[abandoned, outdated and now likely malicious qb64 dot net website - don’t go there]/forum/index.php?topic=1612.0
31.     ' this sub further developed and tested here: \test\Strings\Split test.bas
32.     DIM copy AS STRING, p AS LONG, curpos AS LONG, arrpos AS LONG, dpos AS LONG, LD AS INTEGER
33.  
34.     copy = mystr 'make copy since we are messing with mystr when the delimiter is a space
35.  
36.     'special case if delim is space, probably want to remove all excess space
37.     IF delim = " " THEN
38.         copy = RTRIM$(LTRIM$(copy))
39.         p = INSTR(copy, "  ")
40.         WHILE p > 0
41.             copy = MID$(copy, 1, p - 1) + MID$(copy, p + 1)
42.             p = INSTR(copy, "  ")
43.         WEND
44.     END IF
45.     curpos = 1
46.     arrpos = 0
47.     LD = LEN(delim) 'mod
48.     dpos = INSTR(curpos, copy, delim)
49.     DO UNTIL dpos = 0
50.         arr(arrpos) = MID$(copy, curpos, dpos - curpos)
51.         arrpos = arrpos + 1
52.         IF arrpos > UBOUND(arr) THEN REDIM _PRESERVE arr(UBOUND(arr) + 1000) AS STRING
53.         curpos = dpos + LD
54.         dpos = INSTR(curpos, copy, delim)
55.     LOOP
56.     arr(arrpos) = MID$(copy, curpos)
57.     REDIM _PRESERVE arr(arrpos) AS STRING 'need this line? YES to get the ubound correct
58. END SUB
59.  
60.  
61.  

Binary access works lovely fine with string arrays; you just need to read and parse for them:

I still say, Binary access does not like string arrays, SAM is the one one who works "lovely fine" with string arrays.

You can't do this with strings:

Code: QB64: [Select]

1. DIM myarray(1 TO 20) AS INTEGER
2. FOR i = 1 TO 20
3.     myarray(i) = i ^ 2
4. NEXT
5. OPEN "mySquares.dat" FOR BINARY AS #1
6. PUT #1, , myarray()
7. CLOSE #1
8. ERASE myarray
9. OPEN "mySquares.dat" FOR BINARY AS #1
10. GET #1, , myarray()
11. CLOSE #1
12. FOR i = 1 TO 20
13.     PRINT myarray(i)
14. NEXT
15.  

You can't even do it with fixed length strings.

And now Pete will say, "That's because they have been fixed."

[random1]

Hi all

Thanks for all the replies, got me thinking.  Since all the strings in the files are fixed
length, why not just load the data into a string using open & binary and then process
the string much the same way I would an array.   The goal was to speed up the time
it takes to load and process over 200 files.   
The stuff posted here will not go to waste, it's going into my tool-box as i am sure I will
make use of it down the road.

Thanks again

R1 
     

   
 
 

[SMcNeill]

I still say, Binary access does not like string arrays, SAM is the one one who works "lovely fine" with string arrays.

You can't do this with strings:

Code: QB64: [Select]

1. DIM myarray(1 TO 20) AS INTEGER
2. FOR i = 1 TO 20
3.     myarray(i) = i ^ 2
4. NEXT
5. OPEN "mySquares.dat" FOR BINARY AS #1
6. PUT #1, , myarray()
7. CLOSE #1
8. ERASE myarray
9. OPEN "mySquares.dat" FOR BINARY AS #1
10. GET #1, , myarray()
11. CLOSE #1
12. FOR i = 1 TO 20
13.     PRINT myarray(i)
14. NEXT
15.  

You can't even do it with fixed length strings.

And now Pete will say, "That's because they have been fixed."

For random length strings, you can do it like so:

Code: QB64: [Select]

1. RANDOMIZE TIMER
2. DIM myarray(1 TO 20) AS STRING
3. DIM myarray2(1 TO 20) AS STRING
4.  
5. OPEN "mySquares.dat" FOR BINARY AS #1
6. FOR i = LBOUND(myarray) TO UBOUND(myarray)
7.     junk$ = "This is just some random length junk string"
8.     myarray(i) = MID$(junk$, INT(RND * 10), INT(RND * LEN(junk$)))
9.     l& = LEN(myarray(i)) 'get the string size
10.     PUT #1, , l& 'put the size value
11.     PUT #1, , myarray(i) 'put the string
12. NEXT
13. CLOSE #1
14.  
15. OPEN "mySquares.dat" FOR BINARY AS #1
16. FOR i = LBOUND(myarray2) TO UBOUND(myarray2)
17.     GET #1, , l& 'get the size
18.     myarray2(i) = SPACE$(l&) 'set our string to recieve that size
19.     GET #1, , myarray2(i) 'get the string
20. NEXT
21. CLOSE
22. FOR i = 1 TO 20
23.     PRINT myarray(i); TAB(40); myarray2(i)
24. NEXT
25.  

And when it comes to fixed length string arrays, you can do them even easier: 

Code: QB64: [Select]

1. RANDOMIZE TIMER
2. DIM myarray(1 TO 20) AS STRING * 40
3.  
4. OPEN "mySquares.dat" FOR BINARY AS #1
5. FOR i = LBOUND(myarray) TO UBOUND(myarray)
6.     junk$ = "This is just some random length junk string"
7.     myarray(i) = MID$(junk$, INT(RND * 10), INT(RND * LEN(junk$)))
8. NEXT
9. temp$ = myarray()
10. PUT #1, , temp$
11. CLOSE
12.  
13. OPEN "mySquares.dat" FOR BINARY AS #1
14. temp$ = SPACE$(LEN(myarray())) 'clear the string so we know we're getting it anew.
15. GET #1, , temp$ 'get the whole array
16. myarray() = temp$
17. CLOSE
18. FOR i = 1 TO 20
19.     PRINT i, myarray(i)
20. NEXT

[bplus]

Hi Steve,

The first demo, not so impressive, you are doing things line by line maybe even slower than a sequential file and load, certainly more complex.

2nd code demo opens my eyes, you can set a string to an array and vice versa! ? ! That is cool! There you are doing things in file chunks.

[SMcNeill]

Hi Steve,

The first demo, not so impressive, you are doing things line by line maybe even slower than a sequential file and load, certainly more complex.

Aye, but since you’re working with variable length strings, you either need to know the size or have a separator character to keep them apart.  If your string might hold control characters (Array(1) = “Hello” + CHR$(13) + “World”), you can’t count on LINE INPUT working properly for you (it’d read the above as 2 lines and not one).  When terminators aren’t possible, size has to be stored separate just so you know what’s what in the file.

“Welikecheese” — one word?  Two 6-letter words?  Three words?  With variable length data, we honestly have no way of knowing, without either recording the size or using a terminator code.  ;)