QB64 File I/O speeds compared to QB are really slow

[davidf]

Apologies i must have pressed post halfway through typing

I have just been testing some programs that were originally written in VBDOS (Never used the GUI stuff)

I got the programs running without problems except that the speed was much slower.

One section that very marked difference was taking about 4 seconds (VBDOS) and in QB64 over 65 seconds.

After further investigation it was apparent that the time differences were taking place during file i/o.

The programs are used for nesting different components on sheets, and then generating the Laser CNC cutting programs.
It is during the generation of these CNC programs , there is a lot of file i/o reading and writing of text files.

I then wrote a small program to see the differences between QB, VBDOS and QB64 , the results are significant.
The program creates 100,000 LINES OF CSV data ( 5 data items , ie 4 commas), using the Print #, Write #, functions

The program then reads back the data
Using Input #, and then Line Input # functions
In the QB64 test , i have used the Line input # (Binary mode) it is not available in QB and VBDOS

The times shown are in seconds

Task                  QB          VBDOS        QB64         VB5
Print #              1.76         1.76         12.14        0.89
Write #              1.71         1.80          7.74        0.89
Input #              0.37         0.26         42.47        0.25
Line Input #         0.21         0.17         42.47        0.19
Binary Line Input#    N/A          N/A          1.37         N/A

The program uses a directory of C:\dave
SO if you are testing it, you will need to create that directory or change to different one.
If you want to the Binary Line Input , you need to set testbinio% = 1


I am thinking that there may be something wrong with my installtion, but before i reinstall again.
Has anybody else experienced with slow i/o performance, or is this just how it is.

Code: QB64: [Select]

1. sttimer = TIMER
2.  
3. doprttest% = 1
4.  
5. IF doprttest% = 1 THEN
6.  
7.     OPEN "c:\dave\test_pr1.dat" FOR OUTPUT AS #1
8.  
9.     DIM datastr$(5)
10.  
11.     FOR i& = 1 TO 100000
12.  
13.         datastr$(1) = "Data_p1_" + formalt$((i&), "0")
14.         datastr$(2) = "Data_p2_" + formalt$((i&), "0")
15.         datastr$(3) = "Data_p3_" + formalt$((i&), "0")
16.         datastr$(4) = "Data_p4_" + formalt$((i&), "0")
17.         datastr$(5) = "Data_p5_" + formalt$((i&), "0")
18.  
19.  
20.         PRINT #1, datastr$(1); ","; datastr$(2); ","; datastr$(3); ","; datastr$(4); ","; datastr$(5)
21.  
22.     NEXT
23.  
24.     CLOSE #1
25.  
26.     fitimer = TIMER
27.  
28.     timetaken = fitimer - sttimer
29.     PRINT
30.     PRINT "test_pri1 (time):-"; formalt$((timetaken), "0.00")
31.  
32.  
33.     sttimer = TIMER
34.  
35.     OPEN "c:\dave\test_wr1.dat" FOR OUTPUT AS #1
36.  
37.  
38.     FOR i& = 1 TO 100000
39.  
40.         datastr$(1) = "Data_p1_" + formalt$((i&), "0")
41.         datastr$(2) = "Data_p2_" + formalt$((i&), "0")
42.         datastr$(3) = "Data_p3_" + formalt$((i&), "0")
43.         datastr$(4) = "Data_p4_" + formalt$((i&), "0")
44.         datastr$(5) = "Data_p5_" + formalt$((i&), "0")
45.  
46.  
47.         WRITE #1, datastr$(1), datastr$(2), datastr$(3), datastr$(4), datastr$(5)
48.  
49.     NEXT
50.  
51.     CLOSE #1
52.  
53.     fitimer = TIMER
54.     timetaken = fitimer - sttimer
55.     PRINT
56.     PRINT "test_wri1 (time):-"; formalt$((timetaken), "0.00")
57.  
58.     sttimer = TIMER
59.     OPEN "c:\dave\test_pr1.dat" FOR INPUT AS #1
60.  
61.     linecount& = 0
62.     WHILE NOT EOF(1)
63.         INPUT #1, datastr$(1), datastr$(2), datastr$(3), datastr$(4), datastr$(5)
64.         linecount& = linecount& + 1
65.     WEND
66.  
67.     CLOSE #1
68.  
69.     fitimer = TIMER
70.     timetaken = fitimer - sttimer
71.     PRINT
72.     PRINT "read using input (time):-"; formalt$((timetaken), "0.00")
73.     PRINT "linecount&="; linecount&
74.  
75.  
76.     sttimer = TIMER
77.     OPEN "c:\dave\test_pr1.dat" FOR INPUT AS #1
78.  
79.     linecount& = 0
80.     WHILE NOT EOF(1)
81.         LINE INPUT #1, tdata$
82.         linecount& = linecount& + 1
83.         parsedata% = 0
84.         IF parsedata% = 1 THEN
85.             stpos% = 0
86.             Index% = 1
87.             DO
88.                 nextpos% = INSTR(stpos% + 1, tdata$, ",")
89.                 IF nextpos% > stpos% THEN
90.                     datastr$(Index%) = MID$(tdata$, stpos% + 1, nextpos% - stpos%)
91.                     Index% = Index% + 1
92.                     stpos% = nextpos%
93.                 ELSE
94.                     datastr$(Index%) = MID$(tdata$, stpos% + 1)
95.                     EXIT DO
96.                 END IF
97.             LOOP
98.  
99.             'FOR dli% = 1 TO 5
100.             '  PRINT datastr$(dli%)
101.             'NEXT
102.             'INPUT s$
103.         END IF
104.  
105.     WEND
106.  
107.     CLOSE #1
108.  
109.     fitimer = TIMER
110.     timetaken = fitimer - sttimer
111.     PRINT
112.     PRINT "read using line input (time):-"; formalt$((timetaken), "0.00")
113.     PRINT "linecount&="; linecount&
114. END IF
115.  
116. testbinio% = 0
117. IF testbinio% = 1 THEN
118.  
119.     sttimer = TIMER
120.     OPEN "c:\dave\test_pr1.dat" FOR BINARY AS #1
121.  
122.     SEEK #1, 1
123.  
124.     loffile& = LOF(1)
125.     'PRINT "loffile&="; loffile&
126.     linecount& = 0
127.     DO
128.         'IF SEEK(1) < loffile& THEN
129.         IF NOT EOF(1) THEN
130.             LINE INPUT #1, tdata$
131.             'PRINT "Length of data="; LEN(tdata$); "Current file pos="; SEEK(1)
132.             linecount& = linecount& + 1
133.  
134.         ELSE
135.             'PRINT "Current file pos="; SEEK(1)
136.             EXIT DO
137.         END IF
138.  
139.     LOOP
140.     CLOSE #1
141.  
142.     fitimer = TIMER
143.     timetaken = fitimer - sttimer
144.     PRINT
145.     PRINT "read using Binary line input (time):-"; formalt$((timetaken), "0.00")
146.     PRINT "linecount&="; linecount&
147.  
148. END IF
149.  
150. PRINT "press any key to continue"
151. DO
152.     a$ = INKEY$
153. LOOP WHILE a$ = ""
154.  
155. END
156.  
157. FUNCTION formalt$ (num AS DOUBLE, precstr$)
158.     STATIC tnum AS DOUBLE
159.     dotpos% = INSTR(precstr$, ".")
160.  
161.     'PRINT "called ("; num; ","; precstr$; ")"
162.  
163.     tnum = num
164.     IF dotpos% >= 1 THEN
165.         trailingzeros% = LEN(precstr$) - dotpos%
166.  
167.         leadingzeros% = dotpos% - 1
168.  
169.         'PRINT dotpos%, trailingzeros%
170.  
171.  
172.         multiplyby& = 10 ^ trailingzeros%
173.         tempnum& = INT(tnum * multiplyby&)
174.  
175.  
176.  
177.         tnumstr$ = LTRIM$(RTRIM$(STR$(tempnum&)))
178.         'PRINT tnumstr$, tempnum&
179.  
180.         newtrailingpartnumstr$ = RIGHT$(tnumstr$, trailingzeros%)
181.         newleadingnumstr$ = LEFT$(tnumstr$, LEN(tnumstr$) - trailingzeros%)
182.  
183.  
184.         numberoffleadingzerostoadd% = leadingzeros% - LEN(newleadingnumstr$)
185.  
186.         IF numberoffleadingzerostoadd% > 0 THEN
187.             addleadingzero$ = STRING$(numberoffleadingzerostoadd%, "0")
188.         ELSE
189.             addleadingzero$ = ""
190.         END IF
191.  
192.         newleadingnumstr$ = addleadingzero$ + newleadingnumstr$
193.  
194.  
195.  
196.         tnumstr$ = newleadingnumstr$ + "." + newtrailingpartnumstr$
197.  
198.  
199.     ELSE
200.         leadingzeros% = LEN(precstr$)
201.         tnum = INT(tnum + 0.5)
202.         trailingzeros% = 0
203.         tnumstr$ = RTRIM$(LTRIM$(STR$(tnum)))
204.  
205.         numberoffleadingzerostoadd% = leadingzeros% - LEN(tnumstr$)
206.  
207.         IF numberoffleadingzerostoadd% > 0 THEN
208.             addleadingzero$ = STRING$(numberoffleadingzerostoadd%, "0")
209.         ELSE
210.             addleadingzero$ = ""
211.         END IF
212.  
213.         tnumstr$ = addleadingzero$ + tnumstr$
214.  
215.  
216.     END IF
217.  
218.  
219.     'PRINT tnumstr$
220.  
221.     formalt$ = tnumstr$
222.  
223.  
224. END FUNCTION
225.  

[RhoSigma]

Sounds like a similar task I do with CNC milling programs. Maybe it's a good idea to load the file(s) once into a buffer and then do all operations in the buffer. A couple weeks ago I've developed the String Buffer System for such purposes, see https://www.qb64.org/forum/index.php?topic=2101.msg113328#msg113328

PS. it's the SB-Storage folder in the Library collection.