There is a sequential file line limit in Basic?

[hotlnc]

It looks like I've hit a boundary that I didn't know existed.  I'm doing sequential reads and writes on files that have ASCII text lines (assembler listing output files). My QB64 programs have been working great until I started to write and read files with 10,000+ lines in them. I can process no further than about 10K lines. Is there a line limit in Basic/QB64? I am not holding the lines in memory. I process a line or two and then output that line. So this is not an memory issue.

Is there a work around?

[SMcNeill]

No limit like that.  Can you share your code with us?  I'll be happy to take a look and see if I can help sort out what's going on for you.

[hotlnc]

Sure. Here you go.

[SMcNeill]

And any chance for a test file to run it with?  I just downloaded the BAS and am looking for anything which might stand out even now.  :)

[hotlnc]

Sure.  Thanks for your help.

The file labeled "GpsTx3G.LST" is small enough to work fine. Output file "GpsTx3G0p0.ASM
The file labeled "J13Gps4G.LST has the 10K limit on the output file: "J13Gps4G0p0.ASM"

[hotlnc]

I guess I can only attach one file at a time.  Here is the smaller one.

[SMcNeill]

Oddly enough, I didn't seem to have any issues with it. 

Code: QB64: [Select]

1. '160115 Added Better support for Multiple Temp Macros (Temp_Mack1, Temp_Mack2 etc)
2. '160624 Removed black lines at top of certain macro's
3. DEFSTR A
4. DEFINT X
5. DEFLNG L
6.  
7. _TITLE "Macro Flatten to Assembly File -- REV 160624"
8. '
9. aCommand = COMMAND$
10. x1 = LEN(aCommand) 'we need to strip it out, so get SIZE of string
11. IF x1 > 0 THEN 'IF called by multi-edit, a command will have the entire path and file
12.     DO WHILE x1 <> 0 'This routine will back through the characters, looking for the first
13.         IF MID$(aCommand, x1, 1) <> "\" THEN 'back slash. The characters preceeding the back slash is the file name
14.             x1 = x1 - 1
15.         ELSE
16.             EXIT DO
17.         END IF
18.     LOOP
19.     aFile1 = MID$(aCommand, x1 + 1)
20. END IF
21. '-----------------------------------------------------------------
22. aCommand = COMMAND$
23. '   aCommand = "trolling"
24. 'aCommand = "c:\mplabfls\active\" + aFile1 + "\" + aFile1
25. aCommand = ".\J13Gps4G"
26. x2 = LEN(aCommand)
27. x1 = x2
28.  
29. DO UNTIL MID$(aCommand, x2, 1) = "\"
30.     x2 = x2 - 1
31.     IF x2 <= 1 THEN
32.         EXIT DO
33.     END IF
34. LOOP
35.  
36. IF x2 <= 1 THEN
37.     PRINT aCommand
38.     END 'STOP
39. END IF
40.  
41. 'x2 and x1 ok
42. aPath = LEFT$(aCommand, x2)
43. aFile = MID$(aCommand, x2 + 1, x1 - x2)
44.  
45. aSource = aPath + aFile + ".LST"
46. PRINT "Source--->"; aSource
47. aDest = aPath + "combo.asm"
48. aReverse = aPath + "Rev0p0.asm"
49. PRINT "Dest----->"; aDest
50. aFind = "                       list"
51. '----------------------------------------
52. OPEN aSource FOR INPUT AS 1
53. OPEN aDest FOR OUTPUT AS 2
54. OPEN aReverse FOR OUTPUT AS 3
55. '----------------------------------------
56. DO UNTIL EOF(1)
57.     IF aNext = "" THEN
58.         LINE INPUT #1, aIn
59.     ELSE
60.         aIn = aNext
61.         aNext = ""
62.     END IF
63.     '----------------------------------------
64.     IF INSTR(aIn, ".asm") > 0 THEN
65.         PRINT #2, ";" + aIn
66.         aIn = LCASE$(aIn)
67.         x1 = INSTR(aIn, "include")
68.         x2 = INSTR(aIn, "syscon")
69.         aIn = MID$(aIn, x1) 'get include line less line number
70.         IF (x1 > 0) AND (x2 = 0) THEN
71.             PRINT #3, "        " + aIn
72.             PRINT aIn
73.         ELSEIF x2 > 0 THEN
74.             PRINT #3, ";        " + aIn
75.             PRINT aIn
76.         END IF
77.         aIn = ""
78.     END IF
79.     '----------------------------------------
80.     IF (INSTR(LCASE$(aIn), ".inc") > 0) AND (INSTR(aIn, "#") = 0) THEN
81.         PRINT #2, ";" + aIn
82.         PRINT #3, ";" + aIn
83.         aIn = ""
84.     END IF
85.     '----------------------------------------
86.     IF (INSTR(LCASE$(aIn), ";<delete") > 0) THEN
87.         aIn = ""
88.     END IF
89.     '----------------------------------------
90.     IF (INSTR(aIn, "Bank15On") > 0) THEN
91.         aIn = ""
92.     END IF
93.     '----------------------------------------
94.     IF NOT EOF(1) THEN
95.         LINE INPUT #1, aNext
96.         '        ELSE
97.         '            CLOSE 1
98.         '            print "Closed 1"
99.         '            END
100.     END IF
101.     '----------------------------------------
102.     IF (INSTR(aNext, "                            ") = 1) THEN
103.         aIn = aIn + MID$(aNext, 29) 'fix the line
104.         aNext = ""
105.     END IF
106.     '----------------------------------------
107.     IF (LEN(aIn) > 28) AND (INSTR(aIn, aFind) = 0) THEN 'only work with real lines
108.         xMacro = INSTR(MID$(aIn, 23, 5), "M")
109.         IF (INSTR(MID$(aIn, 23, 5), "0") = 1) OR (xMacro = 5) THEN 'macro and valid assembly lines only pass this point
110.             IF (xMacro = 5) THEN 'Macro Lines are processed here
111.                 IF LEN(aSaved) > 0 THEN
112.                     PRINT #2, ";" + aSaved
113.                     PRINT #3, aSaved
114.                     x1 = 1: aSpace = ""
115.                     DO WHILE MID$(aSaved, x1, 1) = " "
116.                         aSpace = aSpace + " "
117.                         x1 = x1 + 1
118.                     LOOP
119.                     IF LEN(aSpace) > 0 THEN
120.                         aSpace = aSpace + " " 'add 1 for comment character
121.                     END IF
122.                     IF MID$(aIn, 3, 1) <> " " THEN 'only display the macro if code generated in macro
123.                         GOSUB MakeMacroString
124.                         IF LEN(aMacro) > 0 THEN
125.                             PRINT #2, aSpace + aMacro
126.                         END IF
127.                     END IF
128.                     aSaved = ""
129.                 ELSE
130.                     IF (MID$(aIn, 3, 1) <> " ") THEN 'OR (INSTR(aIn, "local") > 0)
131.                         GOSUB MakeMacroString
132.                         IF LEN(aMacro) > 0 THEN
133.                             PRINT #2, aSpace + aMacro
134.                         END IF
135.                     END IF
136.                 END IF
137.             ELSE 'assembly lines are processed
138.                 IF INSTR(aIn, "]:") = 0 THEN
139.                     IF (LEN(aSaved) = 0) THEN
140.                         aSaved = MID$(aIn, 29)
141.                     ELSE
142.                         PRINT #2, aSaved
143.                         PRINT #3, aSaved
144.                         aSaved = MID$(aIn, 29)
145.                     END IF
146.                     IF INSTR(aIn, "This Is The End") > 0 THEN
147.                         PRINT #2, aSaved
148.                         PRINT #3, aSaved
149.                         PRINT aSaved
150.                         EXIT DO
151.                     END IF
152.                 END IF
153.                 ';----------------------------------------------Process Local Macro's if any
154.             END IF
155.         END IF
156.     END IF
157. LOOP
158. CLOSE #1
159. CLOSE #2
160. CLOSE #3
161. aFile1 = aPath + aFile1 + "0p0.ASM"
162.  
163. PRINT aSource
164. PRINT aDest
165. PRINT aFile1
166. 'KILL aFile1
167. 'NAME aDest AS aFile1
168. '        system
169. END
170. END
171. '----------------------------------------------Subroutines
172. MakeMacroString:
173. aMacro = MID$(aIn, 29)
174. x1 = 1
175. DO UNTIL MID$(aMacro, x1, 1) <> " "
176.     x1 = x1 + 1
177. LOOP
178. aMacro = "  " + MID$(aMacro, x1)
179. '----------------------------------------------Process temp labels
180. IF INSTR(aMacro, "local   Mac_Temp") > 0 THEN
181.     aMacro = "" 'delete local macros
182.     xTempLabel = xTempLabel + 1
183.     aTempLabel = "_" + MID$(STR$(xTempLabel), 2)
184. ELSEIF INSTR(aMacro, "Mac_Temp") > 0 THEN
185.     x1 = INSTR(aMacro, "Mac_Temp")
186.     x2 = INSTR(aMacro, "Mac_Temp:")
187.     x2a = INSTR(aMacro, ":")
188.     x3 = INSTR(aMacro, "_")
189.     x4 = INSTR(aMacro, "goto    Mac_Temp")
190.     x5 = INSTR(aMacro, "bra     Mac_Temp")
191.     x6 = INSTR(aMacro, "bnz     Mac_Temp")
192.     x7 = INSTR(aMacro, "set     Mac_Temp")
193.     IF x3 < x1 THEN 'strip any label present and add new one
194.         aMacro = LEFT$(aMacro, x3 - 1) + aTempLabel + MID$(aMacro, x1)
195.     ELSEIF x2 > 0 THEN 'install new label on numberless labels
196.         aMacro = LEFT$(aMacro, x2 - 1) + aTempLabel + MID$(aMacro, x2)
197.     ELSEIF (x4 > 0) OR (x5 > 0) OR (x6 > 0) OR (x7 > 0) THEN 'INstall lable on forward defined labels
198.         aMacro = LEFT$(aMacro, x1 - 1) + aTempLabel + MID$(aMacro, x1)
199.     ELSEIF (x2 = 0) AND (x2a > 0) THEN
200.         aMacro = aTempLabel + LEFT$(aMacro, x2a)
201.         x8 = INSTR(aMacro, "  ")
202.         IF x8 > 0 THEN
203.             aMacro = LEFT$(aMacro, x8 - 1) + MID$(aMacro, x8 + 2)
204.         END IF
205.     END IF
206. END IF
207.  
208. RETURN
209. '-----------------------------------------------------------------------------------------------

The only thing I did here was change the aCommand to: aCommand = ".\J13Gps4G" as I'd copied the file into my QB64 directory and ran it. 

Attached are the output files, which clock in at almost 16,000 lines.


EDIT:
When it runs, it mentions an "include "c:\...\macros_j13_c.asm"   <--- is this an included file which it needs to translate as well?  If so, the issue may be in the code that handles processing it somehow, as I don't have such a file/directory on my PC for it to find and glitch out on.

[hotlnc]

Noted on changing the path.

Look at the output file REV0p0.  It terminated at the same spot mine did -- about 10K lines.

[hotlnc]

The program doesn't crash or anything -- it just stops inputting/outputting (can't tell which.)

[hotlnc]

The file you pointed out "include "c:\...\macros_j13_c.asm"" is a file that was a macro file that was included and is no longer needed.  That line is commented out for the output file.

[SMcNeill]

It's processing to the end of the file...

Here's the end of combo.asm:

Code: QB64: [Select]

1. ;----------------------------------------
2.                 ;       11111111        Default
3. CONFIG2H:       equ     11111111b       ;CONFIGURATION REGISTER 2 HIGH (BYTE ADDRESS 300003h)
4.                 ;       UUUU....-Unimplemented: Program the corresponding Flash Configuration bit to ‘1’
5.                 ;           1111-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:32,768
6.                 ;           1110-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:16,384   65.6 seconds WDT
7.                 ;           1101-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:8,192    Here causes 32.7 seconds WDT trigger
8.                 ;           1100-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:4,096
9.                 ;           1011-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:2,048
10.                 ;           1010-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:1,024
11.                 ;           1001-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:512
12.                 ;           1000-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:256

And the end of Rev0p0.asm is exactly the same:

Code: QB64: [Select]

1. ;----------------------------------------
2.                 ;       11111111        Default
3. CONFIG2H:       equ     11111111b       ;CONFIGURATION REGISTER 2 HIGH (BYTE ADDRESS 300003h)
4.                 ;       UUUU....-Unimplemented: Program the corresponding Flash Configuration bit to ‘1’
5.                 ;           1111-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:32,768
6.                 ;           1110-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:16,384   65.6 seconds WDT
7.                 ;           1101-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:8,192    Here causes 32.7 seconds WDT trigger
8.                 ;           1100-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:4,096
9.                 ;           1011-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:2,048
10.                 ;           1010-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:1,024
11.                 ;           1001-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:512
12.                 ;           1000-WDTPS<3:0>: Watchdog Timer Postscale Select bits -- 1:256

There's something in the middle which isn't being processed 100%, it seems.

[SMcNeill]

Using WinMerge (which lets us compare the differences in 2 files), it's easy to spot some of the items that aren't the same:

combo.ASM:

Code: QB64: [Select]

1. SevenDecade_Ascii2Binary:       ;this is stored in accumulatorH:L
2. ;        Load    TablePointer,Table_7Decades     ;setup to point in decades table
3.            Ac set 0        ;make ACCESS Bank since already bank 0
4.            movlw   upper Table_7Decades
5.            movwf   TBLPTRU,Ac
6.            Ac set 0        ;make ACCESS Bank since already bank 0
7.            movlw   high Table_7Decades
8.            movwf   TBLPTRH,Ac
9.            Ac set 0        ;make ACCESS Bank since already bank 0
10.            movlw   low Table_7Decades
11.            movwf   TBLPTRL,Ac
12. ;        move    DivisorL,7              ;set up outer loop counter
13.            Ac set 1
14.            movlw   7
15.            movwf   DivisorL,Ac
16.         bra     Asc2Bin_0       ;and use common
17. ;----------------------------------------

Rev0p0.ASM:

Code: QB64: [Select]

1. SevenDecade_Ascii2Binary:       ;this is stored in accumulatorH:L
2.         Load    TablePointer,Table_7Decades     ;setup to point in decades table
3.         move    DivisorL,7              ;set up outer loop counter
4.         bra     Asc2Bin_0       ;and use common
5. ;----------------------------------------

There's several lines there missing, and the file is riddled with these deletions, which is what's dropping the length down from ~16000 lines to ~9000.   

[hotlnc]

The purpose of this program is to flatten an listing file that was generated by a macro assembler.  The macros have been expanded and the program is very hard to read.  The Combo program removes the macro lines and just shows native code. This could can then be compiled with an assembler.

Using your example, after running COMBO.BAS, it would look like this:

Code: QB64: [Select]

1. SevenDecade_Ascii2Binary:       ;this is stored in accumulatorH:L
2. ;        Load    TablePointer,Table_7Decades     ;setup to point in decades table
3.            Ac set 0        ;make ACCESS Bank since already bank 0
4.            movlw   upper Table_7Decades
5.            movwf   TBLPTRU,Ac
6.            Ac set 0        ;make ACCESS Bank since already bank 0
7.            movlw   high Table_7Decades
8.            movwf   TBLPTRH,Ac
9.            Ac set 0        ;make ACCESS Bank since already bank 0
10.            movlw   low Table_7Decades
11.            movwf   TBLPTRL,Ac
12. ;        move    DivisorL,7              ;set up outer loop counter
13.            Ac set 1
14.            movlw   7
15.            movwf   DivisorL,Ac
16.         bra     Asc2Bin_0       ;and use common
17.  

[hotlnc]

The REV0p0 file uses the macro file and is not a flattened file. It simply reverses the listing process and regurgitates the pre-combo file.

Look at the file "J13Gps4G0p0.ASM"

It terminates too early. There is more in the LST file to process, but nothing comes out.

[FellippeHeitor]

How do you make those "code" blocks?

There's a button with a # when you're writing a post. You can also manually enter [ code] and [/ code] (without the spaces) and whatever you type in between those tags will be put in a codebox.

Welcome to the forum, btw.