Author Topic: Tutorial: Turning QB64 interpreters into compilers  (Read 3507 times)

0 Members and 1 Guest are viewing this topic.

Offline Dav

  • Forum Resident
  • Posts: 792
    • View Profile
Tutorial: Turning QB64 interpreters into compilers
« on: February 01, 2021, 10:14:39 pm »
Several members here have made excellent interpreters in QB64 that run BAS code.  I ported one of mine to QB64, and wanted to take it further and make it turn BAS code in standalone EXE's instead.  Here's a tutorial on how I did mine.  It's easier to just go through the motions than just explain, so this post will go through the steps of turning a small interpreter into a EXE producing compiler. 

This tutorial will make a mini.exe compiler.  It should be able to turn the demo.bas code into a standalone demo.exe program. Please note - this is not a true compiler, but more like a 'bytecode' one.  The EXE's produced are merely a special interpreter with source coded binded to it - Like RapidQ does, and some other basic 'compilers out there.  I've attached all the needed source files to this post at the bottom for easier downloading.

STEP #1) You are going to need the MarkExeSize tool I posted here, and it's attached.  Compile that to EXE.  The interpreter and compiler we make here will need to be marked by it.  You can read what that does on the MarkExeSize page.

STEP #2)  Compile the sample interpreter.bas to EXE.  This is just an example interpreter.  The main thing is that it's made to open itself up when run, and load source code attached to itself, instead of loading an external BAS file.  Think of this as the runtime file.  But don't attach any BAS code to it yet, just compile it for now.  You will have to adapt your interpreter to load code this way too.

interpreter.bas
Code: QB64: [Select]
  1. 'Mini Interpreter runtime.
  2. 'A compiled EXE of this runs BAS code attached to it.
  3.  
  4. DIM Code$(100) 'space for 100 lines
  5.  
  6. '==========================================================
  7. place = INSTR(1, INPUT$(200, 1), "This program can't")
  8. IF place = 0 THEN
  9.     CLOSE: END
  10.     SEEK 1, place + 35: ExeSize& = CVL(INPUT$(4, 1))
  11. '==========================================================
  12.  
  13. 'Make sure something is attached to exe...
  14. IF ExeSize& + 1 > LOF(1) THEN END
  15.  
  16. SEEK 1, ExeSize& + 1
  17.  
  18. Lines = 1
  19.     LINE INPUT #1, c$
  20.     Code$(Lines) = c$
  21.     Lines = Lines + 1
  22.  
  23.  
  24. FOR t = 1 TO Lines
  25.     ExecuteLine Code$(t)
  26.  
  27. SUB ExecuteLine (cmd$)
  28.     cmd$ = LTRIM$(RTRIM$(cmd$))
  29.     IF LEFT$(cmd$, 1) = "'" THEN EXIT SUB
  30.     IF UCASE$(LEFT$(cmd$, 3)) = "REM" THEN EXIT SUB
  31.     IF UCASE$(LEFT$(cmd$, 5)) = "SLEEP" THEN SLEEP
  32.     IF UCASE$(cmd$) = "BEEP" THEN BEEP
  33.     IF UCASE$(LEFT$(cmd$, 6)) = "COLOR " THEN
  34.         COLOR VAL(RIGHT$(cmd$, LEN(cmd$) - 6))
  35.     END IF
  36.     IF UCASE$(cmd$) = "PRINT" THEN PRINT
  37.     IF UCASE$(LEFT$(cmd$, 7)) = "PRINT " + CHR$(34) THEN
  38.         PRINT MID$(cmd$, 8, LEN(cmd$) - 8)
  39.     END IF
  40.     IF UCASE$(LEFT$(cmd$, 3)) = "CLS" THEN CLS
  41.     IF UCASE$(LEFT$(cmd$, 3)) = "END" THEN END
  42.  

STEP #3) Compile the compiler.bas to EXE.  This little programs whole job is to combine the interpreter+source code together.  But - It will have the interpreter runtime attached to it eventually, like the interpreter has code attached to it.  We will attach that later.  For now just compile it...

compiler.bas
Code: QB64: [Select]
  1. 'Mini Compiler example
  2.  
  3. PRINT "A Mini .BAS Compiler"
  4. PRINT "Compile .BAS to .EXE"
  5. INPUT "BAS to open ->", in$: IF in$ = "" THEN END
  6. INPUT "EXE to make ->", out$: IF out$ = "" THEN END
  7.  
  8. 'First see if this EXE is marked...
  9. place = INSTR(1, INPUT$(200, 1), "This program can't")
  10. IF place = 0 THEN CLOSE: END
  11.  
  12. 'Grab EXE size info
  13. SEEK 1, place + 35: ExeSize& = CVL(INPUT$(4, 1))
  14. 'Make sure data attached...
  15. IF ExeSize& + 1 > LOF(1) THEN END
  16.  
  17. 'Jump to data
  18. SEEK 1, ExeSize& + 1
  19.  
  20. 'Extract data, make EXE file...
  21. outdata$ = INPUT$(LOF(1) - ExeSize&, 1)
  22. PRINT #2, outdata$;: outdata$ = ""
  23.  
  24. 'Add/attach BAS code to EXE
  25. outdata$ = INPUT$(LOF(3), 3)
  26. PRINT #2, outdata$;
  27.  
  28.  
  29. PRINT "Made "; out$
  30.  
  31.  

OPTIONAL STEP:   At this point you could run UPX on those EXE's to reduce their size down to about 500k.  You will have to download UPX from off the internet.  I use it a lot.  Works well on QB64 generated EXE's.  Make sure if you do this step, that you do it right here - BEFORE using MarkExeSize on them.

STEP #4) Now use the MarkExeSize tool on both the interpreter.exe and compiler.exe EXE's.  It saves their EXE size in the EXE's.   IMPORTANT: This is a needed step.  Without it, the EXE's won't know how to open a file attached to them.

STEP #5)  Now it's time to make the mini.exe compiler program.   Drop to a command prompt, into the folder where the new EXE's are, and combine both the compiler.exe+interpreter.exe files like this, making a new file called mini.exe:

copy /b compiler.exe+interpreter.exe mini.exe

If all went well, You just made a new exe file called mini.exe. It's the whole compiler, that contains the interpreter runtime too.  Run mini.exe, and you can now compile the demo.bas.  It will generate a demo.exe out of it.   The interpreter.exe & compiler.exe are no longer needed - mini.exe is the only thing needed to make the EXE files from BAS code.

demo.bas
Code: QB64: [Select]
  1. REM Sample program
  2. PRINT "Hit any key to clear..."
  3. PRINT "Cleared!"
  4.  

Final comments:  The example here is just a simple interpreter, just to show you how to do yours.  Be aware that unless you encode/decode your source code on the interpreter, people will be able to open up your EXE and see the source code, so I would put in an encoding/decoding method in your interpreter.

Try building this sample first, and you will see how easy it is to turn your interpreter into a byte-code compiler using QB64.  Start your own programming language.

Have fun!

 - Dav
« Last Edit: February 01, 2021, 10:19:48 pm by Dav »

Offline STxAxTIC

  • Library Staff
  • Forum Resident
  • Posts: 1091
  • he lives
    • View Profile
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #1 on: February 01, 2021, 10:23:48 pm »
Dav this worked awesomely!! Thank you for promptly whipping this up. I will be studying it well...

  [ You are not allowed to view this attachment ]  
You're not done when it works, you're done when it's right.

Offline Dav

  • Forum Resident
  • Posts: 792
    • View Profile
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #2 on: February 02, 2021, 06:22:38 am »
My pleasure.  If you hadn't of replied in the game jam thread that you wanted to see it, I probably wouldn't had the motivation to post it.

(wow...you sure did that in lightning speed!  9 minutes from the post)

- Dav
« Last Edit: February 02, 2021, 06:24:13 am by Dav »

FellippeHeitor

  • Guest
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #3 on: February 02, 2021, 06:46:34 am »
That's a really clever solution, Dav! I believe AutoHotKey does the same when it generates binaries. Kudos!

Offline Dav

  • Forum Resident
  • Posts: 792
    • View Profile
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #4 on: February 02, 2021, 09:29:43 am »
Thanks, @FellippeHeitor!  This is how my old interpreter/compiler made in QB45 20 years ago worked.  Had to use DOS interrupts back then.  I forgot about AutoHotKey!  That's a nice piece of software.

By the way, just for my own curiosity and experimentation here at home , I tried adapting your interpreter code to compile code like this a while back, and it worked well.  I hope you continue with the interpreter.  It's VERY good, and your code is always so easy to understand.  Kudos to you!

- Dav

FellippeHeitor

  • Guest
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #5 on: February 02, 2021, 09:33:12 am »
Thank you very much, Dav! It's a fun toy project to maintain.

But you just wait and see when L-Basic comes out... https://github.com/flukiluke/L-BASIC
« Last Edit: February 02, 2021, 09:36:28 am by FellippeHeitor »

Offline Dav

  • Forum Resident
  • Posts: 792
    • View Profile
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #6 on: February 02, 2021, 09:38:48 am »
But you just wait and see when L-Basic comes out... https://github.com/flukiluke/L-BASIC

Wow - that sounds exciting!  Thanks for the link.  I didn't know it was on github.  I tried the '65 one on the forum.

With luke and Ashish at the helm, I'm sure it gonna be great!

- Dav

Offline Pete

  • Forum Resident
  • Posts: 2361
  • Cuz I sez so, varmint!
    • View Profile
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #7 on: February 02, 2021, 12:45:57 pm »
Thank you very much, Dav! It's a fun toy project to maintain.

But you just wait and see when L-Basic comes out... https://github.com/flukiluke/L-BASIC

Wait. Wasn't L-Basic 5 builds behind Q-Basic?

Pete
Want to learn how to write code on cave walls? https://www.tapatalk.com/groups/qbasic/qbasic-f1/

Offline luke

  • Administrator
  • Seasoned Forum Regular
  • Posts: 324
    • View Profile
Re: Tutorial: Turning QB64 interpreters into compilers
« Reply #8 on: February 02, 2021, 04:45:06 pm »
If Mr Gates, William can get away with calling it GW-BASIC, I'm gonna stick by L-BASIC.

Any reference to the author's name is completely coincidental, of course. The L stands for whatever you want it to stand for.