Embedding files in programs (FileToDATA convertor)

[luke]

In general the linker will leave out object files that are not providing any required symbols. This basically means that libqb needs to be split into separately-compilable files, while trying to keep the inter-file dependencies down. I believe this is actually somewhat close to the structure of the original BCOM45.LIB, with its B$ functions.

I have tried to separate out the pieces of libqb and it proves to be non-trivial. I suppose the best way to go about it is as a gradual process.

I'm bad with naming things, but how about this:

Code: QB64: [Select]

1. $attach: 'image.png'
2. img& = _LOADIMAGE("ATTACH:image.png")

With the same syntax working for OPEN and the other file functions. This fits the trend we have with SCRN: and COM:

[RhoSigma]

Code: QB64: [Select]

1. $attach: 'image.png'
2. img& = _LOADIMAGE("ATTACH:image.png")

With the same syntax working for OPEN and the other file functions. This fits the trend we have with SCRN: and COM:

Well, this appoch seems similar to resource table extraction, as many professional programs use to embed its toolbar icons. Maybe its possible (instead of attaching/emedding files via DATAs or C-arrays) to simply get better/easier support to access the resouce table in QB64 with some new commands:

eg:

Code: QB64: [Select]

1. '$INCRESOURCE: 'imageA.png'
2. '$INCRESOURCE: 'imageB.png'
3. '$INCRESOURCE: 'imageC.png'
4.  
5. 'open resource by name
6. OPEN "RSRCN:imageB.png" AS #1 'shortcut open syntax, as readonly is implied
7.  
8. 'or open resource by index
9. OPEN "RSRCI:3" AS #1 'should open imageC.png then
10.  
11. ....
12. ....
13.  

Same usage in _LOADIMAGE/_LOADFONT etc., however, I'm not so familiar with resource table specs, is it possible to place font files or simple text files there too (such as INI/CFG or even TXT)?

[SMcNeill]

I don't know if a number reference is necessarily a good idea.  Imagine your code used in a set of $INCLUDE files... Is ImageB.png the 2nd file, or the 22nd?

Personally, I like an idea similar to this:

$RESOURCE:ImageB.PNG, 123456
      First value is the resource name which we want to reference it by.
      The number value sets a hard limit for the max size of the resource, which we can then manually
      copy/paste here, or get the IDE to load from file with a simple Load Resource option in the menu.
      NO ERROR CHECKING/SYNTAX CORRECTION OCCURS HERE.
      It'd be nice if this was a collapsible mid section, for ease of code navigation...
$END RESOURCES

Then you can use "$RESOURCE:name" for any file operation.  (Even OUTPUT/PUT could be supported as long as space was reserved first.)l

handle = _LOADIMAGE("$RESOURCE:ImageB.PNG",32)
font = _LOADFONT("$RESOURCE:InternalFont2.TTF", 16, "MONOSPACE")
OPEN "$RESOURCE:OwnerData.txt" FOR OUTPUT AS #1

And such...

[Pete]

Why not just make an auto-detect function to assign space? If the size varies because the embedded element is changed, the exe would auto-detect it and assign space accordingly. Of course this means the $RESOURCE meta commands would have to be read first, wherever they occurred in the code, to establish the space values, prior to executing the statements statements. Possible? I don't work with C/C++ so memory allocation, pointers, etc. are foreign to me.

Pete

[RhoSigma]

Whatever the developers do, assuming they take the discussed ideas, should be easy to handle and fit into the existing concepts of the QB64 language.

[Dav]

Great that resource files are being considered!  I always thought the way rapidq handles resource files is simple but useful.

'Define the resource
 $RESOURCE BMP1 AS "FUNNY.BMP"

'Use the resource like this
OPEN BMP1 FOR BINARY AS #1

[Pete]

@ Dav: I'm not seeing the usefulness in the rename portion here. I believe I tried RapidQ once, but it was object oriented so I dropped it like a rock. Anyway, do you see any advantage in BMP1 representing "FUNNY.BMP" because I don't. I do see that using $RESOURCE to import a resource into a program is valuable.

On a side note, how is everything after the storms? I hope you only had some minor problems, if any, but that was a lot of water over a long period of time.

Also, please have a look at this thread: https://www.qb64.org/forum/index.php?topic=799.msg100082#new I know you have written some pretty large projects, like your IDE.

Pete

[Dav]

Hi Pete!  Well, that's a better idea there, bypass making a new variable and just use the original filename after declaring it a resource.  I mostly wanted to point out how small and easy it was in rapidq to add/use resource files.  As for that language, I hear ya, I didn't warm up to it that much either.  But it had some ok networking capabilities which I used to make a few projects.

We got through the storms ok.  Lost power about a week, had some tree debris to saw up. Nothing big fell on the house. The creek flooded and came up to about 6 feet from the house (witnesses said, I didn't see it), but water never got into the house.  We got through it a lot better than others did.  Taught me some lessons on being better prepared.  Thank you for asking. 

I was reading that "big one" thread.  80k lines is a big program!  I think my IDE is about 20k.  I remember having some larger programs over the years, due to embedding data in the code probably.  Will have to dig into some old HD's and take a peek.

- Dav

[RhoSigma]

Hi all,
just updated the MakeDATA.bas codebox in the initial post with a new version, which gives a really nice speedup in the created writeback function compared to the old version. This is especially good for somewhat larger files. Tested it with the cyberbit.ttf font file, the writeback call finishes more than 300 times faster now (before approx. half an hour, now 5 seconds).

It's done just by taking out the temporary string operations, which QB64 does under the hood when adding more chars to a variable length string. If you do so in a loop (eg. reading DATAs and concatenate them into one string), then the complexity raises very quickly to an unthinkable amount, causing this immense slowdown.

Now i define the required length of the string before entering the loop with SPACE$(n), and then just replace chars in it using MID$ with an ever increasing char offset.

[RhoSigma]

EDIT: Obsolete informations deleted.

[RhoSigma]

EDIT: Obsolete informations deleted.

[RhoSigma]

EDIT: Obsolete informations deleted.

[RhoSigma]

Major update regarding the recursion issue (https://qb64forum.alephc.xyz/index.php?topic=4209), also the code files generated by these tools are OPTION _EXPLICIT friendly now and the code generated by MakeCARR should now work under Linux and MacOS.

Please take the new updated code from the inital post here and make sure to carfully read the entire post.

@Qwerkey, the code must be updated in the "Utilities" board too. When doing so, please also remove the lzwpacker.bm attachement there, cause it's also outdated since a long time, rather mention my Libraries Collection (https://qb64forum.alephc.xyz/index.php?topic=809) as official place to get the latest version of the required Lzw library.

[bplus]

I'm glad this got bumped up again, I need to do a real study of this, run some tests. 2 years ago I guess I missed it or something, or maybe now the time is ripe for me to review.