Text Only | Text with Attachments

QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: SMcNeill on August 15, 2019, 08:06:23 pm

Title: Help in translating BAS to C and not QB64-C.
Post by: SMcNeill on August 15, 2019, 08:06:23 pm
Here's a simple little program I wrote, which uses the ZLIB library to do compression for us:

Code: QB64: [Select]
  2.     FUNCTION compressBound& (BYVAL len AS LONG)
  3.     FUNCTION compress2& (dest AS STRING, destLen AS LONG, source AS STRING, BYVAL sourceLen AS LONG, BYVAL level AS LONG)
  5.  
  7. FUNCTION Deflate$ (text$)
  8.     DIM FileSize AS LONG, CompSize AS LONG, return$
  9.     FileSize = LEN(text$)
  10.     CompSize = compressBound(FileSize)
  11.     return$ = SPACE$(CompSize)
  12.     Result = compress2(return$, CompSize, text$, FileSize, 9)
  13.     Deflate$ = LEFT$(return$, CompSize)

Simple enough.  Pass FUNCTION deflate a string, it passes you back a compressed string...

Here's my problem -- the c-output that we get from this code:
Code: C++: [Select]
  1. qbs* FUNC_DEFLATE(qbs*_FUNC_DEFLATE_STRING_TEXT){
  2. qbs *tqbs;
  3. ptrszint tmp_long;
  4. int32 tmp_fileno;
  5. uint32 qbs_tmp_base=qbs_tmp_list_nexti;
  6. uint8 *tmp_mem_static_pointer=mem_static_pointer;
  7. uint32 tmp_cmem_sp=cmem_sp;
  8. qbs *_FUNC_DEFLATE_STRING_DEFLATE=NULL;
  9. if (!_FUNC_DEFLATE_STRING_DEFLATE)_FUNC_DEFLATE_STRING_DEFLATE=qbs_new(0,0);
  10. qbs*oldstr1=NULL;
  11. if(_FUNC_DEFLATE_STRING_TEXT->tmp||_FUNC_DEFLATE_STRING_TEXT->fixed||_FUNC_DEFLATE_STRING_TEXT->readonly){
  12. oldstr1=_FUNC_DEFLATE_STRING_TEXT;
  13. if (oldstr1->cmem_descriptor){
  14. _FUNC_DEFLATE_STRING_TEXT=qbs_new_cmem(oldstr1->len,0);
  15. }else{
  16. _FUNC_DEFLATE_STRING_TEXT=qbs_new(oldstr1->len,0);
  17. }
  18. memcpy(_FUNC_DEFLATE_STRING_TEXT->chr,oldstr1->chr,oldstr1->len);
  19. }
  20. int32 *_FUNC_DEFLATE_LONG_FILESIZE=NULL;
  21. if(_FUNC_DEFLATE_LONG_FILESIZE==NULL){
  22. _FUNC_DEFLATE_LONG_FILESIZE=(int32*)mem_static_malloc(4);
  23. *_FUNC_DEFLATE_LONG_FILESIZE=0;
  24. }
  25. int32 *_FUNC_DEFLATE_LONG_COMPSIZE=NULL;
  26. if(_FUNC_DEFLATE_LONG_COMPSIZE==NULL){
  27. _FUNC_DEFLATE_LONG_COMPSIZE=(int32*)mem_static_malloc(4);
  28. *_FUNC_DEFLATE_LONG_COMPSIZE=0;
  29. }
  30. qbs *_FUNC_DEFLATE_STRING_RETURN=NULL;
  31. if (!_FUNC_DEFLATE_STRING_RETURN)_FUNC_DEFLATE_STRING_RETURN=qbs_new(0,0);
  32. byte_element_struct *byte_element_2=NULL;
  33. if (!byte_element_2){
  34. if ((mem_static_pointer+=12)<mem_static_limit) byte_element_2=(byte_element_struct*)(mem_static_pointer-12); else byte_element_2=(byte_element_struct*)mem_static_malloc(12);
  35. }
  36. float *_FUNC_DEFLATE_SINGLE_RESULT=NULL;
  37. if(_FUNC_DEFLATE_SINGLE_RESULT==NULL){
  38. _FUNC_DEFLATE_SINGLE_RESULT=(float*)mem_static_malloc(4);
  39. *_FUNC_DEFLATE_SINGLE_RESULT=0;
  40. }
  41.  
  42. mem_lock *sf_mem_lock;
  43. new_mem_lock();
  44. sf_mem_lock=mem_lock_tmp;
  45. sf_mem_lock->type=3;
  46. if (new_error) goto exit_subfunc;
  47. *_FUNC_DEFLATE_LONG_FILESIZE=_FUNC_DEFLATE_STRING_TEXT->len;
  48. qbs_cleanup(qbs_tmp_base,0);
  49. *_FUNC_DEFLATE_LONG_COMPSIZE=(  int32  )compressBound(*_FUNC_DEFLATE_LONG_FILESIZE);
  50. qbs_set(_FUNC_DEFLATE_STRING_RETURN,func_space(*_FUNC_DEFLATE_LONG_COMPSIZE));
  51. qbs_cleanup(qbs_tmp_base,0);
  52. *_FUNC_DEFLATE_SINGLE_RESULT=(  int32  )compress2((char*)(_FUNC_DEFLATE_STRING_RETURN)->chr,_FUNC_DEFLATE_LONG_COMPSIZE,(char*)(_FUNC_DEFLATE_STRING_TEXT)->chr,*_FUNC_DEFLATE_LONG_FILESIZE, 9 );
  53. qbs_cleanup(qbs_tmp_base,0);
  54. qbs_set(_FUNC_DEFLATE_STRING_DEFLATE,qbs_left(_FUNC_DEFLATE_STRING_RETURN,*_FUNC_DEFLATE_LONG_COMPSIZE));
  55. qbs_cleanup(qbs_tmp_base,0);
  56. exit_subfunc:;
  57. free_mem_lock(sf_mem_lock);
  58. if(oldstr1){
  59. if(oldstr1->fixed)qbs_set(oldstr1,_FUNC_DEFLATE_STRING_TEXT);
  60. qbs_free(_FUNC_DEFLATE_STRING_TEXT);
  61. }
  62. qbs_free(_FUNC_DEFLATE_STRING_RETURN);
  63. if ((tmp_mem_static_pointer>=mem_static)&&(tmp_mem_static_pointer<=mem_static_limit)) mem_static_pointer=tmp_mem_static_pointer; else mem_static_pointer=mem_static;
  64. cmem_sp=tmp_cmem_sp;
  65. qbs_maketmp(_FUNC_DEFLATE_STRING_DEFLATE);return _FUNC_DEFLATE_STRING_DEFLATE;
  66. }
  67.  

Our 6 lines of code end up turning into 66 lines of mind-boggling gibberish!! 


So my question here is:  Does anybody know a nice, clean way to turn that small function into C-code?

QB64 has always boggled my poor brain with how it handles strings.  We don't use strings like they have in C... We have our own custom little type called *qbs (quick basic string), complete with data structures and all sorts of gunk which I get lost in.  Converting QB64 numbers to C numbers doesn't give me any issue, but sorting out how the hell to make a QB64 string work with the things I might want it to in C just leaves me frustrated and scratching my head, as I drool incessantly while trying out one failure after another, looking for the right thing to stick and work!

And why do I need such a SUB, you ask?

I've been playing around with my Repo, and I've now got ZLIB merged into QB64 with it an available library which we can pull from.  You don't need any sort of dll file in your folder for it to work anymore (like with the SaveImage library) -- it's already in QB64 and ready for us to make use of it.  (Only my personal version at the moment, as I'm still testing things out.)

Once I can get a simple little function wrote up to _DEFLATE and _ENFLATE, I can add those to QB64 itself and we'll have compression tools which we can use forever more with the language.

What compress2 is actually looking for is:

Code: C++: [Select]
  1. int compress2(Bytef * dest, uLongf * destLen, const Bytef * source, uLong sourceLen, int level);

Which is the following 5 things:
Offset to the compressed data (or string in this case)
offset to the length of the compressed data
Offset to the raw data which we want to compress
the length of the raw data which we're compressing
And the value for the compression level (9 is max compression)

Any ideas on how to turn this into working C, which I can plug into libqb.cpp for use:

Code: QB64: [Select]
  1. FUNCTION Deflate$ (text$)
  2.     DIM FileSize AS LONG, CompSize AS LONG, return$
  3.     FileSize = LEN(text$)
  4.     CompSize = compressBound(FileSize)
  5.     return$ = SPACE$(CompSize)
  6.     Result = compress2(return$, CompSize, text$, FileSize, 9)
  7.     Deflate$ = LEFT$(return$, CompSize)

There's got to be something better than the auto-translated gook which QB64 kicks out for it!

Any help would be appreciated. 
Title: Re: Help in translating BAS to C and not QB64-C.
Post by: SMcNeill on August 16, 2019, 02:58:02 am
I'm thinking I've got it working now, with the translated code being:

Code: C++: [Select]
  1. qbs *func__deflate(qbs *text){
  2.     uLong filesize = (uLong)text->len;                                    //length of the text
  3.     uLong compsize = compressBound(filesize);                             //size of the initial buffer which we need to compress to
  4.     unsigned char *dest;                                                  //a pointer to a series of characters
  5.     qbs *ret;                                                             //and a qbs string which we can return from
  6.     dest = (unsigned char *)malloc(compsize);                             //allocate memory for the buffer
  7.     int32 result = compress(dest, &compsize, text->chr, filesize);        //and do the compression
  8.     ret = qbs_new(compsize,1);                                            //make the new qbs string
  9.     memcpy(ret->chr, dest, compsize);                                     //and copy the compressed file over into it.
  10.     free(dest);                                                           //free the buffer
  11.     return ret;                                                           //return the string
  12. }

Unless somebody sees a glaring issue with the above, (such as a memory leak or other brain damaged gaff), the code appears to be working as it should.  My version of QB64 now currently has a _DEFLATE$ command in it, where we send it one text$ and it'll compress it and send us back another, and from what I can tell, it's working without any issues.

Now I just need an _INFLATE$ function, and I'll push this into my personal repo to finish sorting the bugs out, and folks can feel free to test the commands if they want to, at that point. 


At the moment, I can write simple little code like this in my version of qb64:

Code: [Select]
text$ = "I like cheese."
FOR i = 1 TO 10: text$ = text$ + text$: NEXT
de$ = _DEFLATE$(text$)
PRINT LEN(de$), LEN(text$)
PRINT de$
The results confirm that we're compressing a file down from 14,336 bytes to only 69 bytes, and by calling the zlib library with DECLARE LIBRARY directly, I can expand that compressed file back into a normal one, without seeming to generate any glitches. 

Soon(tm), we might have compression utilities added into QB64 -- and those are always nice!  ;D
Title: Re: Help in translating BAS to C and not QB64-C.
Post by: RhoSigma on August 16, 2019, 09:45:16 am
Looks perfect to me Steve, and as it works the job is done. However the function could be made a bit more QB64ish by using the typical types we see everywhere in libqb.ccp, uLong should be equal to uint32. Also you can mix declarations and initialisations in C++, so you could put the dest and ret declarations right in front of their first initialisation, just as you did in the compress() call line:
Code: C++: [Select]
  1. qbs *func__deflate(qbs *text){
  2.     uint32 filesize = (uint32)text->len;                            //length of the text
  3.     uint32 compsize = compressBound(filesize);                      //size of the initial buffer which we need to compress to
  4.     unsigned char *dest = (unsigned char *)malloc(compsize);        //allocate memory for the buffer
  5.     int32 result = compress(dest, &compsize, text->chr, filesize);  //and do the compression
  6.     qbs *ret = qbs_new(compsize,1);                                 //make the new qbs string
  7.     memcpy(ret->chr, dest, compsize);                               //and copy the compressed file over into it.
  8.     free(dest);                                                     //free the buffer
  9.     return ret;                                                     //return the string
  10. }
  11.  
it's up to you, on the other hand - never change working code :)
Title: Re: Help in translating BAS to C and not QB64-C.
Post by: RhoSigma on August 16, 2019, 09:51:01 am
Another though, does zlib use affect current QB64 licensing in any way? I mean like that sound stuff, then don't forget to add a note in the respective files too.
Title: Re: Help in translating BAS to C and not QB64-C.
Post by: SMcNeill on August 16, 2019, 01:39:44 pm
uint32 filesize doesnt work without tossing errors.  Zlib actually seems to only like uLong, so I’m thinking it may be a resizable type, depending on the OS.  In a 64-bit OS, it’s likely 8 bytes, otherwise it’s 4...  since I’m working with a 64-bit program, the int32 tosses -fpermissive errors.

Z-lib license is the following; I don’t think it’ll affect us at all.  (After all, zlib used to come packaged automatically with the old SDL version.)

Quote
/* zlib.h -- interface of the 'zlib' general purpose compression library
  version 1.2.11, January 15th, 2017

  Copyright (C) 1995-2017 Jean-loup Gailly and Mark Adler

  This software is provided 'as-is', without any express or implied
  warranty.  In no event will the authors be held liable for any damages
  arising from the use of this software.

  Permission is granted to anyone to use this software for any purpose,
  including commercial applications, and to alter it and redistribute it
  freely, subject to the following restrictions:

  1. The origin of this software must not be misrepresented; you must not
     claim that you wrote the original software. If you use this software
     in a product, an acknowledgment in the product documentation would be
     appreciated but is not required.
  2. Altered source versions must be plainly marked as such, and must not be
     misrepresented as being the original software.
  3. This notice may not be removed or altered from any source distribution.

  Jean-loup Gailly        Mark Adler
  jloup@gzip.org          madler@alumni.caltech.edu

*/
Title: Re: Help in translating BAS to C and not QB64-C.
Post by: RhoSigma on August 16, 2019, 02:37:02 pm
Quote
uint32 filesize doesnt work without tossing errors.  Zlib actually seems to only like uLong, so I’m thinking it may be a resizable type, depending on the OS.  In a 64-bit OS, it’s likely 8 bytes, otherwise it’s 4...  since I’m working with a 64-bit program, the int32 tosses -fpermissive errors.

Makes sense, was only a guess, well then just follow my final note on that post - never change working code :)

The license is pretty permissive, think it's ok, nevertheless add it to the licence folder.
Text Only | Text with Attachments