Author Topic: MemToHex (and back)  (Read 5114 times)

0 Members and 1 Guest are viewing this topic.

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
MemToHex (and back)
« on: August 01, 2019, 08:18:07 am »
A set of little utilities which folks might want to add to their collection:

Code: QB64: [Select]
  1. SCREEN _NEWIMAGE(640, 480, 32)
  2. t = _NEWIMAGE(16, 16, 32)
  3. m = _MEMIMAGE(t)
  4. CIRCLE (8, 8), 7, -1
  5. PAINT (8, 8), -1
  6. _CLIPBOARD$ = MemImagetoDATA$(m, "t", 160) 'Here's where I got the DATA to paste below
  7.  
  8. RESTORE t 'remember to restore using the proper label you associated with the data
  9. t1 = DATAtoImage 'Make a new image using the DATA
  10. _PUTIMAGE (30, 30), t1
  11.  
  12.  
  13. 'Here's where I pasted the clipboard DATA from the memimage
  14.  
  15. t:
  16. DATA 16,16,4
  17. DATA 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  18. DATA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000
  19. DATA 0000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  20. DATA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  21. DATA 00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  22. DATA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000
  23. DATA 000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000000000000000000000000000
  24.  
  25. 'NOTICE:  We turn Checking OFF from this point down, as these are _MEM routines and can greatly benefit from
  26. '         the speed boost.  This means it's up to the USER/PROGRAMMER of these routines to be certain to NOT
  27. '         screw up calling them.  RESTORE proper data labels, if necessary.  Make certain that you're actually
  28. '         passing them valid mem blocks, or valid image blocks...
  29. '
  30. '         If not....
  31. '
  32. '         Meeehhhhh!   It's not *MY* program which'll need debugging later....
  33.  
  34. FUNCTION DATAtoImage&
  35.     'Requires a RESTORE label for the proper data BEFORE calling this routine
  36.     READ w, h, ps
  37.     SELECT CASE ps
  38.         CASE 1: ps = 256 '256 color screen
  39.         CASE 2: ps = 0 'text screen
  40.         CASE 4: ps = 32 '32-bit color screen
  41.     END SELECT
  42.     DATAtoImage& = _NEWIMAGE(w, h, ps)
  43.     DIM m AS _MEM
  44.     m = _MEMIMAGE(DATAtoImage&)
  45.     DO
  46.         READ hx$
  47.         FOR i = 1 TO LEN(hx$) STEP 2
  48.             h = VAL("&H" + MID$(hx$, i, 2))
  49.             _MEMPUT m, m.OFFSET + o, h
  50.             o = o + 1
  51.         NEXT
  52.         LOCATE 1, 1: PRINT o, m.SIZE
  53.     LOOP UNTIL o >= m.SIZE
  54.     _MEMFREE m
  55.  
  56.  
  57. FUNCTION MemImagetoDATA$ (m AS _MEM, label$, break)
  58.     s = ConvertOffset(m.SIZE) - 1
  59.     label$ = _TRIM$(label$)
  60.     IF label$ = "" THEN label$ = "generic_label_placeholder:"
  61.     IF RIGHT$(label$, 1) <> ":" THEN label$ = label$ + ":"
  62.     MemImagetoDATA$ = label$ + CHR$(10) + "DATA "
  63.     MemImagetoDATA$ = MemImagetoDATA$ + STR$(_WIDTH(m.IMAGE)) + ", "
  64.     MemImagetoDATA$ = MemImagetoDATA$ + STR$(_HEIGHT(m.IMAGE)) + ", "
  65.     MemImagetoDATA$ = MemImagetoDATA$ + STR$(_PIXELSIZE(m.IMAGE)) + CHR$(10) + "DATA "
  66.     FOR i = 0 TO s
  67.         _MEMGET m, m.OFFSET + i, b
  68.         h$ = HEX$(b)
  69.         IF LEN(h$) = 1 THEN h$ = "0" + h$
  70.         MemImagetoDATA$ = MemImagetoDATA$ + h$
  71.         IF i MOD break = break - 1 AND i < s THEN
  72.             MemImagetoDATA$ = MemImagetoDATA$ + CHR$(10) + "DATA "
  73.         END IF
  74.     NEXT
  75.  
  76. FUNCTION MemToHex$ (m AS _MEM)
  77.     s = ConvertOffset(m.SIZE) - 1
  78.     FOR i = 0 TO s
  79.         _MEMGET m, m.OFFSET + i, b
  80.         h$ = HEX$(b)
  81.         IF LEN(h$) = 1 THEN h$ = "0" + h$
  82.         MemToHex$ = MemToHex$ + h$
  83.     NEXT
  84.  
  85. SUB HexToMem (hx$, m AS _MEM)
  86.     DIM i AS _INTEGER64
  87.     FOR i = 1 TO LEN(hx$) STEP 2
  88.         h = VAL("&H" + MID$(hx$, i, 2))
  89.         _MEMPUT m, m.OFFSET + i \ 2, h
  90.     NEXT
  91.  
  92. FUNCTION ConvertOffset&& (value AS _OFFSET)
  93.     DIM m AS _MEM 'Define a memblock
  94.     m = _MEM(value) 'Point it to use value
  95.     $IF 64BIT THEN
  96.         'On 64 bit OSes, an OFFSET is 8 bytes in size.  We can put it directly into an Integer64
  97.         _MEMGET m, m.OFFSET, ConvertOffset&& 'Get the contents of the memblock and put the values there directly into ConvertOffset&&
  98.     $ELSE
  99.         'However, on 32 bit OSes, an OFFSET is only 4 bytes.  We need to put it into a LONG variable first
  100.         _MEMGET m, m.OFFSET, temp& 'Like this
  101.         ConvertOffset&& = temp& 'And then assign that long value to ConvertOffset&&
  102.     $END IF
  103.     _MEMFREE m 'Free the memblock

With these, we can convert ANY memblock into a sequence of HEX values (which makes them excellent to paste into another program as DATA statements), and then we can take those hex values and convert them back into a memblock.

This also has a couple of routines which are set up to specialize in taking memimages and converting them to DATA statements, and then back again for us, for ease of embedding and use in our programs.



Enjoy!
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline STxAxTIC

  • Library Staff
  • Forum Resident
  • Posts: 1091
  • he lives
    • View Profile
Re: MemToHex (and back)
« Reply #1 on: August 16, 2019, 09:52:55 pm »
Yello Steve,

So the Librarian told me that this code is a almost good candidate for the Toolbox - but we have to massage it to fit the criteria. It seems this post has a bag of tools in itself, but the thing being demo'd here doesn't use all of them, namely MemToHex$ and its reversing counterpart. If you have time, can you fragment this into smaller independently-robust chunks? They are free to exist as neighbors and refer to each other, but they should be untangled as tools if you don't mind.
« Last Edit: August 16, 2019, 09:54:00 pm by STxAxTIC »
You're not done when it works, you're done when it's right.

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: MemToHex (and back)
« Reply #2 on: August 17, 2019, 12:23:43 am »
Yello Steve,

So the Librarian told me that this code is a almost good candidate for the Toolbox - but we have to massage it to fit the criteria. It seems this post has a bag of tools in itself, but the thing being demo'd here doesn't use all of them, namely MemToHex$ and its reversing counterpart. If you have time, can you fragment this into smaller independently-robust chunks? They are free to exist as neighbors and refer to each other, but they should be untangled as tools if you don't mind.

MemToHex is easy to showcase:

Code: [Select]
DIM junk AS STRING * 12
DIM m AS _MEM: m = _MEM(junk)
junk = “Hello World!”
Hx$ = MemToHex(m)
PRINT Hx$

junk$ = “Overwritten?” ‘See where we overwrite that string?
HexToMem Hx$, m ‘restore it
PRINT junk$ ‘and now print it

FUNCTION MemToHex$ (m AS _MEM)
    DIM b AS _UNSIGNED _BYTE
    s = ConvertOffset(m.SIZE) - 1
    FOR i = 0 TO s
        _MEMGET m, m.OFFSET + i, b
        h$ = HEX$(b)
        IF LEN(h$) = 1 THEN h$ = "0" + h$
        MemToHex$ = MemToHex$ + h$
    NEXT
END FUNCTION
 
SUB HexToMem (hx$, m AS _MEM)
    DIM i AS _INTEGER64
    DIM h AS _UNSIGNED _BYTE
    FOR i = 1 TO LEN(hx$) STEP 2
        h = VAL("&H" + MID$(hx$, i, 2))
        _MEMPUT m, m.OFFSET + i \ 2, h
    NEXT
END SUB
 
FUNCTION ConvertOffset&& (value AS _OFFSET)
    DIM m AS _MEM 'Define a memblock
    m = _MEM(value) 'Point it to use value
    $IF 64BIT THEN
        'On 64 bit OSes, an OFFSET is 8 bytes in size.  We can put it directly into an Integer64
        _MEMGET m, m.OFFSET, ConvertOffset&& 'Get the contents of the memblock and put the values there directly into ConvertOffset&&
    $ELSE
        'However, on 32 bit OSes, an OFFSET is only 4 bytes.  We need to put it into a LONG variable first
        _MEMGET m, m.OFFSET, temp& 'Like this
        ConvertOffset&& = temp& 'And then assign that long value to ConvertOffset&&
    $END IF
    _MEMFREE m 'Free the memblock
END FUNCTION
$CHECKING:ON

The purpose of these is to let us convert a segment of memory into a series of hex values, to write into a DATA field (or some such purpose), without any issues.

After all, it’s hard to print a CRLF (carriage return line feed) sequence in a data statement, but there’s no worries when storing a DATA 0D0A.

The MemImageToData is just a specialized version of the routine which takes a _MEMIMAGE and converts it to Hex, while placing the DATA statement in front of it for us, for quick use pasting it into another program.  It basically lets us store image files inside a BAS file, without needing to _LOADIMAGE them externally.
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline Petr

  • Forum Resident
  • Posts: 1720
  • The best code is the DNA of the hops.
    • View Profile
Re: MemToHex (and back)
« Reply #3 on: August 17, 2019, 12:34:58 pm »
Hi Steve. I experimented a little. Your way of writing in hexadecimal values is great. But if you combine it with your other great text compression program, this image can be saved in 43 bytes (originally 256 bytes). It uses characters outside the visible area of the ascii table on the monitor, so we have our own QB64 compression image format, just insert it into a binary file :-) or, while keeping the idea, the compressed content can be saved to the source file as a hexadecimal compressed record. In our case for this image it is

0100230461000FFFFFFFFFFFF00000FFFFFFFFFFFFFFFFFFFFFFFF00000FFFFFFFFFF

The possibility of further compression is obvious in this chain, but it would probably be better to use the LZW method instead of the economical write method. Thank you for really many materials for next exploration this week!
« Last Edit: August 17, 2019, 12:42:41 pm by Petr »

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: MemToHex (and back)
« Reply #4 on: August 17, 2019, 02:11:40 pm »
Hi Steve. I experimented a little. Your way of writing in hexadecimal values is great. But if you combine it with your other great text compression program, this image can be saved in 43 bytes (originally 256 bytes). It uses characters outside the visible area of the ascii table on the monitor, so we have our own QB64 compression image format, just insert it into a binary file :-) or, while keeping the idea, the compressed content can be saved to the source file as a hexadecimal compressed record. In our case for this image it is

0100230461000FFFFFFFFFFFF00000FFFFFFFFFFFFFFFFFFFFFFFF00000FFFFFFFFFF

The possibility of further compression is obvious in this chain, but it would probably be better to use the LZW method instead of the economical write method. Thank you for really many materials for next exploration this week!

I’d already considered this possibility, which is why I’ve been working on adding compression commands into the language for us.  Grab a copy of my repo and give a test run of the new _DEFLATE$ and _INFLATE$ commands in it.  Eventually, I’d like to push those commands into the main QB64 repo for us, but it’ll probably be a while until that happens, as I still need a Linux/Mac user to help get them working on those platforms.  Until then though, you can always test the commands by grabbing a copy of Steve64 and playing around with it.  ;D
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!