MemToHex (and back)

[SMcNeill]

A set of little utilities which folks might want to add to their collection:

Code: QB64: [Select]

1. SCREEN _NEWIMAGE(640, 480, 32)
2. DIM m AS _MEM
3. t = _NEWIMAGE(16, 16, 32)
4. m = _MEMIMAGE(t)
5. _DEST t
6. CIRCLE (8, 8), 7, -1
7. PAINT (8, 8), -1
8. _DEST 0
9. _CLIPBOARD$ = MemImagetoDATA$(m, "t", 160) 'Here's where I got the DATA to paste below
10.  
11. RESTORE t 'remember to restore using the proper label you associated with the data
12. t1 = DATAtoImage 'Make a new image using the DATA
13. _PUTIMAGE (30, 30), t1
14.  
15.  
16. 'Here's where I pasted the clipboard DATA from the memimage
17.  
18. t:
19. DATA 16,16,4
20. DATA 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
21. DATA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000
22. DATA 0000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
23. DATA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
24. DATA 00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
25. DATA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000000000000000000000000
26. DATA 000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000000000000000000000000000
27.  
28. 'NOTICE:  We turn Checking OFF from this point down, as these are _MEM routines and can greatly benefit from
29. '         the speed boost.  This means it's up to the USER/PROGRAMMER of these routines to be certain to NOT
30. '         screw up calling them.  RESTORE proper data labels, if necessary.  Make certain that you're actually
31. '         passing them valid mem blocks, or valid image blocks...
32. '
33. '         If not....
34. '
35. '         Meeehhhhh!   It's not *MY* program which'll need debugging later....
36.  
37. $CHECKING:OFF
38. FUNCTION DATAtoImage&
39.     'Requires a RESTORE label for the proper data BEFORE calling this routine
40.     READ w, h, ps
41.     SELECT CASE ps
42.         CASE 1: ps = 256 '256 color screen
43.         CASE 2: ps = 0 'text screen
44.         CASE 4: ps = 32 '32-bit color screen
45.     END SELECT
46.     DATAtoImage& = _NEWIMAGE(w, h, ps)
47.     DIM m AS _MEM
48.     m = _MEMIMAGE(DATAtoImage&)
49.     DIM h AS _UNSIGNED _BYTE
50.     DO
51.         READ hx$
52.         FOR i = 1 TO LEN(hx$) STEP 2
53.             h = VAL("&H" + MID$(hx$, i, 2))
54.             _MEMPUT m, m.OFFSET + o, h
55.             o = o + 1
56.         NEXT
57.         LOCATE 1, 1: PRINT o, m.SIZE
58.     LOOP UNTIL o >= m.SIZE
59.     _MEMFREE m
60. END FUNCTION
61.  
62.  
63. FUNCTION MemImagetoDATA$ (m AS _MEM, label$, break)
64.     DIM b AS _UNSIGNED _BYTE
65.     s = ConvertOffset(m.SIZE) - 1
66.     label$ = _TRIM$(label$)
67.     IF label$ = "" THEN label$ = "generic_label_placeholder:"
68.     IF RIGHT$(label$, 1) <> ":" THEN label$ = label$ + ":"
69.     MemImagetoDATA$ = label$ + CHR$(10) + "DATA "
70.     MemImagetoDATA$ = MemImagetoDATA$ + STR$(_WIDTH(m.IMAGE)) + ", "
71.     MemImagetoDATA$ = MemImagetoDATA$ + STR$(_HEIGHT(m.IMAGE)) + ", "
72.     MemImagetoDATA$ = MemImagetoDATA$ + STR$(_PIXELSIZE(m.IMAGE)) + CHR$(10) + "DATA "
73.     FOR i = 0 TO s
74.         _MEMGET m, m.OFFSET + i, b
75.         h$ = HEX$(b)
76.         IF LEN(h$) = 1 THEN h$ = "0" + h$
77.         MemImagetoDATA$ = MemImagetoDATA$ + h$
78.         IF i MOD break = break - 1 AND i < s THEN
79.             MemImagetoDATA$ = MemImagetoDATA$ + CHR$(10) + "DATA "
80.         END IF
81.     NEXT
82. END FUNCTION
83.  
84. FUNCTION MemToHex$ (m AS _MEM)
85.     DIM b AS _UNSIGNED _BYTE
86.     s = ConvertOffset(m.SIZE) - 1
87.     FOR i = 0 TO s
88.         _MEMGET m, m.OFFSET + i, b
89.         h$ = HEX$(b)
90.         IF LEN(h$) = 1 THEN h$ = "0" + h$
91.         MemToHex$ = MemToHex$ + h$
92.     NEXT
93. END FUNCTION
94.  
95. SUB HexToMem (hx$, m AS _MEM)
96.     DIM i AS _INTEGER64
97.     DIM h AS _UNSIGNED _BYTE
98.     FOR i = 1 TO LEN(hx$) STEP 2
99.         h = VAL("&H" + MID$(hx$, i, 2))
100.         _MEMPUT m, m.OFFSET + i \ 2, h
101.     NEXT
102. END SUB
103.  
104. FUNCTION ConvertOffset&& (value AS _OFFSET)
105.     DIM m AS _MEM 'Define a memblock
106.     m = _MEM(value) 'Point it to use value
107.     $IF 64BIT THEN
108.         'On 64 bit OSes, an OFFSET is 8 bytes in size.  We can put it directly into an Integer64
109.         _MEMGET m, m.OFFSET, ConvertOffset&& 'Get the contents of the memblock and put the values there directly into ConvertOffset&&
110.     $ELSE
111.         'However, on 32 bit OSes, an OFFSET is only 4 bytes.  We need to put it into a LONG variable first
112.         _MEMGET m, m.OFFSET, temp& 'Like this
113.         ConvertOffset&& = temp& 'And then assign that long value to ConvertOffset&&
114.     $END IF
115.     _MEMFREE m 'Free the memblock
116. END FUNCTION
117. $CHECKING:ON

With these, we can convert ANY memblock into a sequence of HEX values (which makes them excellent to paste into another program as DATA statements), and then we can take those hex values and convert them back into a memblock.

This also has a couple of routines which are set up to specialize in taking memimages and converting them to DATA statements, and then back again for us, for ease of embedding and use in our programs.


Enjoy!

[STxAxTIC]

Yello Steve,

So the Librarian told me that this code is a almost good candidate for the Toolbox - but we have to massage it to fit the criteria. It seems this post has a bag of tools in itself, but the thing being demo'd here doesn't use all of them, namely MemToHex$ and its reversing counterpart. If you have time, can you fragment this into smaller independently-robust chunks? They are free to exist as neighbors and refer to each other, but they should be untangled as tools if you don't mind.

[SMcNeill]

Yello Steve,

So the Librarian told me that this code is a almost good candidate for the Toolbox - but we have to massage it to fit the criteria. It seems this post has a bag of tools in itself, but the thing being demo'd here doesn't use all of them, namely MemToHex$ and its reversing counterpart. If you have time, can you fragment this into smaller independently-robust chunks? They are free to exist as neighbors and refer to each other, but they should be untangled as tools if you don't mind.

MemToHex is easy to showcase:

Code: [Select]

DIM junk AS STRING * 12
DIM m AS _MEM: m = _MEM(junk)
junk = “Hello World!”
Hx$ = MemToHex(m)
PRINT Hx$

junk$ = “Overwritten?” ‘See where we overwrite that string?
HexToMem Hx$, m ‘restore it
PRINT junk$ ‘and now print it

FUNCTION MemToHex$ (m AS _MEM)
    DIM b AS _UNSIGNED _BYTE
    s = ConvertOffset(m.SIZE) - 1
    FOR i = 0 TO s
        _MEMGET m, m.OFFSET + i, b
        h$ = HEX$(b)
        IF LEN(h$) = 1 THEN h$ = "0" + h$
        MemToHex$ = MemToHex$ + h$
    NEXT
END FUNCTION
 
SUB HexToMem (hx$, m AS _MEM)
    DIM i AS _INTEGER64
    DIM h AS _UNSIGNED _BYTE
    FOR i = 1 TO LEN(hx$) STEP 2
        h = VAL("&H" + MID$(hx$, i, 2))
        _MEMPUT m, m.OFFSET + i \ 2, h
    NEXT
END SUB
 
FUNCTION ConvertOffset&& (value AS _OFFSET)
    DIM m AS _MEM 'Define a memblock
    m = _MEM(value) 'Point it to use value
    $IF 64BIT THEN
        'On 64 bit OSes, an OFFSET is 8 bytes in size.  We can put it directly into an Integer64
        _MEMGET m, m.OFFSET, ConvertOffset&& 'Get the contents of the memblock and put the values there directly into ConvertOffset&&
    $ELSE
        'However, on 32 bit OSes, an OFFSET is only 4 bytes.  We need to put it into a LONG variable first
        _MEMGET m, m.OFFSET, temp& 'Like this
        ConvertOffset&& = temp& 'And then assign that long value to ConvertOffset&&
    $END IF
    _MEMFREE m 'Free the memblock
END FUNCTION
$CHECKING:ON
The purpose of these is to let us convert a segment of memory into a series of hex values, to write into a DATA field (or some such purpose), without any issues.

After all, it’s hard to print a CRLF (carriage return line feed) sequence in a data statement, but there’s no worries when storing a DATA 0D0A.

The MemImageToData is just a specialized version of the routine which takes a _MEMIMAGE and converts it to Hex, while placing the DATA statement in front of it for us, for quick use pasting it into another program.  It basically lets us store image files inside a BAS file, without needing to _LOADIMAGE them externally.

[Petr]

Hi Steve. I experimented a little. Your way of writing in hexadecimal values is great. But if you combine it with your other great text compression program, this image can be saved in 43 bytes (originally 256 bytes). It uses characters outside the visible area of the ascii table on the monitor, so we have our own QB64 compression image format, just insert it into a binary file :-) or, while keeping the idea, the compressed content can be saved to the source file as a hexadecimal compressed record. In our case for this image it is

0100230461000FFFFFFFFFFFF00000FFFFFFFFFFFFFFFFFFFFFFFF00000FFFFFFFFFF

The possibility of further compression is obvious in this chain, but it would probably be better to use the LZW method instead of the economical write method. Thank you for really many materials for next exploration this week!

[SMcNeill]

Hi Steve. I experimented a little. Your way of writing in hexadecimal values is great. But if you combine it with your other great text compression program, this image can be saved in 43 bytes (originally 256 bytes). It uses characters outside the visible area of the ascii table on the monitor, so we have our own QB64 compression image format, just insert it into a binary file :-) or, while keeping the idea, the compressed content can be saved to the source file as a hexadecimal compressed record. In our case for this image it is

0100230461000FFFFFFFFFFFF00000FFFFFFFFFFFFFFFFFFFFFFFF00000FFFFFFFFFF

The possibility of further compression is obvious in this chain, but it would probably be better to use the LZW method instead of the economical write method. Thank you for really many materials for next exploration this week!

I’d already considered this possibility, which is why I’ve been working on adding compression commands into the language for us.  Grab a copy of my repo and give a test run of the new _DEFLATE$ and _INFLATE$ commands in it.  Eventually, I’d like to push those commands into the main QB64 repo for us, but it’ll probably be a while until that happens, as I still need a Linux/Mac user to help get them working on those platforms.  Until then though, you can always test the commands by grabbing a copy of Steve64 and playing around with it.  ;D