Show Posts
1
Programs / Custom Image Format ***with compression***
« on: December 29, 2021, 09:55:43 pm »
Here's my NOTES I kept fooling around... I am gonna post the couple of sample programs now, I dont know HOW long I will be here (and have internet) before I leave again.
By next time I'm here and I can post again... I should have this "genericized" into a couple of SUBs
anyways, heres my notes I kept...
==================================================
NOTESx
Why?
Several reasons. First and foremost, I have a recent interest in images and working with them in most of my projects. I have the classic quandary; I can make neat stuff on the screen, creatively reading in and displaying whole and parts of images... but I cant SAVE this cool and novel image.
Yes, I know, I am aware there are routines developed to save in "one format". And that its a STANDARD format.
The 'problem' is that I want to TINKER. I thought I was going to read about the "structure" of several popular image file formats, pick one, and start reading it in and writing it out as/when i felt like it.
This logic, while techincally sound, had one fatal flaw: I'm not on the "same wavelength" with the file format. ANY file format, there are hard ones, and impossible ones... there doesnt seem to be a single "easy" format for me, the programmer.
What can I do about this?
Well... I simply make up my own image format. I spaghetti-coded a couple of untitled hot mess proof-of-concept programs. I decided I needed a custom image I would recognize... I decided 640x480 was fine for a test.
A certain number of random lines and random circles, in random colors. HEY, a recognizeable 32 bit "image" on the screen.
640x480 yields exactly 307,200 pixels
they are RIGHT there on my screen, I need each pixels COLOR.
I simply make a big array of the image dimensions, its type ~& for not losing the blue component the manual said I think.
I fill up the array by sampling the pixels with POINT(x,y)
now I have an array filled with color values i can work with.
For now, I just need to utilitize this as best i can.
the original format I called .imap (image map?)
the first 2 lines are read in, how wide and how tall, respectively.
after that? follows (640 x 480) = 307200 lines, each line representing a color
since the Y loop is inside the overall x loop... it gets done by vertical stripes
this could be reversed though I dont see how it matters
each color is represented by 2 hex characters, so 6 hex characters on each line represent the 3 color components of that pixel.
since I used a black background, I decided to use a "0" instead of " 0 0 0" for ZERO color, this made some savings. though file was still huge for its image size.
it WORKS though. the MAKE program makes a sample 640x480 image of random lines and citrcles, and saves it in .imap format
the OPEN program, proves the system works by reading it in, and displaying it. YES, its very slow and more slow given its relatively small 640x480 test size. BUT, I have no "hardware acceleration", I have no "commercial compression", nothing.
so, if it just WORKS at all, I'm happy. If it works just fast enough I can actually sort of use it? great!
next, I need some kind of compression. I dont want ANY loss of any kind, so any strategy canonly be "loss-less". Well, there are a LOT of "0" entries in the file, since it was a black background I painted on.
My basic compresion strategy is as follows:
I sweep through the file. Any time the NEXT LINE is the same as the PRESENT LINE? I keep track of it, and keep reading forward, line by line, until I hit the spot its finally a different color line.
there is NO REASON to have that many identical lines, I decided to make a "postprocessor" character "+", followed by the number of identical lines to follow.
The first 345 lines of my sample image? were zero color. After running POST, the post-processed .imap2 file contains none of the skipped (identical) lines.
My final sweep through, put out to .imap3 file, looks for " 0 0 0" entries and replaces them with the "preprocessor" character "@".
its a big savings, and its completely "lossless" of a format.
All I need now is a .imap3 OPEN demo, and I am golden for a start.
====================================
what can I do with this?
well, I can open any image file, and scale and display it any way I want to.
mirror images, text added, whatever.
now, if i WANT to? I can easily experiment with making some kind of "GIMP filter"
type of deal. This gets me STARTED, allows me to make a sample interface, do whatever I want (crop, go B+W, whatever) and save it out.
Later ON, if i can get more comfortable with any real file structures, I can convert to them. I want to know how to work with stuff, this is the best way for me to learn I figure.
I figure no matter WHAT format I end up maybe getting into? I will STILL need an easy internal representation of what each pixel is doing, this format IS that representation. The fact I can save and load and edit it easily myself is just a side benefit.
SIZES @ image size 640x480:
.imap = 2.5MB ;raw file, zero compression
.imap2 = 20.2 kB ;compression by duplication skip-tracking
.imap3 = 17.9 kB ;every instance " 0 0 0" replaced with "@"
NOTE: this is 100% loss-less compression. not one pixel, not one color, has been molested from the original.
NOTE: if i draw less or more random lines and circles? the sizes of the compressed files inches up and down with it. Reason why being, a few less lines and circles, means bigger and longer "zero gaps" of black background, thatget replaced bya single entry at a time. The more single color back ground showing, the smaller the compression gets.
NOTE: I chose " 0 0 0" (black) to switch to "@" on final compression pass, simply because it was the background color and a lot of it.
in the future, i will probably add a line in the file up front, indicating what color is to be replaced by the @.
EG, if you know your background is for instance WHITE, then, obviously you would want to replace THAT color with the "@" shortener symbol.
=========================================================================
As a sample image, I switched over from "some lines and circles" to "giant text font".
In this case? the "compression" numbers are actually pretty great !
no matter WHAT the image is? its ALWAYS 2.5MB for the .imap file, it has no choice. If i write out the colors to every pixel? Thats simply "the size" and thats all there is to it.
20k and 17k are "very respectable" I feel, as rolled-at-home compression alogrithms, given that it is 100 percent "no loss"
By next time I'm here and I can post again... I should have this "genericized" into a couple of SUBs
anyways, heres my notes I kept...
==================================================
NOTESx
Why?
Several reasons. First and foremost, I have a recent interest in images and working with them in most of my projects. I have the classic quandary; I can make neat stuff on the screen, creatively reading in and displaying whole and parts of images... but I cant SAVE this cool and novel image.
Yes, I know, I am aware there are routines developed to save in "one format". And that its a STANDARD format.
The 'problem' is that I want to TINKER. I thought I was going to read about the "structure" of several popular image file formats, pick one, and start reading it in and writing it out as/when i felt like it.
This logic, while techincally sound, had one fatal flaw: I'm not on the "same wavelength" with the file format. ANY file format, there are hard ones, and impossible ones... there doesnt seem to be a single "easy" format for me, the programmer.
What can I do about this?
Well... I simply make up my own image format. I spaghetti-coded a couple of untitled hot mess proof-of-concept programs. I decided I needed a custom image I would recognize... I decided 640x480 was fine for a test.
A certain number of random lines and random circles, in random colors. HEY, a recognizeable 32 bit "image" on the screen.
640x480 yields exactly 307,200 pixels
they are RIGHT there on my screen, I need each pixels COLOR.
I simply make a big array of the image dimensions, its type ~& for not losing the blue component the manual said I think.
I fill up the array by sampling the pixels with POINT(x,y)
now I have an array filled with color values i can work with.
For now, I just need to utilitize this as best i can.
the original format I called .imap (image map?)
the first 2 lines are read in, how wide and how tall, respectively.
after that? follows (640 x 480) = 307200 lines, each line representing a color
since the Y loop is inside the overall x loop... it gets done by vertical stripes
this could be reversed though I dont see how it matters
each color is represented by 2 hex characters, so 6 hex characters on each line represent the 3 color components of that pixel.
since I used a black background, I decided to use a "0" instead of " 0 0 0" for ZERO color, this made some savings. though file was still huge for its image size.
it WORKS though. the MAKE program makes a sample 640x480 image of random lines and citrcles, and saves it in .imap format
the OPEN program, proves the system works by reading it in, and displaying it. YES, its very slow and more slow given its relatively small 640x480 test size. BUT, I have no "hardware acceleration", I have no "commercial compression", nothing.
so, if it just WORKS at all, I'm happy. If it works just fast enough I can actually sort of use it? great!
next, I need some kind of compression. I dont want ANY loss of any kind, so any strategy canonly be "loss-less". Well, there are a LOT of "0" entries in the file, since it was a black background I painted on.
My basic compresion strategy is as follows:
I sweep through the file. Any time the NEXT LINE is the same as the PRESENT LINE? I keep track of it, and keep reading forward, line by line, until I hit the spot its finally a different color line.
there is NO REASON to have that many identical lines, I decided to make a "postprocessor" character "+", followed by the number of identical lines to follow.
The first 345 lines of my sample image? were zero color. After running POST, the post-processed .imap2 file contains none of the skipped (identical) lines.
My final sweep through, put out to .imap3 file, looks for " 0 0 0" entries and replaces them with the "preprocessor" character "@".
its a big savings, and its completely "lossless" of a format.
All I need now is a .imap3 OPEN demo, and I am golden for a start.
====================================
what can I do with this?
well, I can open any image file, and scale and display it any way I want to.
mirror images, text added, whatever.
now, if i WANT to? I can easily experiment with making some kind of "GIMP filter"
type of deal. This gets me STARTED, allows me to make a sample interface, do whatever I want (crop, go B+W, whatever) and save it out.
Later ON, if i can get more comfortable with any real file structures, I can convert to them. I want to know how to work with stuff, this is the best way for me to learn I figure.
I figure no matter WHAT format I end up maybe getting into? I will STILL need an easy internal representation of what each pixel is doing, this format IS that representation. The fact I can save and load and edit it easily myself is just a side benefit.
SIZES @ image size 640x480:
.imap = 2.5MB ;raw file, zero compression
.imap2 = 20.2 kB ;compression by duplication skip-tracking
.imap3 = 17.9 kB ;every instance " 0 0 0" replaced with "@"
NOTE: this is 100% loss-less compression. not one pixel, not one color, has been molested from the original.
NOTE: if i draw less or more random lines and circles? the sizes of the compressed files inches up and down with it. Reason why being, a few less lines and circles, means bigger and longer "zero gaps" of black background, thatget replaced bya single entry at a time. The more single color back ground showing, the smaller the compression gets.
NOTE: I chose " 0 0 0" (black) to switch to "@" on final compression pass, simply because it was the background color and a lot of it.
in the future, i will probably add a line in the file up front, indicating what color is to be replaced by the @.
EG, if you know your background is for instance WHITE, then, obviously you would want to replace THAT color with the "@" shortener symbol.
=========================================================================
As a sample image, I switched over from "some lines and circles" to "giant text font".
In this case? the "compression" numbers are actually pretty great !
no matter WHAT the image is? its ALWAYS 2.5MB for the .imap file, it has no choice. If i write out the colors to every pixel? Thats simply "the size" and thats all there is to it.
20k and 17k are "very respectable" I feel, as rolled-at-home compression alogrithms, given that it is 100 percent "no loss"
