Print Page - Steve's Video Tutorials

Active Forums => Programs => Topic started by: SMcNeill on September 22, 2019, 01:59:55 pm

Title: Steve's Video Tutorials
Post by: SMcNeill on September 22, 2019, 01:59:55 pm

Hello QB64 World:

_MEM Tutorial #1 - Basic Introduction To Mem:

(https://www.qb64.org/forum/index.php?topic=1731.msg109781#msg109781 )

_MEM Tutorial #2 - Little Endians and Big Chiefs:

(https://www.qb64.org/forum/index.php?topic=1731.msg109793#msg109793)

_MEM Tutorial #3 - Finding the Offset:

Tutorial #4 -- REDIM _PRESERVE and Beyond The First Dimension:

(More will follow, so I'm just going to leave this first post as a "quick link" area so folks can see what's up and go take a look at them, if they want.)

The links above will often be in pairs. The first link is to the youtube video which you can watch and study as often as your heart desires, while the second link is to a post in this topic here which has the finalized code from the tutorial (with comments and all) available for ease of reference, study, and questions.

Title: Re: Steve's Video Tutorials
Post by: Petr on September 22, 2019, 02:14:29 pm

Thank you Steve. What is the quality of the source video? Youtube offers me a maximum of 360P and the image is blurry so I can hardly read the statements.

You use camera for saving video, or some screen recorder software?

[ This attachment cannot be displayed inline in 'Print Page' view ]

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 22, 2019, 02:50:08 pm

Screen Recorder software. I'm still learning what works and doesn't work, so this time the video might've been too small. It looked OK on my machine, but once Youtube did it's thing, it seems like it blurred it out almost beyond recognition.

The first video I did was at 1920x1280 resolution, but it was quite large -- 104mb for only a minute (1:06 precisely). The second video is at 640x480, and comes in at around 22 minutes, and a total of 429mb in size. My bandwidth is unlimited here, but the upload speed is rather slowish, and it took over an hour to upload the 429mb file to Youtube....

So, I'm shooting for the best setting which gives me viewable, decent results, and minimal file size to minimize upload speeds. If I'd ran the second video in 1920x1200, like the first, the size would've came in around 2.2GB for the whole video and took ~6 hours or so to upload...

For the next one, I'll try a middle ground between the two and see how it does. With practice, and as I learn more about the software and settings, I'll eventually find what's optimal for my needs (and the QB64's community's needs). At that point, I can always go back and redo any low/poor quality videos where folks have issue reading the text and such, so be certain to post and let me know when there's an issue.

I'm quite used to teaching people things, so I certainly don't mind doing the tutorials (and hope they'll help someone learn), but right now, I'm also learning -- learning how and what works best for this newfangled recording environment, which I've never worked with before.

If anyone has more experience with these things than me (and it doesn't take a whole lot to have more experience than me...), feel free to share any pointers/advice/tips and such. I'm all ears for anything which makes the learning curve faster and smoother on my end, so I can help do tutorials to make things faster/easier for others to learn QB64's commands. ;D

Title: Re: Steve's Video Tutorials
Post by: Petr on September 22, 2019, 03:04:32 pm

For example. This my video:

is in original size 1.482.845.184 bytes (1,5 GB) big. My upload speed is 2 - 3 MB/s. Video lenght is 16 minutes and 37 seconds, Youtube do it as HD, video resolution (in original uploaded file) is 1040 x 840 pixels, 25 fps (europe standard) sound format is MP3 44Khz, stereo, 160 KBps. Total rate is 12625 Kbps. Video format is WMV. All is done also with screen recorder software.

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 22, 2019, 03:12:25 pm

This is really good Steve.

Title: Re: Steve's Video Tutorials
Post by: FellippeHeitor on September 22, 2019, 03:33:14 pm

Glad to finally "meet" you 😉

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 22, 2019, 04:48:41 pm

Quote from: Petr on September 22, 2019, 03:04:32 pm

For example. This my video:
is in original size 1.482.845.184 bytes (1,5 GB) big. My upload speed is 2 - 3 MB/s. Video lenght is 16 minutes and 37 seconds, Youtube do it as HD, video resolution (in original uploaded file) is 1040 x 840 pixels, 25 fps (europe standard) sound format is MP3 44Khz, stereo, 160 KBps. Total rate is 12625 Kbps. Video format is WMV. All is done also with screen recorder software.

You can see my speeds with the attachment below — 12mbs download speed, 780kbs upload speed. (0.78 MBS upload speed.) My ISP Provider has decent enough download rates, but really crappy upload speeds. It’d take me 3-4 times as long to upload the same file as it does you. :(

Future videos may require simply saving them in a higher resolution, and then uploading overnight while I sleep, before announcing them to the public the next day.

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 22, 2019, 05:18:33 pm

I can see your point there. My connection is having an off day, these speeds are half what I usually see. I will have to take that up with my provider considering what I pay and that it is the only service package they offer out here.

Still uploading video, especially a 22min one, on less than 100KB a second (780kbs\8) would make it hard to upload sharper videos. But you read out everything you were typing in, I believe, so reading isn't that important.

Though I think Petr's native language isn't English so that might make it harder.

Title: Re: Steve's Video Tutorials
Post by: Petr on September 22, 2019, 05:26:45 pm

And mine:

[ This attachment cannot be displayed inline in 'Print Page' view ]

Monthly pay for this provider is 500 CZK (=~ 20 USD)

Title: Re: Steve's Video Tutorials
Post by: STxAxTIC on September 22, 2019, 06:15:02 pm

This tutorial was awesome as hell. Do more Steve!

Title: Re: Steve's Video Tutorials
Post by: TempodiBasic on September 22, 2019, 07:00:03 pm

fine to meet you Steve!

Cool tutorial!
Waiting other tutorials.

Title: Re: Steve's Video Tutorials
Post by: _vince on September 22, 2019, 08:44:27 pm

It's really nice to have a fellow human voice over the stuff we put so many hours and passion in to. I appreciate this, Steve, and hope to see more - especially graphics and animation which lends very well to the video format.

Title: Re: Steve's Video Tutorials
Post by: RhoSigma on September 23, 2019, 02:56:17 am

Pog Mo Thoin - Steve this is awesome, looking forward to see more. You did so many good things and expainations to QB64 already, most of the times much to read on those teaching/explaining posts :), now this stuff finally got a voice and a face to associate with and we can simply watch and listen to your teachings rather than the need to read 1, 2 or sometimes even 3 screen pages :)

Thank You Steve

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 23, 2019, 05:20:20 am

Steve, excellent excellent video. Steve: farmer, author, code developer, communicator and now YouTube teacher - a pedagogical polymath! Thank you so much.

Odin, we could definitely do with a new Tutorials Section to put Steve's postings. We don't want to lose these postings getting pushed off page 1 here.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 23, 2019, 07:15:02 am

And now for a technical question. In the tutorial, the difference between file (hard disk) and _MEM objects is demonstrated, with the _MEM objects obviously (much) faster in access & processing than disk objects. But if we exchanged the _MEM object with a "normal" variable / array, that would also be in memory and nowhere near the disk. Is _MEM object processing still much faster than normal variable / array processing? (I am, of course, sat at the back of the class with a pointed hat on, labelled "D").

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 23, 2019, 09:22:41 am

Quote from: Qwerkey on September 23, 2019, 07:15:02 am

And now for a technical question. In the tutorial, the difference between file (hard disk) and _MEM objects is demonstrated, with the _MEM objects obviously (much) faster in access & processing than disk objects. But if we exchanged the _MEM object with a "normal" variable / array, that would also be in memory and nowhere near the disk. Is _MEM object processing still much faster than normal variable / array processing? (I am, of course, sat at the back of the class with a pointed hat on, labelled "D").

It is — sometimes by upwards of 100+ times faster.

When you use _MEM, you go directly to the memory and read/write it.

When you use an array, or variable, your computer first has to look into a table and search for that variable. Where is X stored in our lookup table? Then, once it’s found, what’s it value? What type of variable is it? How big is it? With an array, which element is it, in relation to the rest of the array? The OS references all this information first, and then uses it to go to the memory address and retrieve the information for you.

Which is inherently faster: Me telling you, “Get the third item off the top shelf,” or, “Fetch me the glibbersprocket,” if you have no idea what a glibbersprocket is, and I have to take time to describe it for you. This is the difference between _MEM and variables/arrays.

***************************

And don’t think of yourself as a dunce in the back of the class at all. I’m happy folks are watching, taking interest, and asking questions. That’s the whole point of doing the tutorials after all! ;D

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 23, 2019, 09:43:47 am

Thanks Steve, I had imagined that this would be the case.

Title: Re: Steve's Video Tutorials
Post by: Dimster on September 23, 2019, 12:27:04 pm

What a great tutorial. So very helpful for me on both Binary and _Mem. I am finding my screen is somewhat fuzzy, so the coding you are writing is hard to see but you explain it so well the fuzz makes sense. The old Peek and Poke seemed to be dangerous (in terms of remembering what address did what, and propensity to crash the program) so I did avoid using them, _Mem, I thought was going to be just as dangerous but maybe I do need to relook at this and your future videos. I'm working on an AI program (been at it for years). There's a lot of data to manipulate and multiple formulas. My focus has been on accuracy v's speed, needless to say its a slow moving AI.

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 23, 2019, 12:46:08 pm

Yep, the only thing faster than _MEM are BITWISE operators, the reason for this is the same as _MEM being faster than normal variable interactions. BITWISE operators are processed 'in house' within the processor, they never have to travel on the BUS to memory and back. they stay in the CPU, so you cut travel time nearly completely out. With today's BUS speeds the difference is getting smaller though. And when your working with variables the standard way there is a lot of back and forth between the CPU and RAM before the result is done.

You could think of it as needing a vegetable to eat,your property seen as the CPU, is faster to walk out to your garden and back than it is to take the bullet train to the store, seen as RAM, and back. its all a matter of distance traveled, and even at the speed of light(or 99% of it as electrons travel) it still takes time. As the standard use would be like Calling the store to make sure they had the vegetable, then calling a Cab, waiting on the Cab, going to the store buying the vegetable, then calling the Cab again, waiting on the Cab then getting home with the vegetable. the more travel and talking you can cut out the faster things happen.

_MEM gives you a heck of a lot more options though, as BITWISE operations are very limited being restricted to a single variable. Where as _MEM can act on a large group of variables. but that is getting into Apples vs Oranges territory there.

Granted a rather Simplified explanation, and may be a bit short here and there.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 23, 2019, 01:11:06 pm

Steve and Cobalt, oh wow! I'm looking forward to doing _MEM versions of my Crossword Generator (should produce new completed grids virtually instantly) and my Gravitational Simulation program (should produce a solar system from a collapsing random array of masses in a few minutes).

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 23, 2019, 03:49:14 pm

Tutorial #1's finished code (as per the demo), placed here for ease of reference, in case anyone wants it:

QB64 window #1:

Code: QB64: [Select]

SCREEN _NEWIMAGE(800, 465, 32)
_SCREENMOVE 121, -2
 
F = FREEFILE
OPEN "tempbooger.txt" FOR BINARY AS #F
PRINT LOF(F)
junk$ = "Hello World"
PUT #F, 1, junk$
 
temp$ = SPACE$(5)
GET #F, 1, temp$
PRINT temp$
 
GET #F, 7, temp$
PRINT temp$
 
CLOSE #F

Compared to QB64 window #2:

Code: QB64: [Select]

SCREEN _NEWIMAGE(800, 465, 32)
_SCREENMOVE 921, -2
 
DIM M AS _MEM
M = _MEMNEW(10000)
PRINT M.SIZE
junk$ = "Hello World"
_MEMPUT M, M.OFFSET, junk$
 
temp$ = SPACE$(5)
_MEMGET M, M.OFFSET, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + 6, temp$
PRINT temp$
 
_MEMFREE M
 

As you can hopefully see by comparing the two programs closely against each other, their syntax and usage is almost exactly the same. If you can read and write to a binary file on your hard drive, you can read and write to memory inside your program.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 23, 2019, 04:43:28 pm

Tutorial #2 is now up on youtube (find the link via the quick reference area of the first post in this topic), and here's the finished code which we end up producing from it:

Code: QB64: [Select]

SCREEN _NEWIMAGE(640, 380, 32)
_SCREENMOVE _MIDDLE
 
PRINT "Steve's _MEM Tutorial lesson 2: Little Endian and Big Endian"
PRINT
 
'Little Endian stores values from right to left.
'Big Endian stores values from left to right.
 
'Let's take the value Ninty-One, as an example.
'In English (and most other languages I know), we would write that value as "91".
'The 9 here represents the ten's position, and the 1 represents the one's position.
 
'But what if we were wringing in Arabic, or Hebrew?  Those languages read from
'right to left.  I'm no expert with either language -- in fact, I don't know
'squat of either -- but wouldn't they write the value as "19"?
'The 9 would still represent the ten's position, and the 1 would still represent
'the one's position, but we'd write them from right to left, instead of from left
'to right.
 
'This is basically the difference in Little Endian values and Big Endian values.
'Little Endian stores values from right to left.
'Big Endian stores values from left to right.
 
DIM x AS INTEGER
DIM m AS _MEM
m = _MEM(x)
 
_MEMPUT m, m.OFFSET, 1 AS INTEGER
PRINT x
 
x = 0
PRINT x
 
_MEMPUT m, m.OFFSET, 1 AS _UNSIGNED _BYTE
PRINT x
 
x = 0
PRINT x
 
_MEMPUT m, m.OFFSET + 1, 1 AS _UNSIGNED _BYTE
PRINT x
 
PRINT "Hello World"
SLEEP
_MEMFREE m
m = _MEMIMAGE(0) '0 is the display screen.
DO
    IF _MEMGET(m, m.OFFSET + o, _UNSIGNED LONG) = _RGBA32(255, 255, 255, 255) THEN
        _MEMPUT m, m.OFFSET + o, 256 AS _UNSIGNED _BYTE
    END IF
    o = o + 4
LOOP UNTIL o >= m.SIZE - 4
'White = "&H" FF,FF,FF,FF  AA, RR, GG, BB.
'It's stored LITTLE ENDIAN values.
'Blue, Green, Red, Alpha
 
_MEMFREE m

Title: Re: Steve's Video Tutorials
Post by: FellippeHeitor on September 23, 2019, 07:03:54 pm

Great job with image quality this time.

Title: Re: Steve's Video Tutorials
Post by: Dimster on September 24, 2019, 10:20:03 am

Quality is great, topic fascinating.
Before I start messing around with my rather large program, how do I determine how much memory is actually available before blocking off a section?
Also, I'm thinking, the use of memory is not necessarily to "store" data for the duration of the program run, seems the idea would be to block off what you need, use it for fast calculations or temporary storage / retrieval / update of data and then Free the Memory block. So is there a way to determine exactly how much memory size might be needed for different types of tasks/usage of the memory block set aside?
Is the size of the memory block automatically calculated for an array by use of LEN?
I did a search of the wiki but not of the forum past topics, so I'm sorry Steve if these question have been previously addressed.

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 24, 2019, 10:42:49 am

Quote from: Dimster on September 24, 2019, 10:20:03 am

Before I start messing around with my rather large program, how do I determine how much memory is actually available before blocking off a section?

the answer to that question is the amount of available memory on your machine, and whether or not you are using 32bit or 64bit QB64.

Quote from: Dimster on September 24, 2019, 10:20:03 am

So is there a way to determine exactly how much memory size might be needed for different types of tasks/usage of the memory block set aside?

to answer that, you have to know the sizes of what you want to be working with, _BYTE=1, INTEGER=2, LONG\SINGLE=4, DOUBLE\INTEGER64=8, and STRINGs are 1*len of string.
for more types go into the wiki under Variable Types: http://www.qb64.org/wiki/Variable_Types

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 24, 2019, 11:07:43 am

Quote from: Dimster on September 24, 2019, 10:20:03 am

Quality is great, topic fascinating.
Before I start messing around with my rather large program, how do I determine how much memory is actually available before blocking off a section?
Also, I'm thinking, the use of memory is not necessarily to "store" data for the duration of the program run, seems the idea would be to block off what you need, use it for fast calculations or temporary storage / retrieval / update of data and then Free the Memory block. So is there a way to determine exactly how much memory size might be needed for different types of tasks/usage of the memory block set aside?
Is the size of the memory block automatically calculated for an array by use of LEN?
I did a search of the wiki but not of the forum past topics, so I'm sorry Steve if these question have been previously addressed.

Available memory depends on your OS and PC.

If you’re using 32-bit versions of QB64, you’re going to be capped out at about 1.5GB total memory usage in your programs. (2GB is the true 32-bit cap due to limits with using LONG values for offsets, but your OS reserves some space for its needs. Once you start getting at the 1.5 GB memory mark, on a 32-bit OS, your program is walking dangerously close to crashing.)

For 64-bit systems, you can use as much memory as your system has plus allocated swap space. The largest program I’ve ever ran used about 8-9GB of memory, as it had a huge database associated with it and loaded the WHOLE thing into memory at once, to sort/pack/compress it — and it ran with no issues at all on my home PC (with 32GB RAM), but was impossible to run on my laptop (8GB RAM).

For arrays, you can DIM them as you normally do, and then just point a MEM variable to where they’re already at in memory. (Lesson 3 should cover this topic, I think, but it’ll probably be a few days before I get around to it.)

Basically:

DIM MyArray(1000) AS ArrayType
DIM M AS _MEM
M = _MEM(ARRAY())

And with the above, your MEM variable is pointing to your MyArray in memory.

Title: Re: Steve's Video Tutorials
Post by: Dimster on September 24, 2019, 11:43:14 am

Hi Cobalt - So I have 8 GB of Ram, 64 bit os. So I believe Steve said some memory is already protected by windows and my program itself must be taking up some memory. Also as the years go by and the program grows, more memory usage is expected. I recall in the old days there was a Free Memory command or something like that, which did not Free UP memory but gave you the amount of Free Memory available. Would QB64 have something similar?

I did review the Variable_Types in the wiki and my confusion in determining the how much memory size is specifically how to calculate it. For example, if the variable is LONG, would the memory be set aside as ' M = _MemNew (2,147,483,647)? or if M is a REDIM array which starts out at hold 10 elements = 10 x 4 for long and therefore I would start to set aside M=_MemNew(40) and if that array grows by 10 new element each loop (20 x 4),then M now is M = _MemNew(80). So would if I have 8 gigs, setting the max and then freeing the max would be the way to go??

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 24, 2019, 11:54:11 am

REDIM arrays are a little tricky, as they free MEM handles automatically when you resize them, but all you have to do is make certain to point them at the resized array after.

REDIM MyArray(1000) AS LONG ‘4000 bytes in memory + a little overhead
DIM M AS _MEM
M = _MEM(MyArray()) ‘point to the array’s memory. No need to set aside any more for use.

REDIM _PRESERVE MyArray(10000) AS LONG ‘expand to 40000 bytes in memory
M = _MEM(MyArray()) ‘make certain to point to wherever the resized array is now at in memory.

Title: Re: Steve's Video Tutorials
Post by: Dimster on September 24, 2019, 12:12:46 pm

Oh, thanks Steve

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 24, 2019, 01:16:45 pm

Quote from: Dimster on September 24, 2019, 11:43:14 am

For example, if the variable is LONG, would the memory be set aside as ' M = _MemNew (2,147,483,647)? or if M is a REDIM array which

No, all you need to worry about is how many bytes are needed for each type, a LONG is 4 bytes in memory, not the max value each variable type can contain, LONG - max signed value= 2,147,483,647.
so
M = _MemNew (4)
would hold: 1-LONG, or 2-INTEGERs, or 4 _BYTEs.

If your machine has ~8gigs of ram I would use a conservative approach and say you could allocate 6.25 gigs max to a program, leaves memory for your OS, QB64, and other background programs that might need an amount of ram too.

M = _MemNew (6250000000) 'technically 6710886400 is 6.25GB
and that would allocate as much memory as I would conservative consider safe from crashing your program on a 64bit OS using 64bit QB64.

And Steve nails the Array part of your questions. And I think Steve's explanation of the rest is most likely better than anyway I could put it.

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 24, 2019, 01:17:29 pm

Hey Steve, what screen capture software are you using for your tutorials?

Title: Re: Steve's Video Tutorials
Post by: Petr on September 24, 2019, 01:30:03 pm

Hi Steve. Video quality is perfect. Could you show me in some next tutorials show us this themes?

How to use MEM to do the same as the following loop?

Dim A(1000) as integer
Dim B(1000) as long

For Object = 1 to 1000
For P = 1 to 1000
if a(object) = b(p) then ...
Next
Next

It is about how (in a fundamental way) to speed up the comparison of huge amounts of information in the field. I suppose it will certainly not be easy at all via MEM, especially if I use a TYPE field. Next step for fast speed is i know, is not using FOR... NEXT, but DO... LOOP. Another thing - How is memory organized when using TYPE? Is it in the order in which the individual array elements are defined? This would extremely speed up many of my programs.

I still didn't understand the oversizing of two-dimensional arrays via _PRESERVE. What about _MEM for two-dimensional arrays? Is the first value stored, then the second for one index, or is the first part of the array (Lbound, 1 to Ubound, 1) and then second (Lbound, 2 to Ubound, 2) stored?

Thank you.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 24, 2019, 02:05:05 pm

Quote from: Petr on September 24, 2019, 01:30:03 pm

Hi Steve. Video quality is perfect. Could you show me in some next tutorials show us this themes?

How to use MEM to do the same as the following loop?

Dim A(1000) as integer
Dim B(1000) as long

For Object = 1 to 1000
For P = 1 to 1000
if a(object) = b(p) then ...
Next
Next

It is about how (in a fundamental way) to speed up the comparison of huge amounts of information in the field. I suppose it will certainly not be easy at all via MEM, especially if I use a TYPE field. Next step for fast speed is i know, is not using FOR... NEXT, but DO... LOOP. Another thing - How is memory organized when using TYPE? Is it in the order in which the individual array elements are defined? This would extremely speed up many of my programs.

I still didn't understand the oversizing of two-dimensional arrays via _PRESERVE. What about _MEM for two-dimensional arrays? Is the first value stored, then the second for one index, or is the first part of the array (Lbound, 1 to Ubound, 1) and then second (Lbound, 2 to Ubound, 2) stored?

Thank you.

Simple solution here:

Dim A(1000) as integer
Dim B(1000) as long
DIM M AS _MEM, M1 AS _MEM
M = _MEM(A()): M1 = _MEM(B())

For Object = 0 to 1000
For P = 0 to 1000
‘if a(object) = b(p) then ...
IF _MEMGET(M, M.OFFSET + Object * M.ELEMENTSIZE, INTEGER) = _MEMGET(M1, M1.OFFSET + P * M1.ELEMENTSIZE, LONG) THEN...
Next
Next

(I’m writing the on my iPad, so I hope you can decipher that statement as it is, but if you need me to break it down and explain what each segment is doing, just let me know and I’ll expand on the illustration/details when I get home later this afternoon.)

As for multi-dimensional arrays, I’ll definitely wait to get to those until I’m at a PC where typing/explaining is a lot easier for me. They’re not the simplest to sort out, not without a few pieces of example code to go with them to help illustrate what we’re dealing with. ;)

Title: Re: Steve's Video Tutorials
Post by: Dimster on September 24, 2019, 02:06:25 pm

Very helpful Cobalt - thanks.

Title: Re: Steve's Video Tutorials
Post by: Petr on September 24, 2019, 02:12:50 pm

Oh so! Thank you Steve. As i see to this code, so ELEMENTSIZE can be used also for arrays created with TYPE without worrying about how many bytes in memory a particular element occupies?

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 24, 2019, 04:03:34 pm

Quote from: Petr on September 24, 2019, 02:12:50 pm

Oh so! Thank you Steve. As i see to this code, so ELEMENTSIZE can be used also for arrays created with TYPE without worrying about how many bytes in memory a particular element occupies?

Here's an example of how _MEM works with User Defined Types. See if it makes sense to you:

Code: [Select]

TYPE Any_Type
    X AS INTEGER
    Y AS INTEGER
    Junk AS STRING * 25
END TYPE

DIM array(100) AS Any_Type 'an array of our user type
DIM M AS _MEM 'a mem block
M = _MEM(array()) 'pointed to the array()

_MEMPUT M, M.OFFSET, 1001 AS INTEGER 'start at M.offset for byte 0
_MEMPUT M, M.OFFSET + 2, 9876 AS INTEGER '2 bytes over is where our integer is located
_MEMPUT M, M.OFFSET + 4, "Hello World" 'and 2 bytes over from there is where the string info starts at

PRINT array(0).X, array(0).Y, array(0).Junk

_MEMPUT M, M.OFFSET + M.ELEMENTSIZE, 5000 AS INTEGER 'array(1); same elements
_MEMPUT M, M.OFFSET + M.ELEMENTSIZE + 2, 1234 AS INTEGER
_MEMPUT M, M.OFFSET + M.ELEMENTSIZE + 4, "Goodbye World"
PRINT array(1).X, array(1).Y, array(1).Junk

temp$ = MKI$(7777) + MKI$(3333) + "What the what???" 'array(2) -- ALL AT ONCE
_MEMPUT M, M.OFFSET + M.ELEMENTSIZE * 2, temp$ 'notice how M.ElementSize helps us position ourselves to the start of the array element we're looking for?
PRINT array(2).X, array(2).Y, array(2).Junk

_MEMPUT M, M.OFFSET + M.ELEMENTSIZE * 3, "00" '256 written in ASCII characters, then 512 written in ASCII characters
_MEMPUT M, M.OFFSET + M.ELEMENTSIZE * 3 + 4, "Numeric values written as Strings?"
PRINT array(3).X, array(3).Y, array(3).Junk
PRINT array(4).X, array(4).Y, array(4).Junk 'notice how we put values into array element 3 AND 4 with the last print statement?  It was too large to fit in the STRING * 25 which I'd defined, so it carried over and corrupted the next array index.

If you notice, QB64 doesn't give a hoot about what we put into memory with _MEMPUT. (The last example should make that perfectly clear, I'd think.) When it comes to TYPEs, all it basically tracks for us is the size of the type, and where the elements start at. It's up to you to know your offsets and define them somehow for use in your programs (look at my QBDbase routines and you can see where they do that so we can use _MEM with whatever type a user has). When you _MEMGET or _MEMPUT, you specify what type of variable you expect to get and put into that position in memory, but there's no real requirement forcing you to only put/get a certain type.

Compare it to the following:

OPEN "temp.txt" for BINARY AS #1
a$ = chr$(1)+chr$(2)
PUT #1, 1, a$

DIM b as _UNSIGNED _BYTE
Get #1, 1, b
PRINT b
GET #1, 2, b
PRINT b

You put a string in your file, but then read it as 2 bytes...

You could also read that value as an integer, if you wanted: GET #1, 1, x%

Once it's in the file, your file doesn't care what the information is you stored. It just stores the information you put into it, where you decided to put it. _MEM works the exact same way.

Sometimes, you just have to manually track some of that information for yourself (espcially with user types). ;)

And in case the auto-format screws things up, this line should look like so:

_MEMPUT M, M.OFFSET + M.ELEMENTSIZE * 3, "00" '256 written in ASCII characters, then 512 written in ASCII characters

(That's CHR$(1) + "0" + CHR$(2) + "0", in case all else fails to format it properly.)

Title: Re: Steve's Video Tutorials
Post by: Petr on September 24, 2019, 04:59:52 pm

It is clear to me. Thank you, Steve.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 25, 2019, 11:40:21 am

Quote from: Cobalt on September 24, 2019, 01:17:29 pm

Hey Steve, what screen capture software are you using for your tutorials?

OBS Studio -- free, open source, and does all I'd ever want it to do -- and more! You can even configure it to do live broadcasts, if you want to be one of the internet sensations who uses live streams..

https://obsproject.com/

Simple enough that it didn't take me 5 minutes after installing, before I had it recording from my screen and camera, and without having to search a wiki or documentation to sort it out. I'm certain it can do a lot more than what I've did with it so far, but my needs are rather simple (record what my screen shows, or what my camera shows), and I figure I can learn more as I go. All software has a learning curve, but good software at least lets you do the very basics without needing to read a manual to get started.

One thing which I've found it doesn't do (or at least which I can't do with it yet), is video editing after the recording. If I stop a recording and then start after a small pause to let the dog outside, or such, it starts a new recording rather than appending to the end of the first one, so I have to then use a different piece of video editing software to merge those two mini-recordings into a single one before sharing.

Everything I saw recommended to use Blender as a video editing tool, but.... I can't even figure out how the BLEEP to load an existing video file into it to start with!! It's going to take some serious research into the software commands/set up to sort out WTH I'm doing with it. It's nowhere near as user friendly (from my experience), as OBS Studio is. For now, I'm just using VSCD Video Editor to splice segments together and such -- it doesn't seem to be that powerful of a tool, takes a long time to do its thing, but at least I can get it started, loading, and working! In time, I'll probably need to swap it out with something more powerful, but it works for my needs at the moment. ;)

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 25, 2019, 01:19:28 pm

Quote from: SMcNeill on September 25, 2019, 11:40:21 am

Everything I saw recommended to use Blender as a video editing tool, but.... I can't even figure out how the BLEEP to load an existing video file into it to start with!! It's going to take some serious research into the software commands/set up to sort out WTH I'm doing with it. It's nowhere near as user friendly (from my experience), as OBS Studio is. For now, I'm just using VSCD Video Editor to splice segments together and such -- it doesn't seem to be that powerful of a tool, takes a long time to do its thing, but at least I can get it started, loading, and working! In time, I'll probably need to swap it out with something more powerful, but it works for my needs at the moment. ;)

I may have an advantage there, as I have some knowledge of Blender. However you are correct, its video editing abilities have a severe learning curve. Blender Video Sequence Editor(VSE) should be considered more of a Movie editing software, I mean meant more for actual movie style videos. For simply splicing videos or fixing audio you will want a much simpler software. I use AVS Video Converter which has a very simple editor, though I don't think its free. Had my copy so long I can't remember, probably pulled it from google or some search engine as a bit torrent. But it allows splicing multiple videos together, adjusting audio levels, simple transition FX. Of course my version is old enough to run on VISTA so it might be more advanced on newer OSs.

Camera came today, see if I can get it to work with my antiquated hardware.

Unfortunately that OBS Studio doesn't like VISTA, but I was able to find an old one I thought I lost called Active Presenter, again works with VISTA. It has very basic editing although I think it is limited to a single video file source too, it does allow pausing and resuming without making separate files though.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 28, 2019, 04:53:24 am

Steve, You-Tube, so here I-Tube (sorry that the performance is ghastly but it seemed a good idea: it's the sentiment that counts).

Title: Re: Steve's Video Tutorials
Post by: Dimster on September 28, 2019, 11:32:48 am

That Bob Hope rendition broke all my heavy plastic wine glasses. But have to admit, Qwerkey does carry a tune better than me.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 28, 2019, 11:56:46 am

Quote from: Qwerkey on September 28, 2019, 04:53:24 am

Steve, You_Tube, so here I-Tube (sorry that the performance is ghastly but it seemed a good idea: it's the sentiment that counts).

Nice to meet you Qwerkey! You're more than welcome. I'm happy that folks are following along and (hopefully) learning something from these things. The next lesson is going to be "Finding the proper OFFSET of your memory/array.)", and I hope to get it up sometime in the next few days or so for everyone to enjoy. ;)

Title: Re: Steve's Video Tutorials
Post by: cohearn4 on September 28, 2019, 05:18:47 pm

Steve,

Thanks for your _MEM tutorials. The font size is good and clear. I just signed up so I could post, but I have been following you for a long time. Your explanations have helped me learn a tremendous amount with QB64. Again thanks.

Charles

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 29, 2019, 12:07:09 am

I'm starting work on Tutorial #3: Finding our Offsets right now, so I expect I'll probably start uploading the video to youtube tonight, and I'll share it tomorrow. For those who might be interested, or want a little study guide of what the tutorial will contain, here's basically what I expect this lesson will cover for us:

Code: QB64: [Select]

_TITLE "Locating Offsets of Memory."
'PART 1: Direct Variable Offsets
'Declarations:
DIM B AS _BYTE, I AS INTEGER, L AS LONG, I64 AS _INTEGER64
DIM m AS _MEM
 
'Pointing m at the various variables and putting information into them.
 
'Printing the variables.
 
'Simple enough, right?
 
'PART 2: The same, with basic arrays starting at BASE 0
'Declarations:
REDIM B(10) AS _BYTE, I(10) AS INTEGER, L(10) AS LONG, I64(10) AS _INTEGER64
'm is still our mem block as defined above
 
'Pointing m at the first element in the various arrays and putting information into them.
 
'Printing those first elements
 
'Pointing m at specific (non-first) elements in those arrays, and putting information into them.
 
'Printing those specific elements to make certain we put them where we thought we did.
 
'Still, nothing complex to this point.  Right?
 
'PART 3: Getting the offset values for oddly indexed arrays.
REDIM B(3 TO 10) AS _BYTE, I(-4 TO 10) AS INTEGER, L(-8 TO -5) AS LONG, I64(10 TO 100) AS _INTEGER64
'and m is still our mem block.
 
'Pointing m at the first element of these arrays.
 
'Printing the results for testing.
 
'Pointing m at a specific element of these arrays.
 
'Printing the results for testing.
 
'Things are getting a little more complex at this point.  Depending on time, I might end the
'tuturial with just this amount of information covered, so folks can make certain they follow
'along this far with no issues.
 
'Next up will be finding the offset for BASE 0 multi-dimensional arrays, such as:
REDIM MB(10, 10) AS _BYTE, MI(10, 10) AS INTEGER, ML(10, 10) AS LONG, MI64(10, 10) AS _INTEGER64
'and m is still our memblock for these tutorials, of course...
 
'Honestly, I expect this will be a good spot to stop, so look forward for an explaination of
'multi-dimensional offsets (and screen image offsets as well, while we're at it), in Tutorial
'4: Beyond the First Dimension!

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 29, 2019, 05:47:19 am

Tutorial #3 is up and available, and I have to admit: it’s much longer than I’d anticipated it would be. A few typos and mistakes extended things, and I got myself side-tracked and covered a bit more than intended in this lesson. I’d seriously considered editing out a bunch of it, but then decided not to, as even the mistakes and such can help show folks what to expect when things go wrong with using _MEM. (And some of the mistakes ARE purposeful, just for illustration purposes, but some of the others were just careless goofs which left me scratching my head and pondering, “Why the BLEEP didn’t that work as intended?”)

This tutorial is about 90 minutes long (I SHOT A MOVIE!!), so either watch it in segments, or else be certain to have plenty of popcorn and soda ready before sitting to watch it in a single go.

Quick Link to it is here:

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 29, 2019, 10:27:40 am

Technical Query No. 2

Steve, you have been teaching us dim folk what's happening with _MEM, and I now begin to see that _PUTting and _GETting to/from memory involves addressing individual bytes in a block of memory, each byte being 256 bits (0-255). And so we're only ever talking about numbers: the hardware never uses letters, of course, even if we're storing a string. So, when we store, say, a 4-letter string in _MEM this is four bytes next to each other, each byte storing the ASCII value of each letter.

I hope that I haven't got even this simple idea wrong!

Using your first video and adapting your code, I am now able to directly read a hard disk stored dictionary file into _MEM, and I have used the following code:

Code: QB64: [Select]

OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
NoWords& = LOF(1) / 4
 
DIM M AS _MEM
M = _MEMNEW(LOF(1))
 
$CHECKING:OFF
 
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    _MEMPUT M, M.OFFSET + ((N& - 1) * 4), ULoc$
NEXT N&
 
temp$ = SPACE$(4)
_MEMGET M, M.OFFSET, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + 4, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + 8, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + (NoWords& - 1) * 4, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + M.SIZE - 4, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + M.SIZE - 8, temp$
PRINT temp$
 
_MEMGET M, M.OFFSET + LOF(1) - 8, temp$
PRINT temp$
 
temp$ = SPACE$(5)
_MEMGET M, M.OFFSET + M.SIZE - 5, temp$
PRINT temp$
 
_MEMFREE M

NB This code is not usable: I have not supplied the file from which the data are read. It is illustrative only.

What surprises me is that nowhere in the code have I had to tell the computer to convert individual letters into their ASCII code.

In the statement DIM M AS _MEM, there is no indication that strings are going to be used (as opposed to numbers next to each other).

And in the statement _MEMPUT M, M.OFFSET + ((N& - 1) * 4), ULoc$, we are able to _PUT a string without converting to numbers. And in the statement _MEMGET M, M.OFFSET, temp$, the block of numbers is converted back to a string without having to code such an action.

Is it therefore the case that when you _MEMPUT or _MEMGET and your code has a string, QB64 just does the string to numeric byte conversions? I feel that this must be so, and if using "normal" string assignment, storing and retrieving, QB64 is always doing these conversions without the coder knowing anything about it.

I think that what I'm saying is that I'd have expected the _MEM dimensioning statement to be something like DIM M AS _MEM AS STRING, just to let the computer know that this will deal with strings, not numbers.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 29, 2019, 11:08:53 am

Steve, I think that I've managed to answer my own question. I tried adding 1 to one of the _MEM bytes of previous reply, and indeed the character (was "B") changed to "C".

Code:
_MEMPUT M, M.OFFSET + 1, _MEMGET(M, M.OFFSET + 1, _UNSIGNED _BYTE) + 1 AS _UNSIGNED _BYTE

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 29, 2019, 12:19:33 pm

Quote

I think that what I'm saying is that I'd have expected the _MEM dimensioning statement to be something like DIM M AS _MEM AS STRING, just to let the computer know that this will deal with strings, not numbers.

You're making your computer out to be smarter than it really is. ;)

When teaching my daughter how to code, the very first lesson I tried to stress with her is that computers aren't very smart -- no matter what people commonly think, nowadays. Computers aren't complicated because they're HARD; they're complicated because they're TOO SIMPLE. No matter what you think the computer is doing, all it's actually doing is adding or comparing 0s and 1s. EVERYTHING, in the end, breaks down to bits which are either ON or OFF. (-1 OR 0, as we represent them in QB64.)

The simple truth is: A computer doesn't use any numbers, nor any characters, nor strings. All your computer knows is ON and OFF state of bits. It's up to the PROGRAMMER to define what a sequence of bits represent -- either an integer (in various sizes), a real number (again, in various precision sizes), or a character/string.

As I was trying to stress in the first tutorial, _MEM is inherently nothing more than opening a file for binary access. Tell me, would you ever think of doing the following:

OPEN "temp.txt" FOR BINARY AS STRING , FILENUMBER 1

That OPEN statement simply says, "We want to access a storage area on the hard drive, and read and write data to it." We don't tell it that we want to work with strings with it, or numbers, or a strange combination of the two. We just issue a command that says, "We want to use a segment of the drive, call it "temp.txt", and get and put data to that segment."

_MEM, is more-or-less, a direct mirror to those BINARY access commands. Just as OPEN is the command to set the stage to interact with DATA on the hard drive, DIM m AS _MEM is doing the same job and setting the stage for us to interact with DATA somewhere in memory.

Let me give you a quick example to run and study, which helps highlight the issue:

Code: QB64: [Select]

OPEN "temp.txt" FOR OUTPUT AS #1 'This just opens a new file, erasing any old file with the same name.
CLOSE #1
OPEN "temp.txt" FOR BINARY AS #1 'And this now opens that blank file so I can get/put information to it.
text$ = "1234567890" 'a simple 10 character text string.
PUT #1, 1, text$ 'which I'm now writting to the disk.
 
'All very common practice, and nothing strange or mysterious going on so far.
'I opened a file, I put a string into it.
'But now....
 
'Let's start by making certain that the file contains the data I want:
in$ = SPACE$(10) ' I want a string 10 bytes long, so I can get all 10 bytes at once
GET #1, 1, in$ 'which I get from the drive
PRINT in$ 'and then print to see my string and make certain that I did, indeed, place string information to the drive.
 
SLEEP 'at this point, run the program and verify what the result is for yourself.
'Exit the program after the first pause (Where the SLEEP statement would appear), and then come back and follow the code
'as I comment it below.
 
'READY TO CONTINUE??
 
 
 
'Good!
 
'At this point, you can now see that we wrote a string to the drive...
'OR...
'Did we really???
 
'Let's test a few things:
 
DIM b AS _BYTE, i AS INTEGER, l AS LONG, i64 AS _INTEGER64 'let's first define a few variables as various types
DIM s AS SINGLE, d AS DOUBLE, f AS _FLOAT, c AS STRING * 1 'c for character, otherwise all our variable names are...
'        just the first letter of what they represent (with the exception of i64 for integer64, as i stands for integer)
 
GET #1, 1, b 'get a byte from the start of that file
GET #1, 1, i 'get an integer from the start of that file
GET #1, 1, l 'a long
GET #1, 1, i64 'an integer64
GET #1, 1, s 'a single
GET #1, 1, d 'a double
GET #1, 1, f ' a float
GET #1, 1, c 'and finish with a character
 
'Now surely, since we wrote the data as a string, these commands are going to fail and toss us an error.  Right??
 
'Let's try and print them, then run it and let's see!!
PRINT b, i, l, i64
PRINT s, d, f
PRINT c 'the character will print to the 3rd line, as it's the letter "1" and I don't want it lost in the numbers above.
 
SLEEP 'another pause, like before, so we can stop running the code at the proper spot as we study this example code.
'NOW, RUN THE PROGRAM AGAIN, THIS TIME STOPPING IT AT THE SECOND PAUSE (SLEEP) SEGMENT.
 
'Executed the code, yet?
 
'If so, were you shocked by the results?
 
'We wrote string data to the drive, but then somehow we managed to get it all back as numeric values???
'WTH is up with that??
 
 
'It's simply because it's as I was saying in my post over at the QB64 forums -- computers are much stupider than folks believe.
'All the computer does is set bits on the hard drive to be either ON or OFF, and then we GET and PUT those bits into specific
'areas of the hardrive with our OPEN and GET/PUT statement.
 
'OPEN defines a block of the drive that we're going to use...In this case from where "temp.txt" starts, to where "temp.txt"
'ends at.  I have no real idea where the heck that segment is; I trust the open statement to find the right spot on the
'drive and then make that access available for me.
 
'Then, once that segment of the drive is made available, I futher specify which area of that segment that I want to interact
'with, with the GET and PUT statement.
'GET FILEHANDLE, POSITION, INFORMATION....
'Or, as we commonly see it: GET #1, 1, variable  (for the first byte of that file)
 
'It's by defining our variable types that our computer knows how to interpet those 0 and 1 bits that we want to read/write
'to the drive.
 
'text$ = "1234567890"  <--this tells the computer to convert those 10 characters into the 0 and 1 representations of string values.
'DIM b as _BYTE <-- this tells the computer that those 0s and 1s are going to represent an integer value from -128 to 127
 
'The hard drive is ONLY storing bits which are either ON or OFF (0 or 1).
'It doesn't care what we put in our files -- it doesn't know the difference between a string or a number at all.
'All our hard drive knows is "This bit here is a 0.  This bit after it is a 1.  The next bit is a 0.  Then a 1.  Another 1...."
 
'MEM is the *EXACT* same thing, as I'll illustrate below:
 
DIM M AS _MEM 'just like that OPEN command, we're now preparing to open a segment of memory
M = _MEMNEW(10) 'in this case, I'm deciding it only needs to be 10 bytes of memory.
 
_MEMPUT M, M.OFFSET, text$ 'And, if you still remember our original text$, I just put it in memory, in that memblock.
in2$ = SPACE$(10) 'and here I'm setting a second in$ so I can get and print that information to the screen
_MEMGET M, M.OFFSET, in2$ 'and I just got in2$ from that spot in memory.
 
'Now, I don't think there's any real reason for a third SLEEP statement in this demo, so I'm just going to do as I did
'above and _MEMGET and _MEMPUT that same information into a series of variables:
 
DIM b2 AS _BYTE, i2 AS INTEGER, l2 AS LONG, i642 AS _INTEGER64 'the 2 here says they're all new variables, unrelated
DIM s2 AS SINGLE, d2 AS DOUBLE, f2 AS _FLOAT, c2 AS STRING * 1 'to the ones which I used before.
 
_MEMGET M, M.OFFSET, b2 'get a byte from the start of that mem block
_MEMGET M, M.OFFSET, i2 'an integer
_MEMGET M, M.OFFSET, l2 'long
_MEMGET M, M.OFFSET, i642 'integer64
_MEMGET M, M.OFFSET, s2 'single
_MEMGET M, M.OFFSET, d2 'double
$CHECKING:OFF
'I'm turning error checking off here, as the next line would technically toss an error for us.
'"WHY," you ask?
'Because we only have 10 bytes reserved for memory and a _FLOAT requires 32 of them.  (They're a HUGE variable type.)
'You can't make a blanket from a tablecloth, and you can't make a float from 10 bytes.
'(Well, at least you can't, without tossing an error -- which I'm just blindly ignoring for this demo and which
'COULD cause some serious corruption in my program if I wasn't careful. After all, who knows what variable/data is stored
'in the bytes immediately after the 10 where my memblock is mysteriously located??)
_MEMGET M, M.OFFSET, f2 'float
$CHECKING:ON
'Lesson to take from this:  ONLY USE CHECKING:OFF when you either: A) Know your code is 100% correct
'OR                                                                B) You seriously don't give a damn if you corrupt your data.
_MEMGET M, M.OFFSET, c2 'and finally, let's get that last character
 
PRINT b2, i2, l2, i642 'and now print the results to the screen
PRINT s2, d2, f2
PRINT c2
 
'RUN AND COMPARE WHAT YOU SEE......
 
'.
'..
'...
 
'No difference in the outputs?
 
 
'Shocked??
 
'If so, then don't be.  As I was stressing in my first video: _MEM is nothing more complex than working with a file opened
'FOR BINARY access.  GET/PUT gets and puts sequences of 0s and 1s onto our hard drive.
'                _MEMGET/_MEMPUT does the exact same thing, except it does it somewhere in memory.
 
'Now, as to whether that sequence of 0s and 1s is going to be integers, real numbers, strings, or some user-defined type
'which is a combination of all three...
'Our memory doesn't care about any of that!  All it does is faithfully hold a sequence of 0s and 1s in a specific order.
 
'It's up to us to define how we want to interpet that sequence of 0s and 1s.
'And we do that either via how we define our variable types:  DIM b AS _BYTE, i AS INTEGER
'Or we specify it specifically: _MEMPUT m, m.offset, 123 AS _BYTE
 
'Make sense now?
'Hopefully, the above little demo will clear up all your confusion for you.
'If not, just ask once again and let me know what/where you're lost, and I'll try and help you understand what's going on.  ;)
 

Study the code above and see if it helps answer your query for you. ;)

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 29, 2019, 12:28:44 pm

Quote from: Qwerkey on September 29, 2019, 11:08:53 am

Steve, I think that I've managed to answer my own question. I tried adding 1 to one of the _MEM bytes of previous reply, and indeed the character (was "B") changed to "C".

Code:
_MEMPUT M, M.OFFSET + 1, _MEMGET(M, M.OFFSET + 1, _UNSIGNED _BYTE) + 1 AS _UNSIGNED _BYTE

And this is often the very best way to learn these commands: Simply test them out and try various things with them and see how they behave. Trust that you can't actually break anything -- QB64 and your OS protects you from any serious issues such as destroying your computer. About the worst you can do is make your program crash, and can you name ANY programmer who hasn't already done that a couple of thousand times in their career??

When we crash, we just have to go in and debug whatever the hell went wrong. ;D

The best way to learn any new command in a programming language is simply to use it and learn from trial and error. If it doesn't behave as you expect it should, ask about it and then you'll either:
A) Have someone explain to you what's happening and why it happened
OR...
B) You find out that you discovered a bug in the command (and Lord knows, we have plenty of those), which can be addressed and hopefully corrected in a later version of the software.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 29, 2019, 01:10:24 pm

Thanks Steve. I went through your extra program and with my trial-and-error am begining to to get to grips with it. I'm still used to coding BASIC where you never have to think how to handle individual bytes. No wonder they made the acronym BASIC: for the people who don't actually know what they're doing! The story of my life, but, hey, I'm happy with that.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 30, 2019, 04:46:18 am

Next dim question (I've got a million of them!). As I mentioned above, I have the intention to produce a _MEM version of my Crossword Generator program which uses heaps of processing to produce a correctly filled crossword grid. Am I right in thinking that when I want to have fastest processing, all things that were "normal" variables get replaced by _MEM objects? So, for example

Code: QB64: [Select]

FOR N& = 1 to 1000000000
        Lots of _MEM processing
NEXT N&

even the N& variable would (somehow) be replaced by a _MEM object?

Or, using Steve's code in reply #21:

Code: QB64: [Select]

DO
    IF _MEMGET(m, m.OFFSET + o, _UNSIGNED LONG) = _RGBA32(255, 255, 255, 255) THEN
        _MEMPUT m, m.OFFSET + o, 256 AS _UNSIGNED _BYTE
    END IF
    o = o + 4
LOOP UNTIL o >= m.SIZE - 4

should the variable o be replaced by a _MEM object to speed things up? I'm confused by the mixing of "normal" variables (slow) and _MEM objects (fast).

Maybe things such as this will become clearly when Steve has completed the library of _MEM tutorials.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on September 30, 2019, 07:33:55 am

Default values of memory blocks.

With the code here, I was surprised to find that without line 11, the subsequent WHILE/WEND loop did not execute.

What I was trying to do was to index the memory block byte position of object M with a one-byte object MCount whose value is increased from the starting value by one at each loop cycle.

I originally found that the MCount starting value was 23 and on other runs it was 21. So line 11 sets the starting value of MCount to be zero. Thinking that it would behave like a variable, I assumed that its starting value be zero. But as the program allocates a block of memory, the starting value is just what is in that part of the (unused) memory at that time???

Code: QB64: [Select]

DIM M AS _MEM
M = _MEMNEW(20)
 
Junk$ = "ABCDEFGHIJKLMNOPQRST"
 
_MEMPUT M, M.OFFSET, Junk$
 
DIM MCount AS _MEM
MCount = _MEMNEW(1)
 
_MEMPUT MCount, MCount.OFFSET, 0 AS _UNSIGNED _BYTE
 
PRINT _MEMGET(MCount, MCount.OFFSET, _UNSIGNED _BYTE)
 
WHILE _MEMGET(MCount, MCount.OFFSET, _UNSIGNED _BYTE) < 20
 
    PRINT _MEMGET(MCount, MCount.OFFSET, _UNSIGNED _BYTE); _MEMGET(M, M.OFFSET + _MEMGET(MCount, MCount.OFFSET, _UNSIGNED _BYTE), STRING * 1)
 
    _MEMPUT MCount, MCount.OFFSET, _MEMGET(MCount, MCount.OFFSET, _UNSIGNED _BYTE) + 1 AS _UNSIGNED _BYTE
 
WEND
 
_MEMFREE M
_MEMFREE MCount

Oh, by the way, mucking around with actual blocks of memory is real fun!

A further aside: Steve, you have trouble remembering that the command is _MEMFREE rather than _FREEMEM (cf. _FREEIMAGE etc.). Here's a little aide memoire for you, but you'll need to be familiar with the British TV comedy programme "Are You Being Served?". I don't suppose that this ever reached the US, but if it did and you ever saw it, just remember the Mr Humphries character and his catchphrase - I'm Free! (to be said a very camp fashion). And then keep saying "Mem Free!". I cannot ever get this command wrong.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on September 30, 2019, 10:24:32 am

Quote from: Qwerkey on September 30, 2019, 04:46:18 am

Next dim question (I've got a million of them!). As I mentioned above, I have the intention to produce a _MEM version of my Crossword Generator program which uses heaps of processing to produce a correctly filled crossword grid. Am I right in thinking that when I want to have fastest processing, all things that were "normal" variables get replaced by _MEM objects? So, for example
Code: QB64: [Select]
FOR N& = 1 to 1000000000
Lots of _MEM processing
NEXT N&
even the N& variable would (somehow) be replaced by a _MEM object?

Or, using Steve's code in reply #21:
Code: QB64: [Select]
DO
IF _MEMGET(m, m.OFFSET + o, _UNSIGNED LONG) = _RGBA32(255, 255, 255, 255) THEN
_MEMPUT m, m.OFFSET + o, 256 AS _UNSIGNED _BYTE
END IF
o = o + 4
LOOP UNTIL o >= m.SIZE - 4
should the variable o be replaced by a _MEM object to speed things up? I'm confused by the mixing of "normal" variables (slow) and _MEM objects (fast).

Maybe things such as this will become clearly when Steve has completed the library of _MEM tutorials.

Directly accessing a variable like o isn’t that much of a bottleneck in performance; it’s a fairly straightforward process. Using a command like POINT is often quite slow, as it has a lot of internal error checking, and value translating going on. You’ll see different amounts of improvement for various commands/functions which you can swap out to _MEM usage, so it’s hard to say how much you can improve performance with any single change.

_MEMGET (m, m.OFFSET + o, _UNSIGNED LONG) <— This is the equivalent of a POINT(x, y) command, so even by looking up the value of o and adding it to m.OFFSET, it’s still much faster than looking up x, looking up y, looking at _SOURCE, checking to make certain everything is valid, looking at the point value, reading it as a _FLOAT, and then converting it to an _UNSIGNED LONG when returning it...

Now, would it have been even faster to use _MEMPUT/_MEMGET instead of o?

Probably. I doubt it’d be much faster, but it probably would end up slightly faster. The question is, would it be worth the loss to readability and increase in coding time and complexity to replace it?

Instead of the above, we’d see things like: (where we DIM m2 AS _MEM: m2 = _MEMNEW(4), rather than using o)

_MEMGET (m, m.OFFSET + _MEMGET(m2, m2.OFFSET, LONG), LONG)

Then, instead of o = o + 4, we’d have:

_MEMPUT m2, m2.OFFSET, (_MEMGET(m2, m2.OFFSET, LONG) +4) AS LONG

It might be a nanosecond faster executing, but is it worth it for the increase in coding time and complexity of understanding the code?

Only the programmer can answer that question, and the answer might vary for different programs. (Or even different processes inside a program.) You’d want a recursive SORT process to run as fast as possible, as it can definitely become a program bottleneck, but a process which only adds 1 to each element of an Array isn’t going to be much of a concern.

It’s a trade off: Speed vs readability and complexity, with increased coding time. Onlyyou can determine when it’s worth the trouble for your program’s needs. ;)

Title: Re: Steve's Video Tutorials
Post by: Cobalt on September 30, 2019, 10:38:00 am

Quote from: Qwerkey on September 29, 2019, 10:27:40 am

memory involves addressing individual bytes in a block of memory, each byte being 256 bits (0-255). And so we're only ever talking about

Just wanted to say, each BYTE is 8bits, capable of representing a VALUE of 0-255.

Quote from: SMcNeill on September 29, 2019, 12:19:33 pm

You're making your computer out to be smarter than it really is. ;)

Yep, Computers can only do what they are told and at the fundamental level they only know ON and OFF (1 an 0)[I think you can say they only know voltage hi(+5v~) and voltage lo(~0.5v].

That is why we are supposed to be the USER and not the other way around. And Knowledge is Power!

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on October 01, 2019, 09:29:44 am

"_MEM processing is 100's of times faster than with standard variable processing". This is what the gurus said.

So, before plummeting into adapting my two programs (Crossword Generator and Gravitation Simulator) which use a lot of processing, I thought that I'd check out this statement.

Code Trial 1A loads a file from disk into a _MEM object and then intensely manipulates that _MEM object (swaps bytes around). In my core-I5 processor this uses a whole processor's-worth of CPU and the process takes 69.1s. With $CHECKING:OFF it takes 25.4s.

Code Trial 1B loads the same file into a string, and the same manipulation as in Trial 1A is performed using standard techniques (no _MEM). The process takes 253s. So the _MEM processing is faster, but not by a large factor.

So I thought that maybe the process in Trial 1A was slowed down by the presence of standard variables to do the indexing. Code Trial 1C is as 1A but with everything put into _MEM. I was shocked to find that this took 157s.

An invariant maxim is "never believe gurus unless you know what they say is true" (particularly pertinent to mythologies, of course).

So, either the gurus were wrong or I don't know what I'm doing, which as all members will readily attest is totally without foundation.

Tria 1A

Code: QB64: [Select]

'Trial 1A _MEM processing, Variable Indexing
 
'Load from file into _MEM object
OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
 
PRINT LOF(1)
NoWords& = LOF(1) / 4
 
DIM M AS _MEM
M = _MEMNEW(LOF(1))
PRINT M.SIZE '/ 2147483647
 
DIM MDum AS _MEM
MDum = _MEMNEW(1)
 
$CHECKING:OFF
 
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    _MEMPUT M, M.OFFSET + ((N& - 1) * 4), ULoc$
NEXT N&
 
CLOSE #1
 
'Now do some manipulations in _MEM
 
Start! = TIMER
 
FOR N& = 1 TO 100000
    'for K& = 0 to  M.SIZE -1  !!! Not allowed
    FOR K& = 0 TO 4 * NoWords& - 1
        _MEMPUT MDum, MDum.OFFSET, _MEMGET(M, M.OFFSET + (4 * NoWords& - 1 - K&), _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        _MEMPUT M, M.OFFSET + (4 * NoWords& - 1 - K&), _MEMGET(M, M.OFFSET + K&, _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        _MEMPUT M, M.OFFSET + K&, _MEMGET(MDum, MDum.OFFSET, _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
    NEXT K&
NEXT N&
 
PRINT TIMER - Start!
 
END
 

Trial 1B

Code: QB64: [Select]

'Trial 1B Variable processing, Variable Indexing
 
OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
 
PRINT LOF(1)
NoWords& = LOF(1) / 4
 
S$ = ""
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    S$ = S$ + ULoc$
NEXT N&
 
PRINT LEN(S$)
 
CLOSE #1
 
'Now do some manipulations with the array - same as Trial 1A
 
Start! = TIMER
 
FOR N& = 1 TO 100000
    FOR K& = 1 TO LEN(S$)
        Dum$ = MID$(S$, K&, 1)
        MID$(S$, K&, 1) = MID$(S$, LEN(S$) - K&, 1)
        MID$(S$, LEN(S$) - K&, 1) = Dum$
    NEXT K&
NEXT N&
 
PRINT TIMER - Start!
 
END
 

Trial 1C

Code: QB64: [Select]

'Trial 1C _MEM processing, _MEM Indexing
 
OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
 
PRINT LOF(1)
NoWords& = LOF(1) / 4
 
'M Holds the data
DIM M AS _MEM
M = _MEMNEW(LOF(1))
PRINT M.SIZE
 
'MSize 4-byte _MEM holds size of M
DIM MSize AS _MEM
MSize = _MEMNEW(4)
_MEMPUT MSize, MSize.OFFSET, LOF(1) AS _UNSIGNED LONG
PRINT _MEMGET(MSize, MSize.OFFSET, _UNSIGNED LONG)
 
'MDum 1-byte _MEM used to temporarily hold swapped byte
DIM MDum AS _MEM
MDum = _MEMNEW(1)
 
'$CHECKING:OFF
 
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    _MEMPUT M, M.OFFSET + ((N& - 1) * 4), ULoc$
NEXT N&
 
CLOSE #1
 
'N 4-byte _MEM to index count
DIM N AS _MEM
N = _MEMNEW(4)
_MEMPUT N, N.OFFSET, 0 AS _UNSIGNED LONG
 
PRINT _MEMGET(N, N.OFFSET, _UNSIGNED LONG)
 
'K 4-byte _MEM used to index swap
DIM K AS _MEM
K = _MEMNEW(4)
_MEMPUT K, K.OFFSET, 0 AS _UNSIGNED LONG
 
PRINT _MEMGET(K, K.OFFSET, _UNSIGNED LONG)
 
'Now do some manipulations in _MEM
 
Start! = TIMER
 
WHILE _MEMGET(N, N.OFFSET, _UNSIGNED LONG) < 100000 'Don't know how to do FOR/NEXT with _MEM objects
 
    _MEMPUT K, K.OFFSET, 0 AS _UNSIGNED LONG
    WHILE _MEMGET(K, K.OFFSET, _UNSIGNED LONG) < _MEMGET(MSize, MSize.OFFSET, _UNSIGNED LONG)
 
        _MEMPUT MDum, MDum.OFFSET, _MEMGET(M, M.OFFSET + (_MEMGET(MSize, MSize.OFFSET, _UNSIGNED LONG) - _MEMGET(K, K.OFFSET, _UNSIGNED LONG) - 1), _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        _MEMPUT M, M.OFFSET + (_MEMGET(MSize, MSize.OFFSET, _UNSIGNED LONG) - _MEMGET(K, K.OFFSET, _UNSIGNED LONG) - 1), _MEMGET(M, M.OFFSET + _MEMGET(K, K.OFFSET, _UNSIGNED LONG), _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        _MEMPUT M, M.OFFSET + _MEMGET(K, K.OFFSET, _UNSIGNED LONG), _MEMGET(MDum, MDum.OFFSET, _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
 
        _MEMPUT K, K.OFFSET, _MEMGET(K, K.OFFSET, _UNSIGNED LONG) + 1 AS _UNSIGNED LONG
 
    WEND
 
    _MEMPUT N, N.OFFSET, _MEMGET(N, N.OFFSET, _UNSIGNED LONG) + 1 AS _UNSIGNED LONG
 
WEND
 
PRINT TIMER - Start!
 
_MEMFREE M
_MEMFREE N
_MEMFREE K
_MEMFREE MSize
_MEMFREE MDum
 
END
 

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 01, 2019, 10:14:41 am

Quote

With $CHECKING:OFF it takes 25.4s....

...The process takes 253s. So the _MEM processing is faster, but not by a large factor.

From 253 seconds to 25.4 seconds is 10x faster already, and that’s not a “large factor”?

Also, don’t forget, this guru also told you:

Quote

You’ll see different amounts of improvement for various commands/functions which you can swap out to _MEM usage, so it’s hard to say how much you can improve performance with any single change.

And, I’m thinking that you can tweek that code to speed it up more, with just a little work.

M.OFFSET + (4 * NoWords& - 1 - K&) <— there’s a good bit of calculating going on in this statement, and it’s used multiple times. Assigning it once to a different offset would probably be faster....

And, unless I’m misreading what the code is supposed to do, you’re just swapping contents of your strings?

FOR K& = 1 TO LEN(S$)
Dum$ = MID$(S$, K&, 1) <— a temp$
MID$(S$, K&, 1) = MID$(S$, LEN(S$) - K&, 1) <— move info from one spot to another
MID$(S$, LEN(S$) - K&, 1) = Dum$ <— place info back into that original spot from the temp$
NEXT K&

So far, you’re seeing an improvement of 10x to the time, and you still haven’t learned or used _MEMCOPY, Which is the optimized way to do this sort of process. http://www.qb64.org/wiki/MEMCOPY

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on October 01, 2019, 10:35:00 am

O Great Master, wow! That was quick! I only just popped out for drink.

No, you're not misreading what I was doing in the trial and yes I haven't managed to come across _MEMCOPY (will look into it). But I just wanted to compare exactly the same process without / with _MEM in the trial.

Quote from: SMcNeill on October 01, 2019, 10:14:41 am

M.OFFSET + (4 * NoWords& - 1 - K&) <— there’s a good bit of calculating going on in this statement, and it’s used multiple times. Assigning it once to a different offset would probably be faster....

In Trial 1C, I replaced that calculated stuff with a constant _MEM object and this trial was very much slower. Maybe I just did something wrong there.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 01, 2019, 01:58:46 pm

Just curious; How's this little version perform for you?

Code: QB64: [Select]

'Trial 1A _MEM processing, Variable Indexing
 
'Load from file into _MEM object
OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
 
PRINT LOF(1)
NoWords& = LOF(1) / 4
 
DIM M AS _MEM
M = _MEMNEW(LOF(1))
PRINT M.SIZE '/ 2147483647
 
DIM MDum AS _MEM
MDum = _MEMNEW(1)
 
$CHECKING:OFF
 
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    _MEMPUT M, M.OFFSET + ((N& - 1) * 4), ULoc$
NEXT N&
 
CLOSE #1
 
'Now do some manipulations in _MEM
 
DIM Precalc AS _OFFSET
 
Start! = TIMER
Precalc = M.OFFSET + 4 * NoWords& - 1 'Move as much of your math outside the loop as possible
 
FOR N& = 1 TO 100000
    'for K& = 0 to  M.SIZE -1  !!! Not allowed
    'You can't mix OFFSETs with other values, as above with that FOR statement.
    'Instead, convert it into a simple DO sequence, such as below.
    K& = 0
    DO
        _MEMPUT MDum, MDum.OFFSET, _MEMGET(M, Precalc - K&, _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        _MEMPUT M, Precalc - K&, _MEMGET(M, M.OFFSET + K&, _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        _MEMPUT M, M.OFFSET + K&, _MEMGET(MDum, MDum.OFFSET, _UNSIGNED _BYTE) AS _UNSIGNED _BYTE
        K& = K& + 1
    LOOP UNTIL K& >= M.SIZE
NEXT N&
$CHECKING:ON
PRINT TIMER - Start!
 
END

You mentioned that it was taking you 25.4s to run the code. All I've done here is precalculate the math and simplify the calculations inside the loop a bit, in an effort to showcase how much of a difference they can make sometimes. On my PC, the runtime was about 13 seconds, and after simplifying the math by calculating the values that won't change outside the loop, it dropped runtime down to about 11 seconds. A 15% performance increase, with a very minor alteration completely unrelated to our mem commands at all. ;)

If it affects your PC with a similar change, it should reduce times down from 25 seconds to what? About 21 seconds?

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 01, 2019, 02:18:33 pm

And, completely unrelated to the _MEM commands, I thought I'd take a moment and also showcase something else to help highlight just exactly how SLOOOOOOOOW the QB64 string commands are.

First, run this piece of your original code and time it. (Take a nap, it takes a while, coming in at almost 3 minutes to execute on my PC.)

This is the exact same as TRIAL 1B above:

Code: QB64: [Select]

'Trial 1B Variable processing, Variable Indexing
 
OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
 
PRINT LOF(1)
NoWords& = LOF(1) / 4
 
S$ = ""
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    S$ = S$ + ULoc$
NEXT N&
 
PRINT LEN(S$)
 
CLOSE #1
 
'Now do some manipulations with the array - same as Trial 1A
 
Start! = TIMER
 
FOR N& = 1 TO 100000
    FOR K& = 1 TO LEN(S$)
        Dum$ = MID$(S$, K&, 1)
        MID$(S$, K&, 1) = MID$(S$, LEN(S$) - K&, 1)
        MID$(S$, LEN(S$) - K&, 1) = Dum$
    NEXT K&
NEXT N&
 
PRINT TIMER - Start!
 
END

Now, I'm going to rewrite the routine to work with numeric values and ASCII values, rather than with strings. Compile and test the following for its runtime:

Code: QB64: [Select]

'Trial 1B Variable processing, Variable Indexing
 
OPEN "4UKEng.rnd" FOR RANDOM AS #1 LEN = 4
FIELD #1, 4 AS ULoc$
 
PRINT LOF(1)
NoWords& = LOF(1) / 4
 
S$ = ""
FOR N& = 1 TO LOF(1) / 4
    GET #1, N&
    S$ = S$ + ULoc$
NEXT N&
 
PRINT LEN(S$)
 
CLOSE #1
 
'Now do some manipulations with the array - same as Trial 1A
 
DIM B AS _BYTE
L = LEN(S$) + 1
 
Start! = TIMER
 
FOR N& = 1 TO 100000
    FOR K& = 1 TO LEN(S$)
        'Dum$ = MID$(S$, K&, 1)
        B = ASC(S$, K&)
        'MID$(S$, K&, 1) = MID$(S$, LEN(S$) - K&, 1)
        ASC(S$, K&) = ASC(S$, L - K&)
        'MID$(S$, LEN(S$) - K&, 1) = Dum$
        ASC(S$, L - K&) = B
    NEXT K&
NEXT N&
 
PRINT TIMER - Start!
 
END

Now, just by swapping over to ASC, instead of MID$, the runtime on my PC has now dropped down to less than 1 minute. (56 seconds.)

From 187 seconds (MID$) to 56 seconds (ASC), to 11 seconds (_MEM)...

Which is why I said it's hard to tell folks, "You're going to always see a performance boost of XX percent with _MEM." Various commands have different levels of overhead to them, so it's impossible to say how much of a performance boost you'll see, until you can actually compare routines side-by-side.

In this case, the original code took 187 seconds, the _MEM version took 11 seconds, so that's a performance increase by 17x... Which, I think really helps to highlight exactly why it's nice for QB64 programmers to have the ability to use _MEM inside their code. ;)

Title: Re: Steve's Video Tutorials
Post by: Dimster on October 01, 2019, 04:02:24 pm

Steve, in your _MEMPUT examples you have been using whole numbers - ie _MEMPUT m, m.OFFSET, 986543.
Would _MEMPUT also calculate the value? for example _MEMPUT m, m.OFFSET, 98000 + 6543

or is it better coding practice to have the value calculated before the PUT such as V1 = 98000 + 6543 : _MEMPUT m, m.OFFSET, V1 ?

Really liking your videos : finished #3 with popcorn to spare.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 01, 2019, 04:27:38 pm

Quote from: Dimster on October 01, 2019, 04:02:24 pm

Steve, in your _MEMPUT examples you have been using whole numbers - ie _MEMPUT m, m.OFFSET, 986543.
Would _MEMPUT also calculate the value? for example _MEMPUT m, m.OFFSET, 98000 + 6543

or is it better coding practice to have the value calculated before the PUT such as V1 = 98000 + 6543 : _MEMPUT m, m.OFFSET, V1 ?

Really liking your videos : finished #3 with popcorn to spare.

_MEMPUT m, m.OFFSET, 98000 + 6543 would work just fine, as long as you remember to tell it what type that numeric value represents. (_MEMPUT m, m.OFFSET, 98000 + 6543 AS LONG, or _MEMPUT m, m.OFFSET, 98000 + 6543 AS _FLOAT, for example)

Since you generally only worry about the extra typing and complexity when you need speed, it’s usually a good idea to calculate as much as is feasible outside any MEM loop, such as I was showing a few posts above. Referencing a normal variable takes a sliver of time, but there comes a point where you can reference it quicker than you can do calculations without it.

FOR I = 1 TO 1000000
PRINT 1+2+3+4+5+6+7+8+9+10+11+12+13+14+15+16+17+18+19+20+21+22+23+24+25
NEXT

Temp = 1+2+3+4+5+6+7+8+9+10+11+12+13+14+15+16+17+18+19+20+21+22+23+24+25
FOR I = 1 TO 1000000
PRINT Temp
NEXT

Given the two routines above, which is faster? ;)

PRINT 1 would probably be faster than PRINT Temp.
PRINT 1+2 might be faster than PRINT Temp.
But I seriously doubt PRINT 1+...25, as above, is faster.

So, for your example (V1 = 98000 + 6543 : _MEMPUT m, m.OFFSET, V1), if that V1 is calculated outside your MEM loop, then it might be better (as in faster) than doing the math and calculations with the numeric values.

If calculated inside the MEM loop, you just created a little extra work for the routine to process and look up, before finishing. ;)

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on October 02, 2019, 01:16:28 pm

Steve, thanks for your guidance above on achieving the best balances for speed when converting to _MEM. For my trials, I chose something simple but related to what needs to happen in my Crossword Generator program (where all complicated numerical manipulations are, of course, pre-calculated for that extra bit of speed). Now that I'm beginning to get a slight grip of the _MEM environment (I've even used _MEMCOPY now), I'll carry out some speed comparisons within the Crossword Generator program to see what works best for speed. Will keep you in touch (if all this wittering hasn't put you to sleep).

Title: Re: Steve's Video Tutorials
Post by: _vince on October 03, 2019, 10:42:01 am

Quote from: SMcNeill on September 30, 2019, 10:24:32 am

It’s a trade off: Speed vs readability and complexity, with increased coding time. Onlyyou can determine when it’s worth the trouble for your program’s needs. ;)

Perhaps _MEMx should never be used for anything unless absolutely necessary. It's a kind of dirty hack that was thrown in to make DECLARE LIBRARY viable and probably wasn't meant for anything else. Some would say mem pointers are certainly not BASIC. FreeBASIC for instance implements full blown C-style pointers which are a lot more convenient than _MEMx but most would argue that they're certainly not BASIC and I remember Galleon resented that and did not want it for QB64. I'd love to hear your opinion on the matter, Steve.

You could also talk about using _MEMx where there isn't a really good alternative, ie lists, trees, dynamic memory, etc, see my example below. Anyways, nice tutorials, Steve, I've watched them all. I'd love to see graphic demo videos, like trying to implement some kind of simple demo/animation/pattern/game and just talking through your thinking process and so on. Just a suggestion ;-)

Code: QB64: [Select]

defint a-z
type nodeType
        str as _mem
        strLen as integer
 
        n as _mem
        p as _mem
end type
 
type listType
        head as _mem
        tail as _mem
 
        cur as _mem
 
        cx as integer
        cy as integer
        scroll as integer
        scrollmax as integer
end type
 
cls
system
 
sub addNodeNext (new as _mem, s$, cur as _mem, list1 as listType)
        dim node as nodeType
        dim temp as _mem
        dim n as _mem
 
        list1.scrollmax = list1.scrollmax + 1
 
        temp = _memnew(len(node))
 
        nextNode n, cur
 
        node.strLen = len(s$)
        if node.strLen > 0 then 
                node.str = _memnew(len(s$))
                _memput node.str, node.str.offset, s$
        end if
        node.n = n
        node.p = cur
        _memput temp, temp.offset, node
 
        node = _memget(cur, cur.offset, nodeType)
        node.n = temp
        _memput cur, cur.offset, node
 
        node = _memget(n, n.offset, nodeType)
        node.p = temp
        _memput n, n.offset, node
 
        new = temp
end sub
 
sub addNodePrev (new as _mem, s$, cur as _mem, list1 as listType)
        dim node as nodeType
        dim temp as _mem
        dim p as _mem
 
        list1.scrollmax = list1.scrollmax + 1
        temp = _memnew(len(node))
 
        prevNode p, cur
 
        node.strLen = len(s$)
        if node.strLen > 0 then 
                node.str = _memnew(len(s$))
                _memput node.str, node.str.offset, s$
        end if
 
        node.n = cur
        node.p = p
        _memput temp, temp.offset, node
 
        node = _memget(cur, cur.offset, nodeType)
        node.p = temp
        _memput cur, cur.offset, node
 
        node = _memget(p, p.offset, nodeType)
        node.n = temp
        _memput p, p.offset, node
 
        new = temp
end sub
 
sub rmNode (cur as _mem, list1 as listType)
        dim node as nodeType
        dim n as _mem
        dim p as _mem
 
        list1.scrollmax = list1.scrollmax - 1
        'remove the string first
        node = _memget(cur, cur.offset, nodeType)
        if node.strLen > 0 then
                _memfree node.str
        end if
 
        nextNode n, cur
        prevNode p, cur
 
        node = _memget(p, p.offset, nodeType)
        node.n = n
        _memput p, p.offset, node
 
        node = _memget(n, n.offset, nodeType)
        node.p = p
        _memput n, n.offset, node
 
        _memfree cur
end sub
 
sub nextNode (new as _mem, old as _mem)
        dim node as nodeType
 
        node = _memget(old, old.offset, nodeType)
        new = node.n
end sub
 
sub prevNode (new as _mem, old as _mem)
        dim node as nodeType
 
        node = _memget(old, old.offset, nodeType)
        new = node.p
end sub
 
function lenNode (cur as _mem)
        dim node as nodeType
 
        node = _memget(cur, cur.offset, nodeType)
        lenNode = node.strLen
end function
 
function readNode$ (cur as _mem)
        dim node as nodeType
 
        node = _memget(cur, cur.offset, nodeType)
        if node.strLen = 0 then
                readNode$ = ""
                exit function
        end if
        s$ = string$(node.strLen, 0)
        _memget node.str, node.str.offset, s$
 
        readNode$ = s$
end sub
 
sub writeNode (cur as _mem, s$)
        dim node as nodeType
 
        'remove old string, free memory
        node = _memget(cur, cur.offset, nodeType)
        if node.strLen > 0 then _memfree node.str
 
        'add new string
        node.strLen = len(s$)
        if node.strLen > 0 then
                node.str = _memnew(len(s$))
                _memput node.str, node.str.offset, s$
        end if
        _memput cur, cur.offset, node
end sub
 
sub newList (new as listType)
        dim node as nodeType
 
        new.head = _memnew(len(node))
        new.tail = _memnew(len(node))
        new.cx = 0
        new.cy = 0
        new.scroll = 0
        new.scrollmax = 0
 
        s$ = "head"
        node.strLen = len(s$)
        node.str = _memnew(len(s$))
        node.n = new.tail
        node.p = new.tail
        _memput node.str, node.str.offset, s$
        _memput new.head, new.head.offset, node
 
        s$ = "tail"
        node.strLen = len(s$)
        node.str = _memnew(len(s$))
        node.n = new.head
        node.p = new.head
        _memput node.str, node.str.offset, s$
        _memput new.tail, new.tail.offset, node
 
end sub
 
sub printList (cur as listType)
        dim temp as _mem
 
        nextNode temp, cur.head
        do 
                if temp.offset = cur.tail.offset then exit do
 
                print readNode$ (temp)
                nextNode temp, temp
        loop
end sub
 
sub rmList (cur as listType)
        dim temp as _mem
        dim temp2 as _mem
 
        nextNode temp, cur.head
        do 
                if temp.offset = cur.tail.offset then exit do
                
                temp2 = temp
                nextNode temp, temp2
                rmNode temp2, cur
        loop
 
        rmNode cur.head, cur
        rmNode cur.tail, cur
end sub
 

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on October 04, 2019, 06:17:33 am

Quote from: _vince on October 03, 2019, 10:42:01 am

Perhaps _MEMx should never be used for anything unless absolutely necessary. It's a kind of dirty hack that was thrown in to make DECLARE LIBRARY viable and probably wasn't meant for anything else.

Gosh! That was a bit of a surprise. And taking Steve's comments on board, I thought that I'd look further into _MEM for my application - Crossword Generator. In that program, I need to do lots of comparisons of letters at different places along a string (the new tried word) and at different places in a grid (it's a crossword!). In my existing program, INSTR() is used a lot, for example.

So, I've written some more test code closer to what this program needs.

Code: QB64: [Select]

'Comparisons of string, numeral and _MEM processing (cf. Crossword Generator)
 
CONST False = 0, True = NOT False
DIM EWord%%(9), WWord%%(9)
 
E$ = "**********"
W$ = "@@@@@@@@@@"
FOR P%% = 0 TO 9
    EWord%%(P%%) = ASC(E$, P%% + 1)
    WWord%%(P%%) = ASC(W$, P%% + 1)
NEXT P%%
DIM EMem AS _MEM
EMem = _MEMNEW(10)
_MEMPUT EMem, EMem.OFFSET, E$
DIM WMem AS _MEM
WMem = _MEMNEW(10)
_MEMPUT WMem, WMem.OFFSET, W$
 
Start! = TIMER
'Check the new word fits existing crossing words
FOR N& = 1 TO 100000000
    P%% = 0
    WHILE P%% < 10
        IF _MEMGET(EMem, EMem.OFFSET + P%%, _UNSIGNED _BYTE) <> 41 AND _MEMGET(EMem, EMem.OFFSET + P%%, _UNSIGNED _BYTE) <> _MEMGET(WMem, WMem.OFFSET + P%%, _UNSIGNED _BYTE) THEN
            'Nothing doing
            P%% = P%% + 1
        ELSE
            P%% = P%% + 1
        END IF
    WEND
NEXT N&
PRINT "Mem", TIMER - Start!, P%%
Start! = TIMER
'Check the new word fits existing crossing words
FOR N& = 1 TO 100000000
    P%% = 0
    WHILE P%% < 10
        IF EWord%%(P%%) <> 41 AND EWord%%(P%%) <> WWord%%(P%%) THEN
            'Nothing doing
            P%% = P%% + 1
        ELSE
            P%% = P%% + 1
        END IF
    WEND
NEXT N&
PRINT "Num", TIMER - Start!, P%%
Start! = TIMER
'Check the new word fits existing crossing words
FOR N& = 1 TO 100000000
    P%% = 0
    WHILE P%% < 10
        IF MID$(E$, P%%, 1) <> "@" AND MID$(E$, P%%, 1) <> MID$(W$, P%%, 1) THEN
            'Nothing doing
            P%% = P%% + 1
        ELSE
            P%% = P%% + 1
        END IF
    WEND
NEXT N&
PRINT "Char", TIMER - Start!, P%%
Start! = TIMER
'Check the new word fits existing crossing words
FOR N& = 1 TO 100000000
    P%% = 0
    WHILE P%% < 10
        IF _MEMGET(EMem, EMem.OFFSET + P%%, _UNSIGNED _BYTE) <> 41 AND _MEMGET(EMem, EMem.OFFSET + P%%, _UNSIGNED _BYTE) <> _MEMGET(WMem, WMem.OFFSET + P%%, _UNSIGNED _BYTE) THEN
            'Nothing doing
            P%% = P%% + 1
        ELSE
            P%% = P%% + 1
        END IF
    WEND
NEXT N&
PRINT "Mem", TIMER - Start!, P%%
 
_MEMFREE WMem
_MEMFREE EMem
 

Here, I give comparisons between _MEM processing, numeral processing and string processing doing exactly the same thing... and the results are in:

Without $CHECKING:OFF
_MEM 31s
Numeric 21.5s
String 327.5s

With $CHECKING:OFF
_MEM 7.3s
Numeric 18.7s
String 321s

So the string processing is much much slower (no surprises), but the _MEM processing is faster than standard numeric processing with $CHECKING:OFF, but slower without this (a bit of a surprise).

What I now need to do is remove all string processing from the Crossword Generator and see what happens, and see where _MEM is useful. Weeks and weeks of intense amusement.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 04, 2019, 09:02:18 am

Quote

”Perhaps _MEMx should never be used for anything unless absolutely necessary. It's a kind of dirty hack that was thrown in to make DECLARE LIBRARY viable and probably wasn't meant for anything else.”

Gosh! That was a bit of a surprise.

A surprise for me as well! I’ve used _MEM for a lot of things in the past, but I’ve never used it (or saw anyone else use it) with DECLARE LIBRARY. If _vince has an example of something that utilizes _MEM for DECLARE LIBRARY, I’d love to see it, just to learn when/where it helps us interact with DECLARE.

As for _MEM being slower in your tests than numbers, without checking:off, I’m not all that surprised. _MEM does a lot of error checking to protect us from ourselves, as _MEM errors tend to completely crash a program rather than just toss an error which we can continue from. It’s that error checking which is using your time in that loop (as you can tell from the change when you turn the checking off).

One thing that’s always certain, from my experience, is that strings are sloooooow. If you can replace them for numbers anywhere, your code will run much faster. When dealing with single bytes like this, swapping MID$ comparisons out to ASC comparisons instead, will speed up a program a ton. _MEM is the fastest way we can go, but just about anything is faster than trying to work directly with strings.

Title: Re: Steve's Video Tutorials
Post by: _vince on October 04, 2019, 03:17:51 pm

Quote from: SMcNeill on October 04, 2019, 09:02:18 am

A surprise for me as well! I’ve used _MEM for a lot of things in the past, but I’ve never used it (or saw anyone else use it) with DECLARE LIBRARY. If _vince has an example of something that utilizes _MEM for DECLARE LIBRARY, I’d love to see it, just to learn when/where it helps us interact with DECLARE.

That is what I believe I've heard, a lot of libraries require passing a pointer and that's pretty much the reason that prompted Galleon to add it in. Not that it isn't handy for other things. I believe Phil may have more insight on this. I also think that Galleon was interested in building up the QB64 community and it was important to him that the code we post and share is as true to his idea of BASIC so that newcomers and others could easily read and translate and quickly benefit and contribute. Imagine if someone is learning QB and wants to study a program but is faced with a bunch of complicated alien _MEMs rather than a pure BASIC solution that is totally possible.

Not that I am at all discouraging what you guys are doing, just antagonizing, carry on ;-)

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 04, 2019, 03:50:34 pm

Quote from: _vince on October 04, 2019, 03:17:51 pm

That is what I believe I've heard, a lot of libraries require passing a pointer and that's pretty much the reason that prompted Galleon to add it in. Not that it isn't handy for other things. I believe Phil may have more insight on this. I also think that Galleon was interested in building up the QB64 community and it was important to him that the code we post and share is as true to his idea of BASIC so that newcomers and others could easily read and translate and quickly benefit and contribute. Imagine if someone is learning QB and wants to study a program but is faced with a bunch of complicated alien _MEMs rather than a pure BASIC solution that is totally possible.

Not that I am at all discouraging what you guys are doing, just antagonizing, carry on ;-)

I think you’re getting a little confused. You can use _OFFSET to pass pointers, whereas _MEM is actually a UDT which holds a lot of extra information other than just that offset for you. http://www.qb64.org/wiki/OFFSET

DIM X AS LONG
PRINT _OFFSET(X)
DIM M AS _MEM: M = _MEM(X)
PRINT M.OFFSET

The above will print the same value for you twice — both are the pointer to where X is stored at in memory.

Generally, in DECLARE LIBRARY, you simply pass the _OFFSET to what you want, and not the internal QB64 type which we call _MEM.

Quote

…a bunch of complicated alien _MEMs rather than a pure BASIC solution that is totally possible.

In comparison to what? A bunch of DEF SEG, PEEK, POKE statements which aren’t documented anywhere and require looking up stuff from an external chart to use, and are OS specific?

DEF SEG = &hA000 ‘Screen 13 memory color put
OffY = Y * 320&
OffSet = OffY + X
Poke OffSet, C
DEF SEG

Verses:

DIM M AS _MEM: M = _MEMIMAGE(0) ‘current display screen
_MEMPUT M, M.OFFSET + Y * 320 + X, C
_MEMFREE M

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on October 05, 2019, 07:56:48 am

Steve, is there a _MEM equivalent of the string equality Word1$ = Word2$? I'm avoiding everthing string-like and using only _MEM objects (of variable length).

I have an array of _MEM objects GW() and a _MEM object MInWord and wish to check that MInWord is not identical to any of the existing GW() _MEM elements (not the same length and at least one byte not identical).

The following code works, but I'd like something equivalent to Word1$ <> Word2$

Code: QB64: [Select]

IF MInWord.SIZE <> GW(P%%).SIZE THEN
    C%% = 0
    GoodMidEnd%% = False
    WHILE C%% < MInWord.SIZE AND NOT GoodMidEnd%%
      IF _MEMGET(MInWord, MInWord.OFFSET + C%%, _UNSIGNED _BYTE) <> _MEMGET(GW(P%%), GW(P%%).OFFSET + C%%, _UNSIGNED _BYTE) THEN
         GoodMidEnd%% = True
         ELSE
         C%% = C%% + 1
      END IF
   WEND
END IF

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on October 08, 2019, 09:22:57 am

Steve, at last I've found time to watch Tutorial 3. Very instructive about _MEM usage with arrays. I still find it unsettling that you can point the same memory block to different variables/arrays without having to do a _MEMFREE beforehand: but that's how it is. This tutorial is quite a marathon to watch (no criticism implied - rather the reverse). So it must have been a considerable effort to produce it. Thanks.

Would it be true to say that
_MEMPUT m, m.OFFSET + Index * m.ELEMENTSIZE, 123 AS (Variable Type)
would be (slightly) faster if m.ELEMENTSIZE is replaced by a fixed number if you know that you will never change the ELEMENTSIZE? It looks as if the program has to look that up every time.

Your tutorial is also instructive in showing that we all make the same types of mistakes in coding. We all must have together expended many coding hours staring at a screen wondering where exactly we typed something somewhat awry.

I'm looking forward to Tutorial 4 (>1D arrays). This might be my Holy Grail for a complete _MEM version of the Crossword Generator.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 08, 2019, 09:54:20 am

Quote

Would it be true to say that
_MEMPUT m, m.OFFSET + Index * m.ELEMENTSIZE, 123 AS (Variable Type)
would be (slightly) faster if m.ELEMENTSIZE is replaced by a fixed number if you know that you will never change the ELEMENTSIZE? It looks as if the program has to look that up every time.

Generally speaking, anytime you can remove an operation, or lookup, you save processing power and run faster.

If you know your ELEMENTSIZE isn’t going to change, the best practice is to precalculate the index first, if you can.

Example:

DIM Array(10) AS LONG
DIM M AS _MEM: M = _MEM(Array())
FOR INDEX = 0 TO 10
_MEMPUT M, M.OFFSET + INDEX *M.ELEMENTSIZE, 123 AS LONG
NEXT

Now, with the above, our memblock is pointing to a LONG Array with 11 elements (from 0 to 10).

We could do as you suggest, and use a constant value of 4 inside that loop, instead of M.ELEMENTSIZE, and our program would be slightly faster...

OR, we could do it as so:

DIM Array(10) AS LONG
DIM M AS _MEM: M = _MEM(Array())
FOR INDEX = M.OFFSET + 0 * M.ELEMENTSIZE TO M.OFFSET + 10 * M.ELEMENTSIZE STEP M.ELEMENTSIZE
_MEMPUT M, INDEX, 123 AS LONG
NEXT

Now our program looks up those values once, in the FOR statement, does the math once there, and we remove that whole sequence of lookups and math formulas in our _MEMPUT statement....

And you won’t get any quicker than that.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 08, 2019, 10:02:33 am

Quote

I still find it unsettling that you can point the same memory block to different variables/arrays without having to do a _MEMFREE beforehand: but that's how it is.

Part of this depends on what MEM points to.

DIM X AS LONG
M = _MEM(X) <— You can reuse this memblock without much worry on freeing it. It just points to where X already exists in memory.

M = _MEMNEW(4) <— This you’d need to *always* free when finished. If you reset the value of M to point elsewhere, how the heck would you ever free that block in the future?? Pointing M elsewhere, without freeing that MEMNEW block you reserved, just gave your program a memory leak, and it’ll eventually crash and die on you.

In my examples, I was pointing to already defined variables and arrays, so I could just point it to different things without any real worries. Had I been using MEMNEW, I couldn’t have done that.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 12, 2019, 04:36:32 am

Project update: In case you guys are wondering why I haven’t uploaded the next video yet, it’s all due to my ISP. We’re *finally* being upgraded from DSL to Fiber, so my upload speed should jump from 780 KBS to 400-500 MBS, with GBS download speeds. Only thing is, they’re ripping out the old lines (which were shitty shoddy anyway) for the new lines and my current service is even crappier than normal... I’m supposed to be upgraded and switched over by Halloween, so I’ll try and get a “treat” up for everyone by then, if’n they finish my service swap over by then. ;D

Title: Re: Steve's Video Tutorials
Post by: Cobalt on October 16, 2019, 07:07:36 pm

hope they get that done before your Roll20 game comes around.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on October 16, 2019, 08:11:32 pm

Quote from: Cobalt on October 16, 2019, 07:07:36 pm

hope they get that done before your Roll20 game comes around.

We’ve already got it up and going, though it’s only sound and no video, for now. The overall forum is here: https://app.roll20.net/campaigns/forum/4971370/ , and you can read my group’s first week’s story here: https://app.roll20.net/forum/post/7831591/the-darkness-rising-ongoing-story-slash-summary.

I’m still catching up to where we stopped at, but there’s not much left to write up. Our next game is Friday, 8:00 EST, and so far, it’s been a blast!

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on November 07, 2019, 05:26:32 am

Sorry guys. It seems as if I fibbed to you all. :(

I didn't manage to get a video up in time for Halloween -- my apologies. My ISP simply didn't make it to my home in time to make the swap over to fiber for me, and I've been sitting here waiting on them to move me on up so I can upload and download without having to wait all day for things to finish...

Well, they've finally got me on the schedule, and should be here today to come inside and finish up things. The line from the junction to my house has already been laid. They've made all the changes to the box outside my house which was necessary, and have dug and laid the cable from the box to my house. All that's left is for them to basically come in today and swap out the old modem which worked with DSL for the new one for the fiber, and then I should be good to go!

And what's this mean for all of you??

I'll be a fibber who can once again upload videos of decent quality and stuff to the interwebs for everyone!!

And yes, it seems I'll even fibbed about what the next video will be about. I promised it was going to be about "Beyond the 1st Dimension", and to go into multidimensional arrays with mem, but...

it's not. :(

As I was working on the blueprint which I wanted to follow for the next mem tutorial, I realized that there's an important bit of information which I really should share and stress first for everyone BEFORE I get into the formulas for calculating offsets for multidimensional arrays and such with mem...

... and that's REDIM _PRESERVE and why the heck it's so frustrating to work with! ARRRGGHHH!!

I've heard a ton of folks complain about how it doesn't really preserve our data (it does), and how it's useless (not quite, but sometimes it can be), and since its behavior is all memory related, I thought I'd go ahead and talk about it for the next video and then the one after that will be about how to do the math to calculate multidimensional array offsets with mem.

My question for all you guys to ponder on, while I wait on the ISP folks to show up today and swap out my internets, is this one:

How do you think an array is stored in memory?

Take an array and let it be: DIM A(x, y)

Now obviously, it's just going to be a chunk of continuous memory and not some elaborate grid of data. (If we could use grids of data, that'd be cool, but how would we layout something like DIM A(x, y, z, p, d, q)?? Do we have 6-dimensional memory chips to hold that information???)

Since we don't have multi-dimensional RAM, it's just going to be stored in a simple continuous list in memory.

A(0,0) = whatever
A(0,1) = whatever
A(0,2) = whatever

But HOW is it stored??? Remember my lesson on Little Endians and Big Chiefs? We could store that data left to right, or right to left, or top to bottom, or bottom to top.... As long as the computer maps to the same spot each and every time, we can store it in whatever dang direction we want!

So the question is, how do we actually store it??

And that's what the video I'll be sharing later today (or next week, as reliable as my ISP isn't) will cover for everyone!

The actual code which I'll explain and use to illustrate this lesson is already written, and I'll go ahead and share it so you guys can look it over early if you want. By itself, it's not going to help explain everything the best, as my intention is to talk and explain what it's doing step by step as I go along. I'll show one thing, then uncomment out another and show how it'd behave if we looked at it in another way, and hopefully by the time I'm finished with it, people will understand why REDIM _PRESERVE works the way it does. Hopefully they'll also be able to use it properly in their code with multidimensional arrays as well, after this! ;)

Code: QB64: [Select]

REDIM A(3, 4)
FOR f = 0 TO 3
    FOR s = 0 TO 4
        total = total + 1 'Assign some totals to that arras
        A(f, s) = total
    NEXT
NEXT
 
'Let's ask a simple question: How is that data stored in memory?
'Is it stored in a left to right, top to bottom style grid?
 
FOR f = 0 TO 3
    FOR s = 0 TO 4
        LOCATE 4 * f + 1, 4 * s + 1: PRINT A(f, s)
    NEXT
    PRINT
NEXT
 
SLEEP
CLS
 
'could if be in a top to bottom, left to right stsle grid?
FOR f = 0 TO 3
    FOR s = 0 TO 4
        LOCATE 4 * s + 1, 4 * f + 1: PRINT A(f, s)
    NEXT
    PRINT
NEXT
 
SLEEP
CLS
 
'heck, masbe it's top to bottom, right to left stsle storage?
FOR f = 0 TO 3
    FOR s = 0 TO 4
        LOCATE 4 * s + 1, 4 * f + 1: PRINT A(3 - f, s)
    NEXT
    PRINT
NEXT
 
SLEEP
CLS
 
'or it could even be bottom to top, right to left...  Right?
FOR f = 0 TO 3
    FOR s = 0 TO 4
        LOCATE 4 * s + 1, 4 * f + 1: PRINT A(3 - f, 4 - s)
    NEXT
    PRINT
NEXT
 
SLEEP
CLS
 
'Just just how the HELL is the data actualls stored?
DIM m AS _MEM 'let's take a mem block
DIM o AS _OFFSET 'we alwass need an eftra offset or three when working with mem blocks
m = _MEM(A()) 'and point it at the arras
DIM s1 AS SINGLE 'a single variable, as we didn't bother to dim our arras ansthing different than the default
DO UNTIL finished
    _MEMGET m, m.OFFSET + o, s1
    PRINT s1;
    'count = count + 1                    'These lines are commented out to just to help us understand what we're seeing
    'IF count MOD 4 = 0 THEN PRINT        'I'll uncomment them as I run the program.
    o = o + m.ELEMENTSIZE
    IF o >= m.SIZE THEN finished = -1 'we're done once we've got all the elements from memors
LOOP
 
'END 'our actual useful lesson (I suppose) starts here.  Prior is just kind of background information...
SLEEP
CLS
 
 
'And why the hell does the direction we store our data in even matter??
'Can you explain that to us, Steve??
'(Probably not, but I can at least pretend to...)
REDIM _PRESERVE A(3, 4) 'this is what our original array was, simply placed here again to refresh our memory.
'REDIM _PRESERVE A(4, 4) 'what would happen to our data if we added a row to it?
'REDIM _PRESERVE A(3, 5) 'and what would happen if we added a column to it?
m = _MEM(A()) 'don't forget to point m back to our memblock, since we resized it and moved where its at in memory.
 
finished = 0: count = 0: o = 0 'reset our variables, and let's check both results
DO UNTIL finished
    _MEMGET m, m.OFFSET + o, s1
    PRINT s1;
    o = o + m.ELEMENTSIZE
    IF o >= m.SIZE THEN finished = -1 'we're done once we've got all the elements from memors
LOOP
 
END 'And WTH?  It didn't really seem to affect things very much, no matter whether we redimed the first element
'or the second element.  Now, did it??
'Let's actually check...
 
SLEEP
CLS
FOR f = 0 TO UBOUND(A, 1)
    FOR s = 0 TO UBOUND(A, 2)
        LOCATE 4 * s + 1, 4 * f + 1: PRINT A(f, s)
    NEXT
    PRINT
NEXT
 
 
 
 
'And this is why, we can safely REDIM _PRESERVE the last element in an array, and preserve our data structure,
'And why it'll get all corrupted if we REDIM _PRESERVE any other element in that array.

Title: Re: Steve's Video Tutorials
Post by: Petr on November 07, 2019, 10:02:31 am

Thank you for the sample code, it's getting dawn. My brain is slowly beginning to understand...

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on November 08, 2019, 03:24:21 am

Video Tutorial #4 is now being uploaded into YouTube at:

This video now covers both REDIM _PRESERVE and mulit-dimensional offsets for use with _MEM, and like the last one, it comes in at about an hour and a half total view time. I know these are rather longish view times, but I'm afraid that's just how long it takes for me to ramble and feel like I've properly covered the subject and illustrated the concepts involved in them. For folks who just want the basic "Cliff Note" version of things, what you'll want to take home at the end of the day is these simple concepts:

Quote

'FINAL RULE:
'**ONLY** redim the right-most index when using REDIM _PRESERVE!!
'IF you need to redim **ANY** other index; you'll have to do it manually!!

Code: [Select]

FUNCTION Offset& (x, y)
    xsize = UBOUND(A, 1) - LBOUND(A, 1) + 1 'the number of elements in X
    Offset& = (y * xsize + x) 'Y * number of elements in X + X
END FUNCTION

FUNCTION ScreenOffset& (x, y)
    ScreenOffset& = (y * _WIDTH + x)
END FUNCTION

To understand WHY those are the important keynotes to remember and use, WATCH the video. If you're not interested in WHY, then just remember and use those as they're presented and know, "that's just the way it is". Either way, I hope what's here will help somebody, somewhere, understand and develop better habits to work with memory more efficiently in the future.

Future videos will be:

Covering the power of M.TYPE

Covering some of the other mem commands such as _MEMFILL and _MEMCOPY

Optimizing code to work more efficiently with _MEM, to make the most of its power

Maybe one of how I wish the makes of Reese's Pieces would go to Hell. We have Halloween candy left over, and their damn wrappings are the type which I can't open EVEN WITH A KNIFE!! I've never seen such a stupidly packaged product in all my life! GRRRR!! *Somebody* needs to do a video on how stupid the things are!

But, until then: Enjoy guys!

Note: Video is currently in processing with Youtube as still has an estimated 9 minutes left before it's available to view. This is a HELLUVA difference from the 8-12 hours which I was faced with, with the last video (which was a comparable size and resolution). My upload speed has went from 0.78MBS to almost 100MBS max speed. (This comes in as a 1.8GB video, so you can see how big of a difference that makes for me!)

Once again, grab the popcorn, have a seat, and listen to me ramble. Watch as I baffle myself and wonder just where the BLEEP I screwed up once more, with the most basic of illustrationing... And hopefully, at the end of the day, you'll end up understanding things more than I ended up confusing things...

All feedback, questions, and other such things are more than welcome below, or via personal email at: smcneill@swva.net (or the email button on the left of this post area).

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on November 09, 2019, 11:04:39 am

And an image comparing my internet speeds, and why I waited for the swap before continuing my tutorials.

From 14mbs download to 150mbs+.

And from 0.78mbs upload to 150mbs+.

Over 10x faster download and 200x faster upload speeds!

Title: Re: Steve's Video Tutorials
Post by: OldMoses on November 19, 2019, 12:46:06 pm

Thank you for the work you're doing on this. I'm getting a much better understanding of what seemed a very arcane subject when viewed in the wiki.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on December 04, 2019, 08:06:00 am

Steve, I've been distracted and not got around to #4, yet. Will do so as soon as possible.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on December 11, 2019, 07:50:20 am

Steve, at last, I've got around to video #4. Thank goodness that I'm never going to use REDIM _PRESERVE. It's difficult enough trying to remember to count from the right without the difficulty of _MEM order being completey scrambled by non-righthand dimension increases. Good luck to anybody having to change (increase) array dimension size. If ever I had to do it, I'd use the cack-handed: ExistingArray(N%,M%), DIM TempArray(N%+1,M%), Copy all elements from Existing to Temp, REDIM ExistingArray(N%+1,M%), Copy all elements from Temp to Existing. And that is precisely Steve's default manual method. So, only those who have a regular need for redimming with increase of size will use REDIM _PRESERVE. Will watch second half of video soon. Keep them coming.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on December 12, 2019, 07:15:45 am

At last I've done #4. Thanks Steve. I had not previously considered the optimisations of FOR loops for _MEM processing. I did not view #4 promptly as I was working on my Pi-in-the-Sky graphics program. That program uses a great amount of _MEM processing of images. I had to change R/G/B/Alpha values. Fortunately I stumbled across the Offset = 4*(x+y*_WIDTH) formula, and am relieved to see that this is the correct form.

Incidentally, Steve, the reason why you ended up with 1 to 19 instead of 1 to 20 in your tutorial was that you typed two 8's before the 9. A simple typo error.

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on December 12, 2019, 08:10:28 am

Quote from: Qwerkey on December 12, 2019, 07:15:45 am

Incidentally, Steve, the reason why you ended up with 1 to 19 instead of 1 to 20 in your tutorial was that you typed two 8's before the 9. A simple typo error.

Sometimes, it amazes me the way my poor brain works. Often, I have absolutely no issues with the complicated stuff when programming, but it’s the simple things I just can’t seem to recognize. I went over the code, and the video, several times, and never noticed the double 8’s. My brain seems to know what things should be, and somehow does an automatic substitution for it mentally, and I’ll overlook a typo like that 100 times before smacking myself upside the head once I finally notice it. I honestly had no idea why those damn comments didn’t count from 1 to 20!! Many thanks for pointing out the solution to that mystery for me! ;D

Just curious, but did the video help explain what REDIM _PRESERVE does, so that it makes sense? A lot of people end up having problems with the command when working with multidimensional arrays, and I’m hoping the video helps clear up what it actually does for us, rather than just muddle the water and complicate things more for people.

Title: Re: Steve's Video Tutorials
Post by: Qwerkey on December 12, 2019, 10:21:47 am

Quote from: SMcNeill on December 12, 2019, 08:10:28 am

Sometimes, it amazes me the way my poor brain works.

Steve, I think that that is something common to us all. When you expect something to be a certain way, you can look at the error many times and still not see it. I had many such moments on my Pi-in-the-Sky coding: it wouldn't work properly and no matter how long I stared at the faulty line, it wouldn't click as to what was wrong. I expect that the psychologists have a term for this: many students will have got PhD's on the subject.

As for the REDIM _PRESERVE explanation, yes that was all very clear (with the outputs of columns/rows and linear MEM positions). But as you say, an occasional user of this will easily forget the precise behaviours in converting from Array to MEM. That's why I will never use REDIM _PRESERVE. My sort of work is highly unlikely to use changing arrays.

Title: Re: Steve's Video Tutorials
Post by: EricE on April 02, 2020, 06:19:56 pm

Steve,
QB64 v1.4 is giving your tutorial #2 program a "Name already in use" error on line 44.

Code: QB64: [Select]

DIM White AS _UNSIGNED LONG

What has happened?
Has "White" become a reserved word now in version 1.4?

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on April 02, 2020, 10:18:13 pm

Maybe, if you have $COLOR set for your programs. ;)

Title: Re: Steve's Video Tutorials
Post by: SMcNeill on April 02, 2020, 10:25:19 pm

Quote from: SMcNeill on April 02, 2020, 10:18:13 pm

Maybe, if you have $COLOR set for your programs. ;)

It is:

Code: [Select]

 DIM White AS _UNSIGNED LONG
White = _RGBA32(255, 255, 255, 255)
PRINT "Hello World"
SLEEP
_MEMFREE m
m = _MEMIMAGE(0) '0 is the display screen.
$COLOR:32

With the latest versions of QB64, $COLOR:32 takes precedence for CONST values, causing the issue. Originally, it allowed for custom values to be predefined first, as you see above, but it no longer does that. The code needs a little tweaking to reflect the current version vs what the development build originally offered.

Text Only | Text with Attachments

QB64.org Forum

Active Forums => Programs => Topic started by: SMcNeill on September 22, 2019, 01:59:55 pm