"Out of Memory" mystery

[OldMoses]

UPDATE: Mystery solved, thanks for your help guys.

Been running my updated grain harvest database program. The machine generally sits in the drying room waiting for a truck to come in and someone to enter data. This might occur a couple times an hour to a couple hours between. I updated the program to use four 32 bit display screens and colors, loading 10 button images that range from roughly 2 to 20 kB each in the main module. Once in the program's display/input loop only _PUTIMAGE is used, repeatedly, along with general print, locate and a couple line statements. The basic data handling algorithm has remained largely unchanged and always worked for hours on end before. Only the main display screen is called for updating while awaiting some form of input. All the _PUTIMAGE commands occur during this looping.

After running unattended for somewhere around an hour or two the following error consistently occurs:

A small dialog box with:

"Critical error..." < in the top bar

"Out of Memory" < in the main part of the box

followed by an OK button.

It's never actually happened when I was around to witness it. I ran it this morning and went to church, was gone for a good 2 hours + and true to its M.O., it was stopped when I returned. I don't relish the idea of sitting and staring at it until it occurs...

I opened Notepad to compose this post and it would open, but the dialog box remained on top and until I clicked the OK button; vanquishing both the dialog and my program that caused it, I could do nothing with Notepad.

I've poured over what I believe to be the salient code and can't find an issue so far. The main display/input loop was running _LIMIT 200 and I reasoned that if the program were loading up memory with something while looping unattended, changing to a slower limit value would alter the time it took to throw the error. Does that sound like a reasonable test?

Yesterday I did that. After changing to _LIMIT 30, the program still threw the error around an hour or two. Now I'm wondering if something in the operating environment is to blame. I'm running Windows 10 Home 1803 64bit on a Lenovo Ideapad 330S Core I5 w/ 8GB RAM. QB64 is version 1.2 20180202/85. Sleep modes are disabled while the machine is plugged in.

Does anyone have any experience or insight regarding this issue? I'm nearing the end of my rope...

My next move is to run the application on another system and see if the same thing happens.

[TerryRitchie]

Something in your main loop is continuously needing more RAM and taking it. You can verify this by opening Task Manager and watching your program's RAM usage. You'll more than likely see it climbing little by little over time.

Make sure there is nothing like a _LOADIMAGE stuck in your main loop that would cause this type of behavior. Would you be willing to share your code or the main loop of it for investigation?

[SMcNeill]

Are you using such commands as _LOADIMAGE, _NEWIMAGE' _LOADFONT, or anything else which loads data into the program?

You say to load 10 button images...  Are you positive that you only load them ONCE??  Without seeing the actual code, all I can do is guess at the issue, but it sounds as if the program has a gradual memory leak.

This will cause your issue (pseudocode only):

DO
    LOADIMAGE
    PUTIMAGE
    DISPLAY
LOOP

In the above, we're loading an image over and over and over, until we crash.   There's 2 basic cures for this:

DO
   LOADIMAGE
   PUTIMAGE
   DISPLAY
   FREEIMAGE
LOOP

Method One:  Free the image after using it, as above.

LOADIMAGE
DO
    PUTIMAGE
    DISPLAY
LOOP

Method Two:  Load the image only once and leave it in memory while the program runs.

********************

Just guessing, I'd guess it's a LOAD without FREE issue, somewhere in your code.

[Pete]

Did you update to a different version of QB64 since the time you had it running without a memory issue? I think Rob had to debug QB64 a couple of times for memory leaks. I had one several years ago.

QB45 had far less memory and would throw "Out of Stack Space" error message if a gosub routine was repeatedly looped but not exited properly with a return, as when encountering a GOTO statement within the gosub routine.

I've had a couple of things I've mis-coded in the past, and the exit never occurred in a loop routine, which somewhat quickly lead to the out of memory error. Unfortunately, I can't remember the code the routine at the moment proving that the worst offender for an out of memory error is aging. :(

You'll probably get a lot more feedback shortly. Sorry I' can't be of more help here, I don't do much with non-SCREEN 0 apps. I just wanted to let you know this is situation does occur in other people's programs.

Ah, I see Terry and Steve just responded while I was writing this.

Pete

[OldMoses]

Something in your main loop is continuously needing more RAM and taking it. You can verify this by opening Task Manager and watching your program's RAM usage. You'll more than likely see it climbing little by little over time.

Make sure there is nothing like a _LOADIMAGE stuck in your main loop that would cause this type of behavior. Would you be willing to share your code or the main loop of it for investigation?

I'm fairly certain I've only loaded images in the main module. I hadn't thought of using TM, I'll do that now.

In the meantime, I'm zipping the whole thing along with a sample datafile and a brief "readme" to explain the startup sequence.

[OldMoses]

LOADIMAGE
DO
    PUTIMAGE
    DISPLAY
LOOP

Method Two:  Load the image only once and leave it in memory while the program runs.

********************

Just guessing, I'd guess it's a LOAD without FREE issue, somewhere in your code.

This is the method I am trying to use. I can't find anything but putimage in anything that functions from within the loop.

[OldMoses]

Running Task Manager on it, it definitely is ticking up in memory usage by about .2 to .3 MB per second or so.

[OldMoses]

Alright, Task Manager has yielded some interesting results.

The memory leak only occurs in the main display. It stops when doing data entry.

Also, If I change to a field that has no data in it, the leak slows markedly, suggesting that the leak is in the data displayed. Perhaps my crazy color scheme is doing something. I better look at the data handling subs Echo_Load and Entry_Print more closely.

[TerryRitchie]

Running Task Manager on it, it definitely is ticking up in memory usage by about .2 to .3 MB per second or so.

Yeah, that's pretty significant. I'm glad Task Manager is giving you some insight. I see others have already downloaded your code. I will later on as well (working on something else right now) but I'm sure the others will probably have an answer for you by then.

[OldMoses]

Hi Pete,
I got this computer last month and downloaded the most recent version on the site right after I got it. I was probably running an older version last fall when I first discovered QB64 and used it on an older version (3.2.1) of my program. I can't honestly say whether there would have been memory issues back then as I don't know that I ever let the program just sit on its own long enough. Back then I was sitting down to it and doing data entry sessions that generally lasted under a half hour. It was a blast though, no more DosBox mounting headaches, folks for miles around were wondering what that "squeeee" noise was... :D

Did you update to a different version of QB64 since the time you had it running without a memory issue? I think Rob had to debug QB64 a couple of times for memory leaks. I had one several years ago.

[OldMoses]

OK, by commenting out certain SUB calls I've been able to narrow the issue down to any call to SUB Entry_Print. In a loop it causes a continuous memory leak. One other SUB uses Entry_Print, only not in a loop and with no scrolling function. SUB View_File re-displays the data once through. It is a holdover from earlier versions which I kept as it is a way to display changes in dry bushels caused by changing the target moisture. When it runs there is a one time jump in memory usage only. So it seems consistent with any call to Entry_Print being the issue.

Am I just nuts in that I can't see a problem here???

Code: QB64: [Select]

1. SUB Entry_Print (entry AS LoadRecord)
2.  
3.     '-------------------------DISPLAY----------------------------------------------------
4.     ' SUB: Entry_Print
5.     '
6.     ' Purpose:
7.     ' Displays tabbed data from record number passed by 'entry' when called.
8.     '
9.     ' Passed parameters:
10.     ' entry sends the load record for display
11.     '
12.     ' Called from:
13.     ' Add_Record, Edit_File, Refresh_Screen, Echo_Loads and View_File
14.     '
15.     '-----------------------------------------------------------------------------------
16.  
17.     PRINT TAB(1); USING "###"; entry.load;
18.     PRINT TAB(6); entry.Date;
19.     PRINT TAB(17); entry.Time;
20.     PRINT TAB(28); RTRIM$(entry.truck);
21.     PRINT TAB(37); USING "###,###"; entry.Gross;
22.     PRINT TAB(45); USING "###,###"; entry.Net;
23.     PRINT TAB(54); USING "#,###.##"; entry.Wet;
24.     PRINT TAB(64); USING "##.##"; entry.Test;
25.     PRINT TAB(70); USING "#,###.##"; entry.Dry
26.  
27. END SUB 'Entry_Print
28.  

BTW, View_File is accessed by the "reView" menu choice.

[Cobalt]

I don't believe its with that particular sub, running a test program with just a DO:LOOP calling that SUB  and running a program with only a DO:LOOP i saw the same mem seep which topped out on my machine around 38-40 megs and then fluctuated there.
within your actual program there was a major memory jump with a more rapid climb after entering the farm information using CORN option where the memory jumped to ~58megs then climbed by 200-400k every update in task manager.

[OldMoses]

I don't believe its with that particular sub, running a test program with just a DO:LOOP calling that SUB  and running a program with only a DO:LOOP i saw the same mem seep which topped out on my machine around 38-40 megs and then fluctuated there.
within your actual program there was a major memory jump with a more rapid climb after entering the farm information using CORN option where the memory jumped to ~58megs then climbed by 200-400k every update in task manager.

That's pretty much what I've been getting too. The thing is, I've been able to get the leak to stop by commenting out various instances of calling Entry_Print in those subs that call it. Of course I lose the associated displays when I do so, but most other functionality remains. For instance, SUB Echo_Loads calls Entry_Print in two instances controlled by an IF...THEN structure, one in normal display mode and the other in Edit mode. If I comment out the edit mode call, edit mode goes blank when chosen, the leak stops and I can still edit though I have no visual reference to know which field I'm clicking on. If I return to normal mode the leak continues. The opposite is true if I comment out the normal mode call.

I'm open to the possibility that the leak is caused elsewhere, but if so, the brain teaser is raised exponentially.

I considered the possibility that passing a user defined parameter to Entry_Print was the culprit and adjusted the SUB to accept a straight integer index number and referenced the record variable directly, using the index as an array pointer. I've even tried moving the Entry_Print code out to the subs that were calling it. I found out, as Newt said, "It won't make any difference..."

Now I'm left to wonder if QB64 doesn't like something about the use of TAB or USING in this application. Desperate times call for desperate measures, I suppose.

[SMcNeill]

I dub this one a complete mystery.   

I downloaded, extracted the zip into my QB64 folder, and tested the code....   Never once managed to get the memory usage to go above 50MB, no matter what options I tried to select or enter.

The only change I made was to comment out the line making the EXEICON (It couldn't find the icon for whatever reason), and then I let it run and go.  If there's a leak in there somewhere, I'm not seeing it here on my end.  Somebody will need to walk me through the steps to reproduce the issue, before I'll be much help digging into it, I'm afraid. 


EDIT:   Scratch all that.    Test this:

Code: QB64: [Select]

1. DO
2.     Entry_Print
3. LOOP
4.  
5.  
6. SUB Entry_Print '(entry AS LoadRecord)
7.  
8.     '-------------------------DISPLAY----------------------------------------------------
9.     ' SUB: Entry_Print
10.     '
11.     ' Purpose:
12.     ' Displays tabbed data from record number passed by 'entry' when called.
13.     '
14.     ' Passed parameters:
15.     ' entry sends the load record for display
16.     '
17.     ' Called from:
18.     ' Add_Record, Edit_File, Refresh_Screen, and View_File
19.     '
20.     '-----------------------------------------------------------------------------------
21.  
22.     PRINT TAB(1); USING "###"; entry.load;
23.     PRINT TAB(6); entry.Date;
24.     PRINT TAB(17); entry.Time;
25.     PRINT TAB(28); RTRIM$(entry.truck$);
26.     PRINT TAB(37); USING "###,###"; entry.Gross;
27.     PRINT TAB(45); USING "###,###"; entry.Net;
28.     PRINT TAB(54); USING "#,###.##"; entry.Wet;
29.     PRINT TAB(64); USING "##.##"; entry.Test;
30.     PRINT TAB(70); USING "#,###.##"; entry.Dry
31.  
32. END SUB 'Entry_Print
33.  

[SMcNeill]

Removing the use of TAB corrects the issue.  Apparently TAB has a memory leak in it somewhere:

Code: QB64: [Select]

1. DO
2.     Entry_Print
3. LOOP
4.  
5.  
6. SUB Entry_Print '(entry AS LoadRecord)
7.  
8.     '-------------------------DISPLAY----------------------------------------------------
9.     ' SUB: Entry_Print
10.     '
11.     ' Purpose:
12.     ' Displays tabbed data from record number passed by 'entry' when called.
13.     '
14.     ' Passed parameters:
15.     ' entry sends the load record for display
16.     '
17.     ' Called from:
18.     ' Add_Record, Edit_File, Refresh_Screen, and View_File
19.     '
20.     '-----------------------------------------------------------------------------------
21.  
22.     LOCATE , 1: PRINT USING "###   "; entry.load;
23.     LOCATE , 6: PRINT entry.Date;
24.     LOCATE , 17: PRINT entry.Time;
25.     LOCATE , 28: PRINT RTRIM$(entry.truck$);
26.     LOCATE , 37: PRINT USING "###,###"; entry.Gross;
27.     LOCATE , 45: PRINT USING "###,###"; entry.Net;
28.     LOCATE , 54: PRINT USING "#,###.##"; entry.Wet;
29.     LOCATE , 64: PRINT USING "##.##"; entry.Test;
30.     LOCATE , 70: PRINT USING "#,###.##"; entry.Dry
31.  
32. END SUB 'Entry_Print
33.