Memory leak (revisited)

[Richard]

I know that I have a memory leak in my program - and tried to ensure every time I


img& = _LOADIMAGE(img$)    ' valid img& (handle) <  -1         -1 is  invalid


there is always a _FREEIMAGE to go with it (when I have finished with the image - which occurs after a few lines of code)


..............

IF img& < -1 THEN
        _PUTIMAGE (x0%, y0%)-(x1%, y1%), img& '
END IF
.
.............

IF img& < -1 THEN
_TITLE "@line 512 img&=" + STR$(img&)
_FREEIMAGE img&: img& = 0                  ' ***  this is LINE 512
'END IF


By having the img& value displayed in the TITLE bar I have finally "caught" this error

Unhandled Error #258
Line 512 (in main module0
Invalid handle
Continue?


AT this point the TITLE bar displays

@line 512 img& = -541


Yes I know I have a memory leak (typically only one image is loaded, putted and freed before another image is likewise handled) - just wondering on the handle value of -541 being an error (some internal limit for handles maybe???)

Does it matter if I have more tests for img& <-1 (with _FREEIMAGE being done) then I have img& = LOADIMAGE(img$)?

[SMcNeill]

From my experience, image handles usually start at -12 or -13 and go down from there, the more you load them.  _freeimage usually frees up the handle for reuse, so if you’re only running a single image and freeing it before loading another, the handle should stay at -12 or -13.  A handle of -541 says you missed a freeimage somewhere and that explains your memory leak issue.

[Richard Frost]

Screens also count as images. 

My chess program toggles between 480*600 and 800*600, the larger screen
for displaying extra info like pieces captured, history, thinking, etc.

How I *was* doing that, simply SCREEN _NEWIMAGE(scx,scy,32) was creating
a memory leak.  The right way to do it is:

IF mscreen& < -1 THEN
   SCREEN 0
   _FREEIMAGE mscreen&
END IF

mscreen& = _NEWIMAGE(scx,scy,32)
SCREEN mscreen&

The variable mscreen& needs to be in a SUB declared with STATIC, or DIM
SHARED to preserve the value.

[Richard]

@SMcNeill @Richard Frost  thanks for reply

I wrote a 30 line program (simplified version of my large program) to submit to this forum - but it did not "fail". Results for the 30 line program for all the img& handle values for a loop of 10,000 loading the same 24 Mbyte .bmp image and putting on to 400x400 display window are :-

Either  -1 (first handle reported) and -2  (all other handles)

or  -9  (all handles)

the whole 10,000 iterations was repeated about 5 times and the difference in free memory from first image iteration to 10,000th image iteration was about 30 MByte (which is acceptable to me).


Meanwhile, my actual program now is reporting img& values (in time sequence):-

-9   -8   -7   -6   -5   -4   -3   -14   -15   -16   -17 .....

and for all images the error reported above results (I allow the program to continue in all cases).

So it is interesting that there is an initial incease in handle numbers up to -3 then it jumps to -14 and deceases from there (as expected) - and not recorded by my program (but using Windows Task Manager to monitor) - a rapid increase of memory usage (say over 1 GByte per hour for say maybe 1,000 images per hour).

Note that the same programs have been tested/used on the latest development builds and v1.5 stable builds (Windows x64) and the same results occur.



The only explanation I can offer is that (my program in use first writes the image to a high capacity SHDC card and then loads the image, puts to screen and frees memory) my media is "failing". At present it was initially memory tested with 1 GByte files until "full" and read back (process done once for whole SDHC) using a free 3rd party software. At the time it was prevailant of "fakes" on the market from eBAY (hence the SDHC memory test). So after once only being filled up and then completely erased - to date it is now only about 80% full with say 1 MByte files/images. It took over a year to get this far (80% capacity) and so SDHC is out of warranty.

Meanwhile a few years ago, the same brand/model/capacity SDHC card (#2) also after about a year "really played up" and "failed" at about 80% capacity say.

I will try a totally different media very soon and see what happens (and report back in either case)


As far as QB64 goes - what is the process in handles being allocated - referring the "funny" sequence above - e.g. does QB64 allocate a say -9 value if the media where the image is being loaded from is "too slow" or something??? Or is it Windows that is generating the handle numbers???

[Petr]

Your memory leak is on line 512: img& = 0 is the same as img& = _DEST if you are in default SCREEN. Try delete this statement: img& = 0

[Petr]

None memory leak on my side:

Code: QB64: [Select]

1. 'memory leak test
2. file$ = "mraky.bmp"
3. Cycles = 10000
4. SCREEN _NEWIMAGE(1024, 768, 32)
5. DO UNTIL test = Cycles
6.     img& = _LOADIMAGE(file$, 32)
7.     IF img& < -2 THEN PRINT "Error!, Memory leak!"
8.     _PUTIMAGE (0, 0), img&, 0
9.     _FREEIMAGE img&
10.     LOCATE 1
11.  
12.     prc = _CEIL(test / 100)
13.     msg$ = "[" + LTRIM$(STR$(prc)) + "%]"
14.     PRINT msg$
15.     PRINT "Loop: "; test + 1; "/ 10.000"
16.     test = test + 1
17.     _DISPLAY
18. LOOP
19.  

Program memory usage is cca 100 megabytes, BMP file size is 1.7 megabyte.

[Richard Frost]

Load &  display an image 10,000 times with no _LIMIT or _DELAY? 
Are you using your CPU to toast bread?

[Petr]

Are you using your CPU to toast bread?

Actually, I give the processor a lot of work, but this use did not occur to me :) It is true that in the room with the computer does not heat, but it is warmer than in other rooms. The fan on the processor often roars like a small hammer drill, AMD would probably want a new fan.

[Richard]

I think I have sorted myself out.

Yes I knew my QB64 program had a memory leak (and because not having enough FREEIMAGE img& for every LOADIMAGE).

My problem was affected by the bulk of the program written over a year ago and very recently some condition checks inserted between a LOADIMAGE and the matching FREEIMAGE. To minimize this problem I have tried to keep the

LOADIMAGE  PUTIMAGE  FREEIMAGE as a tight cluster (often on the same QB64 IDE line) and now only about 0.01% of the time it is observed that a known LOADIMAGE will fail (which will now produce a relatively slow memory leak).

As well as Windows Task Manager being used to check for memory leaks - my program determines the free physical RAM memory.

It is interesting that although my program is "memory leaking" at now a rough estimate of 1 Gbyte per week (which is acceptable and manageable for me) - it appears that Windows is "memory leaking" much faster (e.g. by about 8-4 -> ~4 Gbyte over 12 hours. Task Manager reports ~87% memory useage (the big processes are Windows Explorer and Microsoft Edge at 1-2 GByte each).

On first sight, doing the "math", there does not appear to be any sensible correlation being Task Manager Memory% report and a command line free physical memory report.

My best guide now is the free physical memory being updated many times a minute and being displayed on the QB64 program TITLE bar (as well as the LOADIMAGE img& handle value at any time).

So in summary the pseudo code of my program is like so


LOOP

LOADIMAGE  PUTIMAGE FREEIMAGE handle
LOADIMAGE  PUTIMAGE FREEIMAGE handle
LOADIMAGE  PUTIMAGE FREEIMAGE handle

LOADIMAGE - timing problem 0.01% of time - attempt to FREEIMAGE fails - resulting in next time to this line img& handle value is -9 (and subsequently in time sequence -8 -7 -6 -5 -4 -3 -13 -14 -15 -16 -17...)

LOADIMAGE  PUTIMAGE FREEIMAGE handle
LOADIMAGE  PUTIMAGE FREEIMAGE handle
LOADIMAGE  PUTIMAGE FREEIMAGE handle

END LOOP


So as a temporary work-around - use ErrHandler routine with RESUME NEXT for the known error which will occur about 0.01% of the time, let it "memory leak" (at a rate of about 1 Gbyte per week) - but watching the QB64 TITLE bar freePhysicalMemory which indicates a memory leak of about 4 GByte per 12 hours - and auto terminate the QB64 program before Windows "locks up" (typically for me before a 3-day session).

I will try to "fine-tune" the ErrHandler routine (by experimenting) but any gains will not offset the Windows "memory leak"


@Petr - thanks for the suggestion but deleting out img& = 0  did not solve anything (I have changed my code a bit not to rely on img& =0).

@Richard Frost My laptop (about 5 years old) runs hot anyway - the specifications I think stated vent output temperature of 40-60 C and so when I have the laptop on my lap I probably could "fry eggs". The internal thermal design (one heat-sink heat-pipe heat-sink+fan   system) on purpose only services a portion of the processor zones). When only a few processors are running the overall temperature is significantly lower that at "full blast". I think the reason it was designed this way (apart from saving $ on heat pipes) was that I gather the laptop was meant to be a server which does not matter if it runs hot. As far as removing BTU's (heat) is concerned - the greater the temperature differential = the greater the BTU's removed by the same small sized fan - so run "hot" and get away with using a smaller fan.

[SpriggsySpriggs]

1 GB per week? Why on Earth are these programs running for several days at a time?

[Richard]

@SpriggsySpriggs

Why not?

It seems to me that you are "blessed" with a "well-behaved M$ Windows OS"

Overall, to me it seems like half my computer useage is spent "messing around with Windows 10" due to all sorts of issues (including but not limited to Depreciation, DLL's used to work but were changed, .sys files changed, reading M$ documentation (uses a lot of words (fair enough)) but often not working "straight out of the box" for me, overall taking significantly longer to achieve the same task from programs from yearly build to build, overall "sluggish response" and often no-idea has the computer "froze" or is Windows still working on something (and not having a message to user such as a countdown or something) and "bloated software" (though not limited to Windows). etc etc.

I read somewhere that (un-officially) INTEL was responding for quite some time now (many years) to M$ bloatware and software processing inefficiencies buy ever increasing INTEL hardware improvements. It seems that M$ response to users who complain about windows being slow etc is "use a faster processor, more ram, bigger hard drive" - and INTEL's response is they create higher clock cycles processors, bigger caches, pipelining/multitasking, dual core + multi -processor systems, advanced architectural changes and new instruction codes...  And all this is a never ending cycle.

To "make up" for "lost time" (because of M$) I tend to automate (by QB64 code or even M$ code) many tasks but myself often do not try to optimize my approach. For example when searching through my estimated 10 GByte archive of (2008-2016) QB64 .net files I often just let File Explorer Search spend over a day 24/7 to report on search results - whereas on a few occasions doing it "by-hand" I may find the relevant searches with an hour's effort on my part.

So often I just run inefficient programs (up to 3 days) in the background (and when I am sleeping) rather than spend the hours at the keyboard.

Maybe I have a "dodgy" internet connection resulting in "seconds" quality software from M$ which results on many occasions with M$ Windows products that create more problems for me than what its worth. Or maybe I am just unlucky.

Hope this answers your question.

[Richard]

I have run the program now (in $DEBUG mode) for over 5 hours and img& handles within tolerance.  The watchpoint of    img& = -9   
has not been reached and the memory used by my QB64 program is about 350 MByte (OK). The expected 0.01% failure rate has not been achieved. (As the old saying goes ... "A watched kettle never boils").  Meanwhile available Free Memory has gradually dropped down to ~ 1.8 GByte (Task Manager is showing System Memory useage @ 5.66 GByte). I probably will let the computer "crash" to estimate the absolute lower limit for free memory in which to do auto preventive maintenance shutdown (with saving).



Quote from @FellippeHeitor

No "cha-ching", Richard. One is sitting in front of the computer watching the outcome of a buggy program in order to find the bug so the program can run indefinitely. One will see the condition be met. One is not leaving a program run in debug mode and go grab lunch.

So I am not allowed to go away to grab lunch - however I gather that it is OK for me to go to sleep - pity I do not have "cha-ching" as an alarm to wake me up when $DEBUG catches img& = -9 (in my condition test here).

[FellippeHeitor]

For that very particular purpose, you will add an IF img& = -9 THEN BEEP in your program 😉