Author Topic: Flush File Buffers  (Read 4237 times)

0 Members and 1 Guest are viewing this topic.

Offline moises1953

  • Newbie
  • Posts: 55
    • View Profile
Flush File Buffers
« on: August 01, 2020, 02:59:00 am »
Windows has a FlushFileBuffers function to move the pending data from cache to file, without closing the file:
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-flushfilebuffers
I'm interested in implement this function but need the windows file handle, and FILEATTR has not been implemented.
There is any way to perform fileflushing?.
Posible to add a _FILEFLUSH statement (or similary) to QB64?

Offline doppler

  • Forum Regular
  • Posts: 241
    • View Profile
Re: Flush File Buffers
« Reply #1 on: August 01, 2020, 07:13:06 pm »
The only reason windows has the flush function is to accelerate closing and flushing files.  Normally windows will flush at idol times or when a threshold is meet.  The only reason I have had to use a flush function program is when I can't remove a external drive (eject/unmount).  Such as when I can not remove a flash drive because some program is using it. (none really are).  External devices are the last to get flushed.  Hence it seems slow.

As far as QB64 needing it.  QB64 is not fast enough to reach windows threshold.  And any records written out but not flushed are in memory cached.  Quick re-access.  Close and reopen a file will commit a flush faster.  So just use close and reopen.

PS. Beside you can always use a dos shell to execute a flush. (call your program)
« Last Edit: August 01, 2020, 07:15:50 pm by doppler »

Offline moises1953

  • Newbie
  • Posts: 55
    • View Profile
Re: Flush File Buffers
« Reply #2 on: August 03, 2020, 02:37:37 am »
Thank's doppler for your help.
The implementation of FILEATTR may solve the problem:
handle=FILEATTR(fileno, 2)

Offline Dav

  • Forum Resident
  • Posts: 792
    • View Profile
Re: Flush File Buffers
« Reply #3 on: August 03, 2020, 08:45:18 am »
I was wondering...would doing a SEEK to EOF make the file system flush first before seeking?

- Dav 

FellippeHeitor

  • Guest
Re: Flush File Buffers
« Reply #4 on: August 03, 2020, 10:04:21 am »
You could just CLOSE the file handle and then OPEN it again - I believe the flushing would occur immediately.

Offline Petr

  • Forum Resident
  • Posts: 1720
  • The best code is the DNA of the hops.
    • View Profile
Re: Flush File Buffers
« Reply #5 on: August 03, 2020, 10:30:33 am »
I may not have understood the problem right at the beginning of the thread, but:

If you are in the file opened as record number 1 in the mode for binary access and you are in the position, say, byte 150 in the file, then you get the rest of the file as GET 1, LOF (1) - SEEK (1), REC $ where REC $ is fixed a string of length LOF (1) - SEEK (1).

I think it can be solved, it just depends on the way you open the file. I assume that QB64 is fast enough for that (when I write about 1 gigabyte with QB64 to disk in a very short time). When you open a file in a new block of memory (MEMNEW), you gain direct access to the data in memory and you have directly accessible your own cache equivalent.

Offline moises1953

  • Newbie
  • Posts: 55
    • View Profile
Re: Flush File Buffers
« Reply #6 on: August 05, 2020, 06:17:01 am »
In Windows, cache is a self-contained subsystem that follows its own criteria for data flow from a file cache to disk, regardless of the readings that are made, and even if it is closed.
The two easy ways to ensure the data integrity of a transaction on a file is:
1) Move data at the end of transaction by calling FlushFilebuffers in the case of Windows
2) Open file without cache, making it more slow

Say that, if you capture a set of data that may be recorded in a file, it's a good practice to call FlushFilebuffers at the end of all recordings (not every put), that informs to cache subsystem to move data to disc, without a need to close the file. But to do this, is necesary to use handle=FILEATTR(fileno, 2) and use this obtained handle in the call to FlushFilebuffers from the windows API, unless another way is implemented.

EDIT: May be FellippeHeitor tesis, that on close the system flushes the data, is correct, but needs the filename to reopen.
« Last Edit: August 05, 2020, 06:54:08 am by moises1953 »