Bas Files Count and List by Recursive Algorithm

[SMcNeill]

It should work just as you're used to, with the addition of an error message if something slides past detection as either file or directory.  (And you shouldn't need to CHDIR for it to work, I wouldn't think.)

[SpriggsySpriggs]

All I was showing was how you can do a recursive search with PowerShell. It isn't necessarily only searching files. It's searching based on the string. So it will return anything that matches the string, no matter if it is a file or a folder. It is interesting that one would have a folder with an extension in the name, however. I think the PowerShell call could be edited to do only files.

[bplus]

OK I have 5 programs that count and list Bas files from a start Directory, they are described with a couple times I got testing the programs. Turns out when I fixed the original recursive code, it turned in the fastest times but Steve's code is checking everything which I think accounts for difference in times, more than double.

Anyway here is code and comments tested on my old Windows 10 laptop 64-bit with a Desktop containing over 11,850 files. Of course you will have different results.

PS to test in your system edit a couple of lines near the top of each program specifying the path and the pathed file list file name to use. Oh, also if direntry.h is not in the same folder as QB64.exe, there is a copy that is commented out at the bottom of programs that use it.

[bplus]

Update: I was working on clearing all these experiments off my Desktop and retesting them in my Downloads Folder.

Big snag with all SMcNeills code because he was using Integer Type for FileCount and DirCount. Turns out I had a folder with over 40,000 files and I got subscript error repeated over and over for that Folder.

Also after fixing that with Long Type, I ran into a problem file caught by Steve's code "that should never happen", here is a snapshot of that file:
 

Here is revised GetLists subroutine:

Code: QB64: [Select]

1. ' fixed 2021-11-16 for folders or files OVER 32,000+ Integer Range
2. ' ref 2021-11-09 Steve update GetLists:  https://www.qb64.org/forum/index.php?topic=4360.msg138031#msg138031
3. Sub GetLists (SearchDirectory As String, DirList() As String, FileList() As String)
4.     ' Thanks SNcNeill ! for a cross platform method to get file and directory lists
5.     'put this block in main code section of your program close to top
6.     '' direntry.h needs to be in QB64 folder '<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
7.     'Declare CustomType Library ".\direntry"
8.     '    Function load_dir& (s As String)
9.     '    Function has_next_entry& ()
10.     '    Sub close_dir ()
11.     '    Sub get_next_entry (s As String, flags As Long, file_size As Long)
12.     'End Declare
13.  
14.     ' fix 2021-11-16 I have a folder in Downloads that is over 40,000 files mostly HTML which caused subscript errors
15.     ' change DirCount and FileCount to Long from Integer
16.     Dim flags As Long, file_size As Long, DirCount As Long, FileCount As Long, length As Long
17.     Dim nam$, slash$
18.     ReDim _Preserve DirList(100), FileList(100)
19.     DirCount = 0: FileCount = 0
20.     $If WIN Then
21.         slash$ = "\"
22.     $Else
23.         slash$ = "/"
24.     $End If
25.     If Right$(SearchDirectory$, 1) <> "/" And Right$(SearchDirectory$, 1) <> "\" Then SearchDirectory$ = SearchDirectory$ + slash$
26.  
27.     If load_dir(SearchDirectory + Chr$(0)) Then
28.         Do
29.             length = has_next_entry
30.             If length > -1 Then
31.                 nam$ = Space$(length)
32.                 get_next_entry nam$, flags, file_size
33.                 If _DirExists(SearchDirectory + nam$) Then
34.                     DirCount = DirCount + 1
35.                     If DirCount > UBound(DirList) Then ReDim _Preserve DirList(UBound(DirList) + 100)
36.                     DirList(DirCount) = nam$
37.                 ElseIf _FileExists(SearchDirectory + nam$) Then
38.                     FileCount = FileCount + 1
39.                     If FileCount > UBound(FileList) Then ReDim _Preserve FileList(UBound(FileList) + 100)
40.                     FileList(FileCount) = nam$
41.                 Else 'This else should never actually trigger
42.                     Print: Print: Print "zzz...  Unknown file found: "; SearchDirectory; slash$; nam$, _DirExists(nam$)
43.                     Beep: Sleep ' alert the user to
44.                 End If
45.             End If
46.         Loop Until length = -1
47.     End If
48.     close_dir
49.  
50.     ReDim _Preserve DirList(DirCount)
51.     ReDim _Preserve FileList(FileCount)
52. End Sub
53.  

copy of direntry.h to put in same folder as your QB64.exe

Code: [Select]

#include <dirent.h>
#include <sys/stat.h>
#include <unistd.h>

const int IS_DIR_FLAG = 1, IS_FILE_FLAG = 2;

DIR *pdir;
struct dirent *next_entry;
struct stat statbuf1;

char current_dir[FILENAME_MAX];
#ifdef QB64_WINDOWS
  #define GetCurrentDir _getcwd
#else
  #define GetCurrentDir getcwd
#endif

int load_dir (char * path) {
  struct dirent *pent;
  struct stat statbuf1;
//Open current directory
pdir = opendir(path);
if (!pdir) {
return 0; //Didn't open
}
return -1;
}

int has_next_entry () {
  next_entry = readdir(pdir);
  if (next_entry == NULL) return -1;
 
  stat(next_entry->d_name, &statbuf1);
  return strlen(next_entry->d_name);
}

void get_next_entry (char * nam, int * flags, int * file_size) {
  strcpy(nam, next_entry->d_name);
  if (S_ISDIR(statbuf1.st_mode)) {
    *flags = IS_DIR_FLAG;
  } else {
    *flags = IS_FILE_FLAG;
  }
  *file_size = statbuf1.st_size;
  return ;
}

void close_dir () {
  closedir(pdir);
  pdir = NULL;
  return ;
}

int current_dir_length () {
  GetCurrentDir(current_dir, sizeof(current_dir));
  return strlen(current_dir);
}

void get_current_dir(char *dir) {
  memcpy(dir, current_dir, strlen(current_dir));
  return ;
}



[SMcNeill]

Question: How'd you generate two slashes in the filename?  And if it's not a file, nor a directory, what is it?  Does it even exist, or is that some false positive it generated somehow?

[bplus]

I think I was trying out a Wiki access from something by you @SMcNeill

A view from InForm Project: