Author Topic: Bas Files Count and List by Recursive Algorithm  (Read 13903 times)

0 Members and 1 Guest are viewing this topic.

This topic contains a post which is marked as Best Answer. Press here if you would like to see it.

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Bas Files Count and List by Recursive Algorithm
« on: November 04, 2021, 11:05:25 pm »
Here is something that might actually be useful, temperamental as hell but I finally got results after several compromises:
 
I had to ChDir to the search Directory to get good results from the GetLists Subroutine. I had never noted that before but it turns out I did do that for Oh Interpreter and other places I successfully used the cross platform files retriever. Dang! It's so frustrating to think you've got something nailed down, go to use it 6 months later and stumble into problems.

Then Windows wont let me ChDir in allot of places where _DirExists so I couldn't just start looking from C:\ for bas files nor even from C:\users\me and my stuff! blah!
But it did work OK from my Desktop and my Downloads Folders where I learn tonight over 10,000 bas files reside.

Takes about 1 sec to do the downloads folder that had 2115 bas files.
Code: QB64: [Select]
  1. _Title "Count BAS files from a start Directory" ' b+ 2021-11-04
  2.  
  3. ' direntry.h needs to be in QB64.exe folder, see below for a commented copy
  4.     Function load_dir& (s As String)
  5.     Function has_next_entry& ()
  6.     Sub close_dir ()
  7.     Sub get_next_entry (s As String, flags As Long, file_size As Long)
  8.  
  9. ReDim Shared BasList$(10000) 'store our Bas pathed Files here
  10. Dim Shared GrandTotal As _Unsigned Long ' ha, ha not that many!
  11.  
  12. FullPathedDir$ = "C:\Users\marka\downloads" ' <<<<<<<<<<<<<  change this line to top directory of your search for
  13. '                                                            best results use places where Windows lets you write bas files.
  14.  
  15. FindAndCountBasFileFrom FullPathedDir$
  16. Print " Grand Total .bas ="; GrandTotal
  17. Open FullPathedDir$ + "\Bas List.txt" For Output As #1
  18. For i = 1 To GrandTotal
  19.     Print #1, BasList$(i)
  20. Print FullPathedDir$ + "\Bas List.txt file is ready."
  21.  
  22.  
  23. Sub FindAndCountBasFileFrom (startDir$)
  24.     If startDir$ <> "." And startDir$ <> ".." And _Trim$(startDir$) <> "" Then
  25.         If Right$(startDir$, 1) <> "\" Then startDir$ = startDir$ + "\"
  26.         If _DirExists(startDir$) Then ChDir startDir$ Else Exit Sub ' >>> There are allot of places where dir exists but cant CD to go!
  27.         'Print "Changing Directory to "; startDir$; ""
  28.         ReDim ds(0) As String, fs(0) As String
  29.         GetLists startDir$, ds(), fs()
  30.         'Print startDir$ + " .bas Files:"
  31.         For i = LBound(fs) To UBound(fs)
  32.             If UCase$(Right$(fs(i), 4)) = ".BAS" Then
  33.                 GrandTotal = GrandTotal + 1
  34.                 Print GrandTotal, startDir$ + fs(i)
  35.                 BasList$(GrandTotal) = startDir$ + fs(i)
  36.                 'If i Mod 20 = 19 Then Print "Press any to cont..": Sleep
  37.             End If
  38.         Next
  39.         'Print
  40.         'Print startDir$ + " Sub-Directories:  zzz...": Sleep
  41.         For j = LBound(ds) To UBound(ds)
  42.             If ds(j) <> "." And ds(j) <> ".." And _Trim$(ds(j)) <> "" Then
  43.                 newD$ = startDir$ + ds(j)
  44.                 'Print "Press any to FindAndCountBasFileFrom " + newD$ + "... zzz": Sleep
  45.                 FindAndCountBasFileFrom newD$
  46.             End If
  47.         Next
  48.         'Print "Press any to cont...": Sleep
  49.     End If
  50.  
  51. Sub GetLists (SearchDirectory As String, DirList() As String, FileList() As String)
  52.  
  53.     '   !!! For this sub to work the _CWD has to be the Search Directory! !!!
  54.  
  55.     ' Thanks SNcNeill ! for a cross platform method to get file and directory lists
  56.     'put this block in main code section of your program close to top
  57.     '' direntry.h needs to be in QB64 folder '<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
  58.     'DECLARE CUSTOMTYPE LIBRARY ".\direntry"
  59.     '    FUNCTION load_dir& (s AS STRING)
  60.     '    FUNCTION has_next_entry& ()
  61.     '    SUB close_dir ()
  62.     '    SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
  63.     'END DECLARE
  64.  
  65.     Const IS_DIR = 1
  66.     Const IS_FILE = 2
  67.     Dim flags As Long, file_size As Long, DirCount As Integer, FileCount As Integer, length As Long
  68.     Dim nam$
  69.     ReDim _Preserve DirList(100), FileList(100)
  70.     DirCount = 0: FileCount = 0
  71.  
  72.     If load_dir(SearchDirectory + Chr$(0)) Then
  73.         Do
  74.             length = has_next_entry
  75.             If length > -1 Then
  76.                 nam$ = Space$(length)
  77.                 get_next_entry nam$, flags, file_size
  78.                 If (flags And IS_DIR) Then
  79.                     DirCount = DirCount + 1
  80.                     If DirCount > UBound(DirList) Then ReDim _Preserve DirList(UBound(DirList) + 1000)
  81.                     DirList(DirCount) = nam$
  82.                 ElseIf (flags And IS_FILE) Then
  83.                     FileCount = FileCount + 1
  84.                     If FileCount > UBound(FileList) Then
  85.                         ReDim _Preserve FileList(UBound(FileList) + 1000)
  86.                         'Print Left$(SearchDirectory, Len(searchDiretory) - 1)
  87.                         'getting allot of wierd misfires
  88.                         If Left$(SearchDirectory, Len(searchDiretory) - 1) = "C:\" Then Exit Sub
  89.                     End If
  90.                     FileList(FileCount) = nam$
  91.                 End If
  92.             End If
  93.         Loop Until length = -1
  94.         'close_dir 'move to after end if  might correct the multi calls problem
  95.     Else
  96.     End If
  97.     close_dir 'this  might correct the multi calls problem
  98.  
  99.     ReDim _Preserve DirList(DirCount)
  100.     ReDim _Preserve FileList(FileCount)
  101.  
  102. '  Remove comments below and save as direntry.h
  103. ' in your QB64.exe folder if you don't have it already
  104. '=============================================================
  105.  
  106. '#include <dirent.h>
  107. '#include <sys/stat.h>
  108. '#include <unistd.h>
  109.  
  110. 'const int IS_DIR_FLAG = 1, IS_FILE_FLAG = 2;
  111.  
  112. 'DIR *pdir;
  113. 'struct dirent *next_entry;
  114. 'struct stat statbuf1;
  115.  
  116. 'char current_dir[FILENAME_MAX];
  117. '#ifdef QB64_WINDOWS
  118. '  #define GetCurrentDir _getcwd
  119. '#else
  120. '  #define GetCurrentDir getcwd
  121. '#endif
  122.  
  123. 'int load_dir (char * path) {
  124. '  struct dirent *pent;
  125. '  struct stat statbuf1;
  126. '//Open current directory
  127. 'pdir = opendir(path);
  128. 'if (!pdir) {
  129. 'return 0; //Didn't open
  130. '}
  131. 'return -1;
  132. '}
  133.  
  134. 'int has_next_entry () {
  135. '  next_entry = readdir(pdir);
  136. '  if (next_entry == NULL) return -1;
  137.  
  138. '  stat(next_entry->d_name, &statbuf1);
  139. '  return strlen(next_entry->d_name);
  140. '}
  141.  
  142. 'void get_next_entry (char * nam, int * flags, int * file_size) {
  143. '  strcpy(nam, next_entry->d_name);
  144. '  if (S_ISDIR(statbuf1.st_mode)) {
  145. '    *flags = IS_DIR_FLAG;
  146. '  } else {
  147. '    *flags = IS_FILE_FLAG;
  148. '  }
  149. '  *file_size = statbuf1.st_size;
  150. '  return ;
  151. '}
  152.  
  153. 'void close_dir () {
  154. '  closedir(pdir);
  155. '  pdir = NULL;
  156. '  return ;
  157. '}
  158.  
  159. 'int current_dir_length () {
  160. '  GetCurrentDir(current_dir, sizeof(current_dir));
  161. '  return strlen(current_dir);
  162. '}
  163.  
  164. 'void get_current_dir(char *dir) {
  165. '  memcpy(dir, current_dir, strlen(current_dir));
  166. '  return ;
  167. '}
  168.  
  169.  

Theoretically this is cross platform, would non Windows users verify?

So how many bas files have you got laying around your computer?

Update: This code has nasty error in it and is missing allot of folders.
« Last Edit: November 08, 2021, 12:47:45 pm by bplus »

Offline Petr

  • Forum Resident
  • Posts: 1720
  • The best code is the DNA of the hops.
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #1 on: November 05, 2021, 06:43:02 am »
Quote
So how many bas files have you got laying around your computer?

Hi, just for curiosity - 7717 (but not all are my own sources)

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #2 on: November 05, 2021, 09:10:27 am »
Thanks Petr, I can assume it's working for you too on Windows probably.

Yeah most my bas files are probably not mine, some are yours.

For Linux I am wondering if the slash is pointed the right way for paths, supposedly for Windows it doesn't matter.

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #3 on: November 08, 2021, 12:50:26 pm »
oops!

I started digging into a smaller directory count and found it missing folders from a certain point onward.

I don't know if GetLists is still buggy or just trying to use it for this recursive code.

Offline Cobalt

  • QB64 Developer
  • Forum Resident
  • Posts: 878
  • At 60 I become highly radioactive!
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #4 on: November 08, 2021, 02:02:43 pm »
finds 2 files(in drive d:\, 3 files in drive E:\ and 0 files in c:\) then errors out with "PATH NOT FOUND" error.
Granted after becoming radioactive I only have a half-life!

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #5 on: November 08, 2021, 02:31:23 pm »
Well I never intended it to jump drives, probably should have. Do you think ChDir (without change drives also like we needed in DOS) could keep up with that?

GetLists routine needs to be in the Directory it's finding files and folders from = the compromise I am willing to accept to be able to do cross platform. No one else, including Spriggsy with PipeCom, has been able to produce simple File and Directory lists that one can use for navigation or as I am trying here: counting Bas Files in a directory and all it's subdirectories. (It's kinda odd because the IDE is doing it and very well, thank very much!, and I am assuming it uses something like the GetLists routine.)

Currently I am attempting a fix with Stack Techniques. I am pretty sure whatever is broken is from trying it with recursive calls. I am having no trouble navigating and finding files with GetLists in both TextFetch and the Oh Interpreter even after the QB64 v2.0 update, I've run both now with QB64 v 2.0. (Did have to fix Oh Interpreter in a couple of places.)

Offline TempodiBasic

  • Forum Resident
  • Posts: 1792
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #6 on: November 08, 2021, 02:37:05 pm »
Hi Bplus
here my feedback from Windows 8.1 pro

 
AllBAS  Bplus vs DIR.PNG


as you can see your application found more file than CMD : DIR *.BAS /S

What do I to think about it?
Programming isn't difficult, only it's  consuming time and coffee

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: Bas Files Count and List by Recursive Algorithm
« Reply #7 on: November 08, 2021, 03:10:21 pm »
Why use CHDIR?  There's a lot of hidden and system only directories which won't let you into them, but you can still get a directory listing from them.

For example, this is a recursive routine which gets ALL the file and directories from a folder and its subfolders -- warning, this might take a while to run if you choose a directory with a zillion files such as the root directory of C:\.  If you get tired of waiting, hit ESC and it'll dump what it's found so far to disk to and screen for you to view.

Code: QB64: [Select]
  1. _Title "Count BAS files from a start Directory" ' b+ 2021-11-04
  2.  
  3. ReDim Shared BasList$(10000) 'store our Bas pathed Files here
  4.  
  5. ReDim As String Dir(0)
  6.  
  7. GetAll "C:\", Dir()
  8. Open "DirList.txt" For Output As #1
  9. For i = 1 To UBound(Dir)
  10.     If _DirExists(Dir(i)) Then Color 7 Else Color 15
  11.     Print #1, Dir(i)
  12.     Print i, Dir(i)
  13.  
  14.  
  15. Sub GetAll (Path As String, DirList() As String)
  16.     If _KeyDown(27) Then Exit Sub
  17.     ReDim tempDir(0) As String
  18.     GetDir Path, tempDir()
  19.     For i = 1 To UBound(tempDir)
  20.         DirCount = UBound(DirList) + 1
  21.         ReDim _Preserve DirList(DirCount)
  22.         DirList(DirCount) = tempDir(i)
  23.         If _DirExists(tempDir(i)) And Right$(tempDir(i), 1) <> "." Then
  24.             DirList(DirCount) = DirList(DirCount) + "\"
  25.             GetAll tempDir(i), DirList()
  26.         End If
  27.  
  28.     Next
  29.  
  30.  
  31. Sub GetDir (SearchDirectory As String, DirList() As String)
  32.     Declare CustomType Library ".\direntry"
  33.         Function load_dir& (s As String)
  34.         Function has_next_entry& ()
  35.         Sub close_dir ()
  36.         Sub get_next_entry (s As String, flags As Long, file_size As Long)
  37.     End Declare
  38.  
  39.     Dim flags As Long, file_size As Long, DirCount As Integer, FileCount As Integer, length As Long
  40.     Dim nam$
  41.     DirCount = UBound(DirList)
  42.     If Right$(SearchDirectory, 1) = "\" Or Right$(SearchDirectory, 1) = "/" Then SearchDirectory = Left$(SearchDirectory, Len(SearchDirectory) - 1)
  43.     If load_dir(SearchDirectory + Chr$(0)) Then
  44.         Do
  45.             length = has_next_entry
  46.             If length > -1 Then
  47.                 nam$ = Space$(length)
  48.                 get_next_entry nam$, flags, file_size
  49.                 If Right$(nam$, 1) <> "." Then
  50.                     DirCount = DirCount + 1
  51.                     If DirCount > UBound(DirList) Then ReDim _Preserve DirList(UBound(DirList) + 1000)
  52.                     DirList(DirCount) = SearchDirectory + "\" + nam$
  53.                 End If
  54.             End If
  55.         Loop Until length = -1
  56.     End If
  57.     close_dir
  58.     ReDim _Preserve DirList(DirCount)
  59.  
  60. '  Remove comments below and save as direntry.h
  61. ' in your QB64.exe folder if you don't have it already
  62. '=============================================================
  63.  
  64. '#include <dirent.h>
  65. '#include <sys/stat.h>
  66. '#include <unistd.h>
  67.  
  68. 'const int IS_DIR_FLAG = 1, IS_FILE_FLAG = 2;
  69.  
  70. 'DIR *pdir;
  71. 'struct dirent *next_entry;
  72. 'struct stat statbuf1;
  73.  
  74. 'char current_dir[FILENAME_MAX];
  75. '#ifdef QB64_WINDOWS
  76. '  #define GetCurrentDir _getcwd
  77. '#else
  78. '  #define GetCurrentDir getcwd
  79. '#endif
  80.  
  81. 'int load_dir (char * path) {
  82. '  struct dirent *pent;
  83. '  struct stat statbuf1;
  84. '//Open current directory
  85. 'pdir = opendir(path);
  86. 'if (!pdir) {
  87. 'return 0; //Didn't open
  88. '}
  89. 'return -1;
  90. '}
  91.  
  92. 'int has_next_entry () {
  93. '  next_entry = readdir(pdir);
  94. '  if (next_entry == NULL) return -1;
  95.  
  96. '  stat(next_entry->d_name, &statbuf1);
  97. '  return strlen(next_entry->d_name);
  98. '}
  99.  
  100. 'void get_next_entry (char * nam, int * flags, int * file_size) {
  101. '  strcpy(nam, next_entry->d_name);
  102. '  if (S_ISDIR(statbuf1.st_mode)) {
  103. '    *flags = IS_DIR_FLAG;
  104. '  } else {
  105. '    *flags = IS_FILE_FLAG;
  106. '  }
  107. '  *file_size = statbuf1.st_size;
  108. '  return ;
  109. '}
  110.  
  111. 'void close_dir () {
  112. '  closedir(pdir);
  113. '  pdir = NULL;
  114. '  return ;
  115. '}
  116.  
  117. 'int current_dir_length () {
  118. '  GetCurrentDir(current_dir, sizeof(current_dir));
  119. '  return strlen(current_dir);
  120. '}
  121.  
  122. 'void get_current_dir(char *dir) {
  123. '  memcpy(dir, current_dir, strlen(current_dir));
  124. '  return ;
  125. '}
  126.  

Note that I've also tweaked the GetDir routine here so that it just does a simple file dump for us, without presorting files and directories for us.  (I also added the full path onto the file names, which I think is essential in a case of recursion so you can tell what the heck it found where...)

This has no issues on my PC starting at the root drive and working its way down into the various subdirectories, and as far as I can tell it's not missing anything.

Note 2:  This doesn't autoclear the old directory listing and builds upon it as you call it.  If you're going to call the routine multiple times, reset your listing with a fresh ReDim As String Dir(0) before making the call to GetAll.
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: Bas Files Count and List by Recursive Algorithm
« Reply #8 on: November 08, 2021, 03:26:59 pm »
And to just see a list of all the BAS files which you have, the following should work:

Code: QB64: [Select]
  1. _Title "Count BAS files from a start Directory" ' b+ 2021-11-04
  2.  
  3. ReDim Shared BasList$(100000) 'store our Bas pathed Files here
  4.  
  5. ReDim As String Dir(0)
  6.  
  7. GetAll "D:\", Dir()
  8. Open "BasList.txt" For Output As #1
  9. For i = 1 To UBound(Dir)
  10.     If _DirExists(Dir(i)) Then Color 7 Else Color 15
  11.     If _StriCmp(Right$(Dir(i), 4), ".bas") = 0 Then
  12.         'it's a BAS file
  13.         BasCount = BasCount + 1
  14.         BasList$(BasCount) = Dir(i)
  15.         Print BasCount, BasList$(BasCount)
  16.         Print #1, BasCount, BasList$(BasCount)
  17.     End If
  18.  
  19.  
  20. Sub GetAll (Path As String, DirList() As String)
  21.     If _KeyDown(27) Then Exit Sub
  22.     ReDim tempDir(0) As String
  23.     GetDir Path, tempDir()
  24.     For i = 1 To UBound(tempDir)
  25.         DirCount = UBound(DirList) + 1
  26.         ReDim _Preserve DirList(DirCount)
  27.         DirList(DirCount) = tempDir(i)
  28.         If _DirExists(tempDir(i)) And Right$(tempDir(i), 1) <> "." Then
  29.             DirList(DirCount) = DirList(DirCount) + "\"
  30.             GetAll tempDir(i), DirList()
  31.         End If
  32.  
  33.     Next
  34.  
  35.  
  36. Sub GetDir (SearchDirectory As String, DirList() As String)
  37.     Declare CustomType Library ".\direntry"
  38.         Function load_dir& (s As String)
  39.         Function has_next_entry& ()
  40.         Sub close_dir ()
  41.         Sub get_next_entry (s As String, flags As Long, file_size As Long)
  42.     End Declare
  43.  
  44.     Dim flags As Long, file_size As Long, DirCount As Integer, FileCount As Integer, length As Long
  45.     Dim nam$
  46.     DirCount = UBound(DirList)
  47.     If Right$(SearchDirectory, 1) = "\" Or Right$(SearchDirectory, 1) = "/" Then SearchDirectory = Left$(SearchDirectory, Len(SearchDirectory) - 1)
  48.     If load_dir(SearchDirectory + Chr$(0)) Then
  49.         Do
  50.             length = has_next_entry
  51.             If length > -1 Then
  52.                 nam$ = Space$(length)
  53.                 get_next_entry nam$, flags, file_size
  54.                 If Right$(nam$, 1) <> "." Then
  55.                     DirCount = DirCount + 1
  56.                     If DirCount > UBound(DirList) Then ReDim _Preserve DirList(UBound(DirList) + 1000)
  57.                     DirList(DirCount) = SearchDirectory + "\" + nam$
  58.                 End If
  59.             End If
  60.         Loop Until length = -1
  61.     End If
  62.     close_dir
  63.     ReDim _Preserve DirList(DirCount)
  64.  
  65. '  Remove comments below and save as direntry.h
  66. ' in your QB64.exe folder if you don't have it already
  67. '=============================================================
  68.  
  69. '#include <dirent.h>
  70. '#include <sys/stat.h>
  71. '#include <unistd.h>
  72.  
  73. 'const int IS_DIR_FLAG = 1, IS_FILE_FLAG = 2;
  74.  
  75. 'DIR *pdir;
  76. 'struct dirent *next_entry;
  77. 'struct stat statbuf1;
  78.  
  79. 'char current_dir[FILENAME_MAX];
  80. '#ifdef QB64_WINDOWS
  81. '  #define GetCurrentDir _getcwd
  82. '#else
  83. '  #define GetCurrentDir getcwd
  84. '#endif
  85.  
  86. 'int load_dir (char * path) {
  87. '  struct dirent *pent;
  88. '  struct stat statbuf1;
  89. '//Open current directory
  90. 'pdir = opendir(path);
  91. 'if (!pdir) {
  92. 'return 0; //Didn't open
  93. '}
  94. 'return -1;
  95. '}
  96.  
  97. 'int has_next_entry () {
  98. '  next_entry = readdir(pdir);
  99. '  if (next_entry == NULL) return -1;
  100.  
  101. '  stat(next_entry->d_name, &statbuf1);
  102. '  return strlen(next_entry->d_name);
  103. '}
  104.  
  105. 'void get_next_entry (char * nam, int * flags, int * file_size) {
  106. '  strcpy(nam, next_entry->d_name);
  107. '  if (S_ISDIR(statbuf1.st_mode)) {
  108. '    *flags = IS_DIR_FLAG;
  109. '  } else {
  110. '    *flags = IS_FILE_FLAG;
  111. '  }
  112. '  *file_size = statbuf1.st_size;
  113. '  return ;
  114. '}
  115.  
  116. 'void close_dir () {
  117. '  closedir(pdir);
  118. '  pdir = NULL;
  119. '  return ;
  120. '}
  121.  
  122. 'int current_dir_length () {
  123. '  GetCurrentDir(current_dir, sizeof(current_dir));
  124. '  return strlen(current_dir);
  125. '}
  126.  
  127. 'void get_current_dir(char *dir) {
  128. '  memcpy(dir, current_dir, strlen(current_dir));
  129. '  return ;
  130. '}
  131.  

No CHDIR involved.  Just calls to our GetDir routine with the path updating as it goes.
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #9 on: November 08, 2021, 04:25:10 pm »
Hi Bplus
here my feedback from Windows 8.1 pro

 
AllBAS  Bplus vs DIR.PNG


as you can see your application found more file than CMD : DIR *.BAS /S

What do I to think about it?

@TempodiBasic

Do you have some Basic files as .bas? Maybe your command only does capital BAS?

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #10 on: November 08, 2021, 04:38:25 pm »
And to just see a list of all the BAS files which you have, the following should work:

Code: QB64: [Select]
  1. _Title "Count BAS files from a start Directory" ' b+ 2021-11-04
  2.  
  3. ReDim Shared BasList$(100000) 'store our Bas pathed Files here
  4.  
  5. ReDim As String Dir(0)
  6.  
  7. GetAll "D:\", Dir()
  8. Open "BasList.txt" For Output As #1
  9. For i = 1 To UBound(Dir)
  10.     If _DirExists(Dir(i)) Then Color 7 Else Color 15
  11.     If _StriCmp(Right$(Dir(i), 4), ".bas") = 0 Then
  12.         'it's a BAS file
  13.         BasCount = BasCount + 1
  14.         BasList$(BasCount) = Dir(i)
  15.         Print BasCount, BasList$(BasCount)
  16.         Print #1, BasCount, BasList$(BasCount)
  17.     End If
  18.  
  19.  
  20. Sub GetAll (Path As String, DirList() As String)
  21.     If _KeyDown(27) Then Exit Sub
  22.     ReDim tempDir(0) As String
  23.     GetDir Path, tempDir()
  24.     For i = 1 To UBound(tempDir)
  25.         DirCount = UBound(DirList) + 1
  26.         ReDim _Preserve DirList(DirCount)
  27.         DirList(DirCount) = tempDir(i)
  28.         If _DirExists(tempDir(i)) And Right$(tempDir(i), 1) <> "." Then
  29.             DirList(DirCount) = DirList(DirCount) + "\"
  30.             GetAll tempDir(i), DirList()
  31.         End If
  32.  
  33.     Next
  34.  
  35.  
  36. Sub GetDir (SearchDirectory As String, DirList() As String)
  37.     Declare CustomType Library ".\direntry"
  38.         Function load_dir& (s As String)
  39.         Function has_next_entry& ()
  40.         Sub close_dir ()
  41.         Sub get_next_entry (s As String, flags As Long, file_size As Long)
  42.     End Declare
  43.  
  44.     Dim flags As Long, file_size As Long, DirCount As Integer, FileCount As Integer, length As Long
  45.     Dim nam$
  46.     DirCount = UBound(DirList)
  47.     If Right$(SearchDirectory, 1) = "\" Or Right$(SearchDirectory, 1) = "/" Then SearchDirectory = Left$(SearchDirectory, Len(SearchDirectory) - 1)
  48.     If load_dir(SearchDirectory + Chr$(0)) Then
  49.         Do
  50.             length = has_next_entry
  51.             If length > -1 Then
  52.                 nam$ = Space$(length)
  53.                 get_next_entry nam$, flags, file_size
  54.                 If Right$(nam$, 1) <> "." Then
  55.                     DirCount = DirCount + 1
  56.                     If DirCount > UBound(DirList) Then ReDim _Preserve DirList(UBound(DirList) + 1000)
  57.                     DirList(DirCount) = SearchDirectory + "\" + nam$
  58.                 End If
  59.             End If
  60.         Loop Until length = -1
  61.     End If
  62.     close_dir
  63.     ReDim _Preserve DirList(DirCount)
  64.  
  65. '  Remove comments below and save as direntry.h
  66. ' in your QB64.exe folder if you don't have it already
  67. '=============================================================
  68.  
  69. '#include <dirent.h>
  70. '#include <sys/stat.h>
  71. '#include <unistd.h>
  72.  
  73. 'const int IS_DIR_FLAG = 1, IS_FILE_FLAG = 2;
  74.  
  75. 'DIR *pdir;
  76. 'struct dirent *next_entry;
  77. 'struct stat statbuf1;
  78.  
  79. 'char current_dir[FILENAME_MAX];
  80. '#ifdef QB64_WINDOWS
  81. '  #define GetCurrentDir _getcwd
  82. '#else
  83. '  #define GetCurrentDir getcwd
  84. '#endif
  85.  
  86. 'int load_dir (char * path) {
  87. '  struct dirent *pent;
  88. '  struct stat statbuf1;
  89. '//Open current directory
  90. 'pdir = opendir(path);
  91. 'if (!pdir) {
  92. 'return 0; //Didn't open
  93. '}
  94. 'return -1;
  95. '}
  96.  
  97. 'int has_next_entry () {
  98. '  next_entry = readdir(pdir);
  99. '  if (next_entry == NULL) return -1;
  100.  
  101. '  stat(next_entry->d_name, &statbuf1);
  102. '  return strlen(next_entry->d_name);
  103. '}
  104.  
  105. 'void get_next_entry (char * nam, int * flags, int * file_size) {
  106. '  strcpy(nam, next_entry->d_name);
  107. '  if (S_ISDIR(statbuf1.st_mode)) {
  108. '    *flags = IS_DIR_FLAG;
  109. '  } else {
  110. '    *flags = IS_FILE_FLAG;
  111. '  }
  112. '  *file_size = statbuf1.st_size;
  113. '  return ;
  114. '}
  115.  
  116. 'void close_dir () {
  117. '  closedir(pdir);
  118. '  pdir = NULL;
  119. '  return ;
  120. '}
  121.  
  122. 'int current_dir_length () {
  123. '  GetCurrentDir(current_dir, sizeof(current_dir));
  124. '  return strlen(current_dir);
  125. '}
  126.  
  127. 'void get_current_dir(char *dir) {
  128. '  memcpy(dir, current_dir, strlen(current_dir));
  129. '  return ;
  130. '}
  131.  

No CHDIR involved.  Just calls to our GetDir routine with the path updating as it goes.

@SMcNeill

Thanks so much for this! Now I have confirmation that I have fixed my stuff. Though obviously I'd prefer to do the whole thing without the ChDir as you have.

There is some slight difference between GetLists from Oh Interpreter where it was working and here in OP that I've yet to pin down. Turns out I was using a Windows Only Technique for TextFetch.

I have 11,843 Bas files on my Desktop alone not 8,000+- I have 3 backup copies at least of all my BAS work files in various QB64 version numbers. I get this number from your code and the one I fixed with Oh Interpreter GetLists AND used a Non Recursive Stack because something dies in recursive version at 8080.

I will slowly get to using your much better version but first I am curious what the heck was the difference in GetLists.


Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Bas Files Count and List by Recursive Algorithm
« Reply #11 on: November 08, 2021, 07:16:32 pm »
Have to make Steve's solution Best Answer it so much better than recursive!

Just a tiny mod to put the file that it writes in the top directory of your search and count and list.
Code: QB64: [Select]
  1. FullPathedDir$ = "C:\Users\marka\Desktop\QB64 work"
  2. 'GetAll "D:\", Dir()
  3. GetAll FullPathedDir$, Dir()
  4. Open FullPathedDir$ + "\Count BasList SMcNeill.txt" For Output As #1 ' <put this in the directory we are doing
  5.  
« Last Edit: November 08, 2021, 07:38:25 pm by bplus »

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: Bas Files Count and List by Recursive Algorithm
« Reply #12 on: November 08, 2021, 07:55:50 pm »
Have to make Steve's solution Best Answer it so much better than recursive!

Just a tiny mod to put the file that it writes in the top directory of your search and count and list.
Code: QB64: [Select]
  1. FullPathedDir$ = "C:\Users\marka\Desktop\QB64 work"
  2. 'GetAll "D:\", Dir()
  3. GetAll FullPathedDir$, Dir()
  4. Open FullPathedDir$ + "\Count BasList SMcNeill.txt" For Output As #1 ' <put this in the directory we are doing
  5.  

Umm...  But it is recursive.  GetAll calls itself until done...  😂😂

And remember: ReDim As String Dir(0) before making the call to GetAll if you're going to call it again.  Reset your UBOUND to 0.
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Bas Files Count and List by Recursive Algorithm
« Reply #13 on: November 08, 2021, 08:10:13 pm »
I'd like to submit a simple alternative, if I may. I know this isn't recursive but I'm too lazy to do that right now.

Code: QB64 $NOPREFIX: [Select]
  1.  
  2. Type FILETIME
  3.     As Unsigned Long dwLowDateTime, dwHighDateTime
  4.  
  5. Type WIN32_FIND_DATA
  6.     As Unsigned Long dwFileAttributes
  7.     As FILETIME ftCreationTime, ftLastAccessTime, ftLastWriteTime
  8.     As Unsigned Long nFileSizeHigh, nFileSizeLow, dwReserved0, dwReserved1
  9.     As String * 260 cFileName
  10.     As String * 14 cAlternateFileName
  11.     As Unsigned Long dwFileType, dwCreatorType
  12.     As Unsigned Integer wFinderFlags
  13.  
  14.     Function FindFirstFile%& (ByVal lpFileName As Offset, Byval lpFindFileData As Offset)
  15.     Function FindNextFile& (ByVal hFindFile As Offset, Byval lpFindFileData As Offset)
  16.     Sub FindClose (ByVal hFindFile As Offset)
  17.  
  18. Dim As String searchTerm
  19. Dim As WIN32_FIND_DATA attr
  20.  
  21. searchTerm = ".\*.bas" + Chr$(0)
  22.  
  23. find = FindFirstFile(Offset(searchTerm), Offset(attr))
  24.  
  25. If find <> -1 And find <> 0 Then
  26.     Do
  27.         Print Mid$(attr.cFileName, 1, InStr(attr.cFileName, Chr$(0)) - 1)
  28.     Loop While FindNextFile(find, Offset(attr))
  29.  
  30. FindClose find
« Last Edit: November 08, 2021, 08:17:39 pm by SpriggsySpriggs »
Shuwatch!

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Bas Files Count and List by Recursive Algorithm
« Reply #14 on: November 08, 2021, 08:19:19 pm »
@TempodiBasic

Do you have some Basic files as .bas? Maybe your command only does capital BAS?

If I remember correctly, Steve had a bug in his code that would falsely identify some directories as files or vice-versa. Maybe that's why you have a difference in number of files found between his code and CMD.
Shuwatch!