Reading Directories and Files without SHELLing out

[Cobalt]

So I'm trying to add the ability to save (or load) game files from different directories for some odds and ends I am working on. Last time I did this I used DIRECTQBs routines. And I'm finding QB64 lacks any inbuilt way of loading Directories. and _DIR$ is sadly all but useless.(BTW why is that?)  I did however find a little gem by none other than Steve.

But I found a problem Steve,
it only recognizes  '.' and '..' as directories and treats everything else as files. it comes from a post on your forum which I found searching this forum for ways of obtaining a directory list.

Code: QB64: [Select]

1. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
2.  FUNCTION load_dir& (s AS STRING)
3.  FUNCTION has_next_entry& ()
4.  SUB close_dir ()
5.  SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
6. END DECLARE
7.  
8. REDIM Dir(0) AS STRING, File(0) AS STRING
9.  
10. 'D$ = _CWD$
11. D$ = "E:\SMB64\"
12. GetFileList D$, Dir(), File()
13.  
14. PRINT "SUBDIRECTORIES"; UBOUND(dir)
15. FOR i = 1 TO UBOUND(dir)
16.  PRINT Dir(i),
17. NEXT
18. PRINT
19. SLEEP
20.  
21. PRINT "FILES"; UBOUND(file): PRINT: PRINT
22. SLEEP
23. FOR i = 1 TO UBOUND(file)
24.  PRINT File(i),
25. NEXT
26. PRINT
27.  
28. SUB GetFileList (SearchDirectory AS STRING, DirList() AS STRING, FileList() AS STRING)
29.  CONST IS_DIR = 1
30.  CONST IS_FILE = 2
31.  DIM flags AS LONG, file_size AS LONG
32.  
33.  REDIM _PRESERVE DirList(100), FileList(100)
34.  DirCount = 0: FileCount = 0
35.  
36.  IF load_dir(SearchDirectory) THEN
37.   DO
38.    length = has_next_entry
39.    IF length > -1 THEN
40.     nam$ = SPACE$(length)
41.     get_next_entry nam$, flags, file_size
42.     PRINT flags;
43.     IF flags AND IS_DIR THEN
44.      DirCount = DirCount + 1
45.      IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
46.      DirList(DirCount) = nam$
47.     ELSEIF flags AND IS_FILE THEN
48.      FileCount = FileCount + 1
49.      IF FileCount > UBOUND(filelist) THEN REDIM _PRESERVE FileList(UBOUND(filelist) + 100)
50.      FileList(FileCount) = nam$
51.     END IF
52.    END IF
53.   LOOP UNTIL length = -1
54.   close_dir
55.  ELSE
56.  END IF
57.  REDIM _PRESERVE DirList(DirCount)
58.  REDIM _PRESERVE FileList(FileCount)
59. END SUB
60.  

I added a couple of lines to see what was going on, printing the UBOUNDs for DIR() and FILE() as well as the FLAGS variable in the sub.

I've attached the output I get from my Super Mario directory, as you can see it only says there are 2 directories, yet anything listed in the file section without an extension is a directory too. This is true only cause I always make sure to have an extension on my files and never in my directories. Not everybody does so I couldn't use that as a sorting method.

I've also attached the header file needed to run the program in case anybody else wanted to take a look. as I can't remember the link to the topic its under on your forum right off this late. If nobody beats me to it I'll find it and post the link when I wake up. as for now I'm going to sleep.

[doppler]

Usable now, but when fully fleshed out.  Will become another tool bag trick for me.

[SMcNeill]

What is drive D for you?  I've never ran into any trouble with the routine sorting the files and directories for me, so I'm curious why it's failing  for you.  These are just simple little POSIX routines, so they're generally very reliable, and, so far, I don't think I've  ever gotten the results you're having.

Is D a USB drive?  An old drive, perhaps in a format it might not recognize?  (FAT16 or some such?)  Are you using an old OS which might not have the fully functional POSIX functions?  There's a lot of little things which it could be, but I don't have a clue as to why it's not sorting properly for you.  Test it out for me, if you can, on a few different drives/machines and let me know which ones it does and doesn't work for.  Once I know exactly what architecture it fails to sort on, then I might be able to figure out a way to patch the issue so we can work around it without any issues.

When it does fail to autosort, the easy solution is to just place them all into a single list and then use _FILEEXISTS and _DIREXISTS to sort them for file/directory usage.

EDIT:  In fact, since it's a simple enough change, I'll just add it directly into the routine itself as a secondary means of sorting/testing for us.  Try this and see if it sorts on your system for you:

Code: QB64: [Select]

1. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
2.     FUNCTION load_dir& (s AS STRING)
3.     FUNCTION has_next_entry& ()
4.     SUB close_dir ()
5.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
6. END DECLARE
7.  
8. REDIM Dir(0) AS STRING, File(0) AS STRING
9.  
10. 'D$ = _CWD$
11. D$ = _CWD$
12. GetFileList D$, Dir(), File()
13.  
14. PRINT "SUBDIRECTORIES"; UBOUND(dir)
15. FOR i = 1 TO UBOUND(dir)
16.     PRINT Dir(i),
17. NEXT
18. PRINT
19. SLEEP
20.  
21. PRINT "FILES"; UBOUND(file): PRINT: PRINT
22. SLEEP
23. FOR i = 1 TO UBOUND(file)
24.     PRINT File(i),
25. NEXT
26. PRINT
27.  
28. SUB GetFileList (SearchDirectory AS STRING, DirList() AS STRING, FileList() AS STRING)
29.     CONST IS_DIR = 1
30.     CONST IS_FILE = 2
31.     DIM flags AS LONG, file_size AS LONG
32.  
33.     REDIM _PRESERVE DirList(100), FileList(100)
34.     DirCount = 0: FileCount = 0
35.  
36.     IF load_dir(SearchDirectory) THEN
37.         DO
38.             length = has_next_entry
39.             IF length > -1 THEN
40.                 nam$ = SPACE$(length)
41.                 get_next_entry nam$, flags, file_size
42.                 PRINT flags;
43.                 IF flags AND IS_DIR OR _DIREXISTS(nam$) THEN
44.                     DirCount = DirCount + 1
45.                     IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
46.                     DirList(DirCount) = nam$
47.                 ELSEIF flags AND IS_FILE OR _FILEEXISTS(nam$) THEN
48.                     FileCount = FileCount + 1
49.                     IF FileCount > UBOUND(filelist) THEN REDIM _PRESERVE FileList(UBOUND(filelist) + 100)
50.                     FileList(FileCount) = nam$
51.                 END IF
52.             END IF
53.         LOOP UNTIL length = -1
54.         close_dir
55.     ELSE
56.     END IF
57.     REDIM _PRESERVE DirList(DirCount)
58.     REDIM _PRESERVE FileList(FileCount)
59. END SUB

[Cobalt]

If you mean drive E, its a partition on my laptop, on the main hard disk. Does the same thing on my desktop when trying to read the structure of a different specified location.

However, it does seem to work when using D$ = _CWD$ and reading the execution directory. Though that is not very helpful if a user wants to change directories when saving or loading.

[SMcNeill]

Does it work with the change above?  If not, I’ll need to do a lot of digging to see what the difference is. 

[Cobalt]

Only if your reading the Execution directory, as soon as you pick a new directory to read it fails to separate the directories from the files again.

what does yours do if you have it try to read D$="C:\" instead of D$= _CWD$ ?

[SMcNeill]

Only if your reading the Execution directory, as soon as you pick a new directory to read it fails to separate the directories from the files again.

what does yours do if you have it try to read D$="C:\" instead of D$= _CWD$ ?

You're right.  The issue is relative paths for us.   _DIREXISTS("my folder\") isn't the same thing as _DIREXISTS("C:\my folder\").  We strip those paths out so all we get back is the file structure for the specified folder, and not the WHOLE path + file/directory listing each time.

Try this patch and see how it performs for you:

Code: QB64: [Select]

1. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
2.     FUNCTION load_dir& (s AS STRING)
3.     FUNCTION has_next_entry& ()
4.     SUB close_dir ()
5.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
6. END DECLARE
7.  
8. REDIM Dir(0) AS STRING, File(0) AS STRING
9.  
10. 'D$ = _CWD$
11. D$ = "C:\"
12. GetFileList D$, Dir(), File()
13.  
14. PRINT "SUBDIRECTORIES"; UBOUND(dir)
15. FOR i = 1 TO UBOUND(dir)
16.     PRINT Dir(i),
17. NEXT
18. PRINT
19. PRINT
20. SLEEP
21.  
22. PRINT "FILES"; UBOUND(file): PRINT: PRINT
23. SLEEP
24. FOR i = 1 TO UBOUND(file)
25.     PRINT File(i),
26. NEXT
27. PRINT
28.  
29. SUB GetFileList (SearchDirectory AS STRING, DirList() AS STRING, FileList() AS STRING)
30.     CONST IS_DIR = 1
31.     CONST IS_FILE = 2
32.     DIM flags AS LONG, file_size AS LONG
33.  
34.     REDIM _PRESERVE DirList(100), FileList(100)
35.     DirCount = 0: FileCount = 0
36.  
37.     IF load_dir(SearchDirectory) THEN
38.         DO
39.             length = has_next_entry
40.             IF length > -1 THEN
41.                 nam$ = SPACE$(length)
42.                 get_next_entry nam$, flags, file_size
43.                 IF (flags AND IS_DIR) OR _DIREXISTS(SearchDirectory + nam$) THEN
44.                     DirCount = DirCount + 1
45.                     IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
46.                     DirList(DirCount) = nam$
47.                 ELSEIF (flags AND IS_FILE) OR _FILEEXISTS(SearchDirectory + nam$) THEN
48.                     FileCount = FileCount + 1
49.                     IF FileCount > UBOUND(filelist) THEN REDIM _PRESERVE FileList(UBOUND(filelist) + 100)
50.                     FileList(FileCount) = nam$
51.                 END IF
52.             END IF
53.         LOOP UNTIL length = -1
54.         close_dir
55.     ELSE
56.     END IF
57.     REDIM _PRESERVE DirList(DirCount)
58.     REDIM _PRESERVE FileList(FileCount)
59. END SUB

[Cobalt]

That seems to do the trick. Thanks Steve.

Odd that something that should be really simple takes so much effort anymore.

[bplus]

I am back to . and .. with all the sub-directories as files when I change D$ = "C:\" to my users folder "C:\users\marka"

Am I misunderstanding how to use this? Should SearchDirectory really mean search drive?

[Petr]

Hi. I think, this very small SUB solved all...
PLEASE TEST IT, if it works for you.

Code: QB64: [Select]

1. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
2.     FUNCTION load_dir& (s AS STRING)
3.     FUNCTION has_next_entry& ()
4.     SUB close_dir ()
5.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
6. END DECLARE
7.  
8. REDIM Dir(0) AS STRING, File(0) AS STRING
9.  
10. D$ = _CWD$
11. 'D$ = "" 'INSERT PATH HERE
12.  
13.  
14.  
15. SetPath D$
16. GetFileList D$, Dir(), File()
17.  
18. PRINT "SUBDIRECTORIES"; UBOUND(dir)
19. FOR i = 1 TO UBOUND(dir)
20.     PRINT Dir(i),
21. NEXT
22. PRINT
23. PRINT
24. SLEEP
25.  
26. PRINT "FILES"; UBOUND(file): PRINT: PRINT
27. SLEEP
28. FOR i = 1 TO UBOUND(file)
29.     PRINT File(i),
30. NEXT
31. PRINT
32.  
33.  
34. SUB SetPath (p$)
35.     L$ = RIGHT$(p$, 1)
36.     $IF WIN THEN
37.         IF L$ <> "\" THEN p$ = p$ + "\"
38.     $ELSEIF LIN THEN
39.         if L$ <> "/" then p$ = p$ + "/"
40.     $END IF
41. END SUB
42.  
43.  
44.  
45. SUB GetFileList (SearchDirectory AS STRING, DirList() AS STRING, FileList() AS STRING)
46.     CONST IS_DIR = 1
47.     CONST IS_FILE = 2
48.     DIM flags AS LONG, file_size AS LONG
49.  
50.     REDIM _PRESERVE DirList(100), FileList(100)
51.     DirCount = 0: FileCount = 0
52.  
53.     IF load_dir(SearchDirectory) THEN
54.         DO
55.             length = has_next_entry
56.             IF length > -1 THEN
57.                 nam$ = SPACE$(length)
58.                 get_next_entry nam$, flags, file_size
59.                 IF (flags AND IS_DIR) OR _DIREXISTS(SearchDirectory + nam$) THEN
60.                     DirCount = DirCount + 1
61.                     IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
62.                     DirList(DirCount) = nam$
63.                 ELSEIF (flags AND IS_FILE) OR _FILEEXISTS(SearchDirectory + nam$) THEN
64.                     FileCount = FileCount + 1
65.                     IF FileCount > UBOUND(filelist) THEN REDIM _PRESERVE FileList(UBOUND(filelist) + 100)
66.                     FileList(FileCount) = nam$
67.                 END IF
68.             END IF
69.         LOOP UNTIL length = -1
70.         close_dir
71.     ELSE
72.     END IF
73.     REDIM _PRESERVE DirList(DirCount)
74.     REDIM _PRESERVE FileList(FileCount)
75. END SUB
76.  
77.  

[bplus]

Thanks Petr,

I guess I was using it incorrectly, I needed a slash at the end for the SearchDirectory variable.

Well this should work nicely in my Tiny Navigator which could then work in Linux?

For Windows, either slash direction works, you can even mix and match :)

[doppler]

Thank-you, Thank-you, Thank-you

I have been using shell and capture output to mange files and directories for so long.

Using less shells and this code is so much more efficient.

Why anyone would want to use the "FILES" command is beyond me at this point now.
Why look at files listed, when you can't do a damn thing with them.

[Petr]

Hi BPlus, yeah, direntry.h is compatible with Linux, so your file browser will also work in Linux. You just have to be careful of the characters size in the names of folders and files, that's what Linux is sensitive to.
But there is also an option in Linux such as DIR to a file in windows, so you do this in Linux -   with command "ls > file.txt".

Hi Doppler. FILES is a prehistory maintained for compatibility with older Stone Age ...  :)

[Cobalt]

Glad so many people are happy I brought this up, As I stated in my first post it is a little Gem of a routine.

Won't actually return file size, but you could always add a simple OPEN:LOF:CLOSE to get file sizes as it reads in the files(I mean put the code in the ELSEIF area). I do not need that functionality in what I'm working on, but something like Bplus is talking about you might what to show file size as well.

Wonder why they made the Linux OS so picky? Maybe its just me but I can't see the need for case sensitivity in file names. Be like having a variable named ABCD and having 15 variations on it simply by using upper and lower case, that would be horrible to keep track of I would think! The forward-backward slash thing I don't know, I tend to use either one at random which may be bad practice in reality, but I still feel its rather nit picky to be arbitrarily enforcing one or the other.

[SMcNeill]

Glad so many people are happy I brought this up, As I stated in my first post it is a little Gem of a routine.

Won't actually return file size, but you could always add a simple OPEN:LOF:CLOSE to get file sizes as it reads in the files(I mean put the code in the ELSEIF area). I do not need that functionality in what I'm working on, but something like Bplus is talking about you might what to show file size as well.

It *does* get the file_size for us; we just never return it anywhere to make use of it:

Code: [Select]

get_next_entry nam$, flags, file_size
If you need file size, change the routine just a bit and make your file listing an array to hold that information, like so:

Code: QB64: [Select]

1. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
2.     FUNCTION load_dir& (s AS STRING)
3.     FUNCTION has_next_entry& ()
4.     SUB close_dir ()
5.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
6. END DECLARE
7.  
8. TYPE File_Type
9.     Size AS _INTEGER64
10.     Name AS STRING
11. END TYPE
12.  
13.  
14. REDIM Dir(0) AS STRING, File(0) AS File_Type
15.  
16. D$ = _CWD$
17. D$ = "C:\"
18. GetFileList D$, Dir(), File()
19.  
20. PRINT "SUBDIRECTORIES"; UBOUND(dir)
21. FOR i = 1 TO UBOUND(dir)
22.     PRINT Dir(i)
23. NEXT
24. PRINT
25. PRINT
26. SLEEP
27.  
28. PRINT "FILES"; UBOUND(file): PRINT: PRINT
29. SLEEP
30. FOR i = 1 TO UBOUND(file)
31.     PRINT File(i).Name, File(i).Size
32. NEXT
33. PRINT
34.  
35. SUB GetFileList (SearchDirectory AS STRING, DirList() AS STRING, FileList() AS File_Type)
36.     CurrDir$ = _CWD$
37.     CHDIR SearchDirectory
38.     CONST IS_DIR = 1
39.     CONST IS_FILE = 2
40.     DIM flags AS LONG, file_size AS LONG
41.  
42.     REDIM _PRESERVE DirList(100), FileList(100) AS File_Type
43.     DirCount = 0: FileCount = 0
44.  
45.     IF load_dir(_CWD$) THEN
46.         DO
47.             length = has_next_entry
48.             IF length > -1 THEN
49.                 nam$ = SPACE$(length)
50.                 get_next_entry nam$, flags, file_size
51.                 IF flags AND IS_DIR THEN
52.                     DirCount = DirCount + 1
53.                     IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
54.                     DirList(DirCount) = nam$
55.                 ELSEIF flags AND IS_FILE THEN
56.                     FileCount = FileCount + 1
57.                     IF FileCount > UBOUND(filelist) THEN REDIM _PRESERVE FileList(UBOUND(filelist) + 100) AS File_Type
58.                     FileList(FileCount).Name = nam$
59.                     FileList(FileCount).Size = file_size
60.                 END IF
61.             END IF
62.         LOOP UNTIL length = -1
63.         close_dir
64.     ELSE
65.     END IF
66.     REDIM _PRESERVE DirList(DirCount)
67.     REDIM _PRESERVE FileList(FileCount) AS File_Type
68.     CHDIR CurrDir$
69. END SUB
70.  

(And, since this also temporarily changes the path for us, it corrects the issue you were experiencing before, where relative path vs absolute path caused issues.  No worries though; it changes it back once it's finished, so your program won't have any pathing problems when you plug it in.)