Development of a program for automatic search for music and movies

[bplus]

Yes! I was trying to get that from something like: _SHELLHIDE "Tree / >Tree.txt"
but no file was getting created. My DOS command lines is rusty, maybe some CHR$(34)'s were needed?

I forgot that you didn't get full paths on each line.

[SMcNeill]

@bplus; @Petr:

Repeat after me: "I'm a big dummy.  I'm a big dummy.  I'm a big dummy."   Repeat for the next several minutes, then read WHY we're all such failures with such simple code -- yours truly included....


C strings are NOT the same as QB64 strings!!!

Everyone knows this, and yet, we all let it bite us in the ass, in this case!

Let's take for example a simple call to our function like so:

Code: [Select]

cd$ = _CWD$ 'testing in QB64 folder
MakeTree cd$

This fails to perform as expected.  (In some ways, I'm surprised to find that it performs at all!)

WHY??

Because _CWD$ returns a QB64 formatted string, and *NOT* a string as we'd normally expect to find in C.  It's *not* null terminated!!

Maketree cd$ + CHR$(0) is probably what we need to get it to work, in all honesty!  At least, playing around with my own little recursive routine, it's what was needed to (mostly) fix things.  My problem now is that I'm capturing too many recursions, and rechecking everything over and over endlessly...  The exit to my loop is broken somewhere.  :P

Anyway, I'd think if you want to sort out the bugs in your routines, you might want to null terminate your strings, and then a lot of the issues might end up going away for you guys.  It's worth a shot anyways.  I'm going to keep playing around with my own little thing, and I'll keep you updated with what I come up with as I progress.  ;)

[SMcNeill]

See if this works as expected for getting the directories:

Code: QB64: [Select]

1. OPTION _EXPLICIT 'Tree builder recursive.bas  b+ Petr 2020-06-26
2. DEFLNG A-Z
3. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
4.     FUNCTION load_dir& (s AS STRING)
5.     FUNCTION has_next_entry& ()
6.     SUB close_dir ()
7.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
8. END DECLARE
9.  
10. REDIM SHARED Tree(0) AS STRING
11.  
12. SCREEN _NEWIMAGE(1280, 740, 32)
13. _DELAY .25
14. _SCREENMOVE 80, 0
15. DIM cd$, i, w$
16.  
17.  
18. MakeTree _CWD$ 'testing in QB64 folder
19.  
20. PRINT "Showing Tree"
21. FOR i = 1 TO UBOUND(Tree) 'show tree
22.     PRINT _TRIM$(STR$(i)); ": "; Tree(i)
23.     IF i MOD 40 = 0 THEN INPUT "Press enter to continue... "; w$: CLS
24. NEXT
25.  
26.  
27. SUB GetSubDirs (SearchDirectory AS STRING)
28.     CONST IS_DIR = 1
29.     DIM flags AS LONG, file_size AS LONG, length, nam$
30.     IF load_dir(SearchDirectory + CHR$(0)) THEN
31.         DO
32.             length = has_next_entry
33.             IF length > -1 THEN
34.                 nam$ = SPACE$(length)
35.                 get_next_entry nam$, flags, file_size
36.                 IF RIGHT$(nam$, 1) <> "." AND RIGHT$(nam$, 2) <> ".." THEN
37.                     IF flags = IS_DIR OR _DIREXISTS(SearchDirectory + "\" + nam$) THEN
38.                         REDIM _PRESERVE Tree(UBOUND(Tree) + 1)
39.                         Tree(UBOUND(Tree)) = SearchDirectory + "\" + nam$
40.                     END IF
41.                 END IF
42.             END IF
43.         LOOP UNTIL length = -1
44.     ELSE
45.         PRINT "Dir not loaded"
46.     END IF
47.     close_dir
48. END SUB
49.  
50. SUB MakeTree (Dir$)
51.     DIM OnDir AS LONG
52.     REDIM Tree(0) AS STRING
53.     Tree(0) = Dir$
54.     DO
55.         GetSubDirs Tree(OnDir)
56.         OnDir = OnDir + 1
57.     LOOP UNTIL OnDir > UBOUND(Tree)
58. END SUB
59.  

Biggest and most important change??

As I pointed out above -- null terminate your strings:     IF load_dir(SearchDirectory + CHR$(0)) THEN


NOTE: This change should probably carry over to ANY routine which uses load_dir with direntry.h.  I think we've all been goofing on calling the function all this time, and I'm surprised nobody has noticed or found the glitch before now.  Multiple, repetitive calls, all in a row, is a good way to stress test these routines, I guess.  :D

Anywho....  Test it out, see if it looks and acts about like it should, and we'll debug and sort out any issues with things as we go.  ;D

[bplus]

Bravo! Steve! that did the trick alright.

Curiously, I get one more Directory with your code, that I like because you eliminated the need to CHDIR.

[SpriggsySpriggs]

I forgot that you didn't get full paths on each line.

It doesn't matter because since you have the formatting of the tree, you always know what the path is. You would just have to write a function to parse that tree.

[SpriggsySpriggs]

See if this works as expected for getting the directories:

@SMcNeill I'm thinking the tree structure isn't correct, Steve. Or maybe this is intentional but see my screenshot from running your code:
 

Shouldn't all the folders in that auto\ folder be together and not separated out? Same for include\
Something seems to be amiss in the sorting, perhaps.
It appears to do that in several spots from what I can see. I would have assumed that the tree structure puts all subdirectories of a folder with the parent folder and then move on to the next folder like my version:
 

[SMcNeill]

Easy way to test that Spriggsy:  run both routines, sort them, and then compare line by line entries to see what's different (if anything).  To be honest, I'm not certain why there's jumps in the output like that either, yet.  ;)

[SpriggsySpriggs]

Easy way to test that Spriggsy:  run both routines, sort them, and then compare line by line entries to see what's different (if anything).  To be honest, I'm not certain why there's jumps in the output like that either, yet.  ;)

Both programs give me the same number of directories. It's just a matter of the structure.

[SMcNeill]

Both programs give me the same number of directories. It's just a matter of the structure.

It's just the sorting method.  I'm thinking, if you look close, my little routine finds the directories in levels deep that it goes.

....\auto\core is 14 levels deep.
....\build\vc10 is 14 levels deep.
....\auto\core\gl is 15 levels deep.

As long as it finds all the directories properly, they can be sorted and arraigned as desired later.  ;)

[bplus]

Well the disadvantage of not using CHDIR is having to sort later?

This looks to be in pretty good order:

Code: QB64: [Select]

1. 'Tree builder recursive.bas  b+ Petr 2020-06-26  now with SMCNeill Fix!! add CHR$(0) to C string
2. $CONSOLE:ONLY
3. _DEST _CONSOLE
4. DECLARE CUSTOMTYPE LIBRARY ".\direntry"
5.     FUNCTION load_dir& (s AS STRING)
6.     FUNCTION has_next_entry& ()
7.     SUB close_dir ()
8.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
9. END DECLARE
10.  
11. REDIM SHARED Tree(0) AS STRING '<<<<<<<<<<<<<<<<<<<<<< G(0) will remain empty
12.  
13. MakeTree _CWD$
14. FOR i = 1 TO UBOUND(Tree) 'show tree
15.     PRINT _TRIM$(STR$(i)); ": "; Tree(i)
16. NEXT
17. SLEEP
18.  
19. SUB sAppend (arr() AS STRING, addItem$)
20.     REDIM _PRESERVE arr(LBOUND(arr) TO UBOUND(arr) + 1) AS STRING
21.     arr(UBOUND(arr)) = addItem$
22. END SUB
23.  
24. SUB GetSubDirs (SearchDirectory AS STRING, DirList() AS STRING)
25.     CONST IS_DIR = 1
26.     DIM flags AS LONG, file_size AS LONG
27.     IF load_dir(SearchDirectory + CHR$(0)) THEN 'Steve's fix here with CHR$(0) for C call
28.         DO
29.             length = has_next_entry
30.             IF length > -1 THEN
31.                 nam$ = SPACE$(length)
32.                 get_next_entry nam$, flags, file_size
33.                 IF (flags AND IS_DIR) OR _DIREXISTS(SearchDirectory + nam$) THEN
34.                     IF nam$ <> "." AND nam$ <> ".." THEN
35.                         DirCount = DirCount + 1
36.                         IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
37.                         DirList(DirCount) = SearchDirectory + "\" + nam$
38.                     END IF
39.                 END IF
40.             END IF
41.         LOOP UNTIL length = -1
42.     END IF
43.     close_dir 'Steve first fix that got navigator working
44.     REDIM _PRESERVE DirList(DirCount)
45. END SUB
46.  
47. SUB MakeTree (startDir AS STRING)
48.     REDIM D(100) AS STRING
49.     GetSubDirs startDir, D()
50.     FOR i = 1 TO UBOUND(D)
51.         sAppend Tree(), D(i)
52.         CHDIR D(i)
53.         MakeTree D(i)
54.     NEXT
55. END SUB
56.  
57.  

 

[SpriggsySpriggs]

As long as it finds all the directories properly, they can be sorted and arraigned as desired later.  ;)

Your logic is showing a file in the QB64 directory as a folder but is just a file without an extension.

\internal\c\c_compiler\licenses\libffi\LICENSE

I can run a sort on your file using a PowerShell command and the entire program takes around 6 seconds to complete from getting the directories to spitting out the sorted version.
If I run my program using the PowerShell version, it only takes 3 seconds because it is already sorted. I verified this by using the PowerShell script to sort my file and then compared it in Notepad ++. The files were identical.
EDIT: It looks like @bplus has solved the issue with using direntry

[bplus]

I am impressed that Steve managed it without a recursive procedure.

[SpriggsySpriggs]

@bplus I did a test to see which way was faster and it looks like your code and mine are quite close. Mine finished in 3.380 seconds whereas yours took 3.626 seconds. I also compared the files in Notepad++ and they were identical. Way to go! EDIT: I ran it again using only _TRIM$ on the value from the file and mine finished in 3.109 seconds. Only a half second's difference is quite impressive! I thought for sure that mine would lose against yours.
See screenshot:
 

 

[bplus]

I thought for sure that mine would lose against yours.

Ha! mine was probably slowed down by all the CHDIR's. Your's is working under the Power of Power Shell ;-))

[SpriggsySpriggs]

Ha! mine was probably slowed down by all the CHDIR's. Your's is working under the Power of Power Shell ;-))

I expected that with shelling out to PowerShell that I would lose precious time because I'm having to wait on an external process to complete. I know that when I generate my WPF Message Boxes from PowerShell they are much slower than calling a native message box (of course, mine are more graphical and the script is quite long. Eh, whatever)