Author Topic: Development of a program for automatic search for music and movies  (Read 21643 times)

0 Members and 1 Guest are viewing this topic.

Offline Petr

  • Forum Resident
  • Posts: 1720
  • The best code is the DNA of the hops.
    • View Profile
Re: Development of a program for automatic search for music and movies
« Reply #75 on: June 27, 2020, 12:26:08 pm »
Quote
Which tags do you already have code to get? Here is code I have for tags:

Hi, SpriggsySpriggs i find my source code now. It is from 7.3.2017 :) so it is really not fresh in my memory.
Here:

Code: QB64: [Select]
  1. 'After writing ID3v1.1 decoder for Qb64 I wrote this decoder ID3TAGV2.3.0 for QB64. I want in first thank Clippy for sharing his program, which dealt with the same thing in another way.
  2. 'Hardest of all was to understand how the author ID3TAG thinks with the calculation of the size HEAD in binary code. If he wrote it straight - sort by itself without the eighth bit and convert to decimal form,
  3. 'probably would have consumed much paper. That gave me the most work.
  4.  
  5. 'this is extended version. Program first use V2.3.0 program. IF this ends with error (no record found), is automatically runned upgraded version ID3V1.1. Only when one can not find even a record
  6. 'identifier TAG, then the program will report that in the file is none ID3 and ends. Read NOT ID3V2.2 (V2.2 is with 3 characters indetificator), but read ID3V2.4
  7.  
  8.  
  9.  
  10. _TITLE "ID3V2.3 PLUS ID3V1.1 (V1.1 upgraded) Reader"
  11. INPUT "Input MP3 filename without extension to view ID3:"; file$
  12. file$ = file$ + ".MP3"
  13. 'file$ = "14.mp3" 's.mp3 hav e APIC.
  14. IF _FILEEXISTS(file$) = 0 THEN PRINT "Not found": SLEEP: SYSTEM
  15.  
  16. TYPE ID3HEAD
  17.     Identifier AS STRING * 3
  18.     VersionMajor AS STRING * 1 '                                           Classical way - ASC number of this characters in file is Version number
  19.     VersionRevis AS STRING * 1
  20.     BinarFlags AS _UNSIGNED _BYTE
  21.     BinarSizeA AS STRING * 1
  22.     BinarSizeB AS STRING * 1
  23.     BinarSizeC AS STRING * 1
  24.     BinarSizeD AS STRING * 1
  25.  
  26. TYPE FRAME
  27.     Id AS STRING * 4
  28.     SizeA AS STRING * 1
  29.     SizeB AS STRING * 1
  30.     SizeC AS STRING * 1
  31.     SizeD AS STRING * 1
  32.     Flags AS _UNSIGNED INTEGER
  33.  
  34. DIM SHARED ID3 AS ID3HEAD, FRM AS FRAME
  35. DIM OUTList(150) AS STRING * 80 '                                                 output list with infos from ID3TAG
  36.  
  37. OPEN file$ FOR BINARY AS #1
  38. GET #1, , ID3
  39.  
  40. IF ID3.Identifier$ <> "ID3" THEN PRINT "ID3 mark not found": IDENTIT$ = "ID3V1.1": GOTO ID3V11 ' BEEP: PRINT ID3.Identifier$: SLEEP: SYSTEM
  41. Version$ = "2." + LTRIM$(RTRIM$(STR$(ASC(ID3.VersionMajor$)))) + "." + LTRIM$(STR$(ASC(ID3.VersionRevis$))): PRINT "Detected ID3 v."; Version$: IF Version$ <> "2.3.0" THEN PRINT "WARNING, WRONG VERSION, THIS IS V2.3!!!"
  42. IDENTIT$ = Version$
  43. PRINT "Binaries (bites) in FLAGS: ";: PRINT DECtoBIN$(ID3.BinarFlags);: PRINT " HEX: ";: PRINT BINtoHEX$(DECtoBIN$(ID3.BinarFlags))
  44. Ba$ = DECtoBIN$(ASC(ID3.BinarSizeA$))
  45. Bb$ = DECtoBIN$(ASC(ID3.BinarSizeB$)) '                                     FUNCTION converted decimal numbers to binar numbers
  46. Bc$ = DECtoBIN$(ASC(ID3.BinarSizeC$))
  47. Bd$ = DECtoBIN$(ASC(ID3.BinarSizeD$))
  48.  
  49.  
  50. PRINT "Binaries (bites) in SizeRecordA Byte: "; Ba$;: PRINT " HEX: ";: PRINT HEX$(ASC(ID3.BinarSizeA$));: PRINT " DEC: "; ID3.BinarSizeA$; " ("; ASC(ID3.BinarSizeA$); ")"
  51. PRINT "Binaries (bites) in SizeRecordB Byte: "; Bb$;: PRINT " HEX: ";: PRINT HEX$(ASC(ID3.BinarSizeB$));: PRINT " DEC: "; ID3.BinarSizeB$; " ("; ASC(ID3.BinarSizeB$); ")" ' It is such a pleasantry
  52. PRINT "Binaries (bites) in SizeRecordC Byte: "; Bc$;: PRINT " HEX: ";: PRINT HEX$(ASC(ID3.BinarSizeC$));: PRINT " DEC: "; ID3.BinarSizeC$; " ("; ASC(ID3.BinarSizeC$); ")"
  53. PRINT "Binaries (bites) in SizeRecordD Byte: "; Bd$;: PRINT " HEX: ";: PRINT HEX$(ASC(ID3.BinarSizeD$));: PRINT " DEC: "; ID3.BinarSizeD$; " ("; ASC(ID3.BinarSizeD$); ")"
  54. headd$ = RIGHT$(Ba$, 7) + RIGHT$(Bb$, 7) + RIGHT$(Bc$, 7) + RIGHT$(Bd$, 7) '            BINARY WAR
  55. h = HEAD(headd$) / 2
  56. PRINT "Head size calculated AS STRING: "; h; " bytes"
  57.  
  58. IF VAL(LEFT$(DECtoBIN$(ID3.BinarFlags), 1)) = 0 THEN PRINT "Synchronisation is not used" ELSE PRINT "Synchronisation is used"
  59. IF VAL(RIGHT$(LEFT$(DECtoBIN$(ID3.BinarFlags), 1), 1)) = 0 THEN PRINT "Extended header for ID3TAG is not used" ELSE PRINT "Extended header for ID3TAG is used - not supported by this program"
  60. IF VAL(RIGHT$(LEFT$(DECtoBIN$(ID3.BinarFlags), 2), 1)) = 1 THEN PRINT "Experimental ID3 TAG!" 'VAL is possile to use because strings contains zero or one
  61.  
  62.  
  63. '----------- cteni framu --------------               frames reading
  64. w = 0 '                                                                    FRAME head is 10 bytes long. 4 byte = name "AENC" or other, 4 byte size, 2 byte flags. Size is calculated as ASC sum of all four
  65. home: '                                                                    bytes.
  66. GET #1, , FRM '                                                             hlava ma 10 bytu: 4 byty jmeno, 4 byty velikost, 2 byty flags. Vse je psano klasicky hexadecimalne, uz zadny nesmysly s bitama.
  67. '                                                                                                   minimalni krok k dalsimu zaznamu ma byt 11 bytu (1byt je minimum pro kazdy identifikator, 10 byt
  68. '                                                                                                  je velikost hlavy. Velikost framu je dana souctem ASC ctyr bytu v hlave Framu.
  69. FrameSize = ASC(FRM.SizeA$) + ASC(FRM.SizeB$) + ASC(FRM.SizeC$) + ASC(FRM.SizeD$)
  70. InFrame$ = SPACE$(FrameSize) 'this statement using is Clippy method, thank Clippy!
  71. GET #1, , InFrame$
  72. q$ = FRM.Id$
  73. SELECT CASE FRM.Id$ '   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - X =  HAVE OWN SECOND HEAD and can be more defined
  74.     CASE "AENC": q$ = "Audio encryption:" '                    X
  75.     CASE "APIC": q$ = "Attached Picture:": APICSUB: GOTO home 'X
  76.     CASE "COMM": q$ = "Comments:" '                            X
  77.     CASE "COMR": q$ = "Commercial frame:" '                    X
  78.     CASE "ENCR": q$ = "Encryption method:" '                   X
  79.     CASE "EQUA": q$ = "Equalization:" '                        X
  80.     CASE "ETCO": q$ = "Event timing codes:" '                  X
  81.     CASE "GEOB": q$ = "General encapsulated object:" '         X
  82.     CASE "GRID": q$ = "Group identification registration:" '   X
  83.     CASE "IPLS": q$ = "Involved people list:"
  84.     CASE "LINK": q$ = "Linked information:" '                  X
  85.     CASE "MCDI": q$ = "Music CD identifier:" '                 X
  86.     CASE "MLLT": q$ = "MPEG location lookup table:" '          X
  87.     CASE "OWNE": q$ = "Ownership frame:" '                     X
  88.     CASE "PRIV": q$ = "Private frame:" '                       X
  89.     CASE "PCNT": q$ = "Play counter:" '                        X
  90.     CASE "POPM": q$ = "Popularimeter:"
  91.     CASE "POSS": q$ = "Position synchronisation frame:" '      X
  92.     CASE "RBUF": q$ = "Recommended buffer size:" '             X
  93.     CASE "RVAD": q$ = "Relative volume adjustment:" '          X
  94.     CASE "RVRB": q$ = "Reverb" '                               X
  95.     CASE "SYLT": q$ = "Synchronized lyric / text:"
  96.     CASE "SYTC": q$ = "Synchronized tempo codes:" '            X
  97.     CASE "TALB": q$ = "Album / Movie / Show title:"
  98.     CASE "TBPM": q$ = "Beats per minute:"
  99.     CASE "TCOM": q$ = "Composer: "
  100.     CASE "TCON": q$ = "Content type:" '                        X - number = style as V1.1
  101.     CASE "TCOP": q$ = "Copyright message:"
  102.     CASE "TDAT": q$ = "Date:" '                                    numeric record always 4 bytes long
  103.     CASE "TDLY": q$ = "Playlist delay:"
  104.     CASE "TENC": q$ = "Encoded by:"
  105.     CASE "TEXT": q$ = "Lyricist / Text Writer"
  106.     CASE "TFLT": q$ = "File type:" '                           X -  1/2/3/2.5(MPGs)/AAC/VQF/PCM
  107.     CASE "TIME": q$ = "Time:" '                                      format HH:MM
  108.     CASE "TIT1": q$ = "Content group description:"
  109.     CASE "TIT2": q$ = "Title / songname:"
  110.     CASE "TIT3": q$ = "Subtitle / Description refinement:"
  111.     CASE "TKEY": q$ = "Initial key:"
  112.     CASE "TLAN": q$ = "Language:"
  113.     CASE "TLEN": q$ = "Length:"
  114.     CASE "TMED": q$ = "Media type" '                           X
  115.     CASE "TOAL": q$ = "Original album / movie / show title:"
  116.     CASE "TOFN": q$ = "Original filename:"
  117.     CASE "TOLY": q$ = "Original lyricist / text writer:"
  118.     CASE "TOPE": q$ = "Original artist / performer:"
  119.     CASE "TORY": q$ = "Original release year:"
  120.     CASE "TOWN": q$ = "File owner / license:"
  121.     CASE "TPE1": q$ = "Lead performer / Soloist:"
  122.     CASE "TPE2": q$ = "Band / orchestra / accompaniment:"
  123.     CASE "TPE3": q$ = "Conductor / performer refinement:"
  124.     CASE "TPE4": q$ = "Modified by:"
  125.     CASE "TPOS": q$ = "Part of a set:"
  126.     CASE "TPUB": q$ = "Publisher:"
  127.     CASE "TRCK": q$ = "Track number / Position in set:"
  128.     CASE "TRDA": q$ = "Recording dates:"
  129.     CASE "TRSN": q$ = "Internet radio station name:"
  130.     CASE "TRSO": q$ = "Internet radio station owner:"
  131.     CASE "TSIZ": q$ = "Size:"
  132.     CASE "TSRC": q$ = "International standard recording code (ISRC):"
  133.     CASE "TSSE": q$ = "Software / Hardware used for encoding:"
  134.     CASE "TYER": q$ = "Year:"
  135.     CASE "TXXX": q$ = "User defined text frame:" '             X - text encoding, description and value
  136.     CASE "UFID": q$ = "Unique file identifier:"
  137.     CASE "USER": q$ = "Terms of use:" '                        X
  138.     CASE "USLT": q$ = "Unsychronized lyric/text transcription:": USLTSUB 'X
  139.     CASE "WCOM": q$ = "Commercial information:"
  140.     CASE "WCOP": q$ = "Copyright / Legal information:"
  141.     CASE "WOAF": q$ = "Official audio file webpage:"
  142.     CASE "WOAR": q$ = "Official artist / performer webpage:"
  143.     CASE "WOAS": q$ = "Official audio source webpage:"
  144.     CASE "WORS": q$ = "Official internet radio webpage:"
  145.     CASE "WPAY": q$ = "Payment:"
  146.     CASE "WPUB": q$ = "Publishers official webpage:"
  147.     CASE "WXXX": q$ = "User defined URL link frame:" '           X
  148. 'GET #1, , InFrame$ ' first muss i analyzing metadata
  149.  
  150. IF FRM.Id$ = "USLT" THEN
  151.     textencod1 = ASC(LEFT$(InFrame$, 1))
  152.     textencod2 = ASC(LEFT$(MID$(InFrame$, 1), 1))
  153.     PRINT textencod1, textencod2
  154.  
  155. IF LEFT$(FRM.Id$, 1) = CHR$(0) OR LEFT$(FRM.Id$, 1) = CHR$(32) OR LEFT$(FRM.Id$, 1) = CHR$(255) OR RIGHT$(FRM.Id$, 1) = "d" THEN GOTO EndRec '+ LTRIM$(CHR$(0)) + LTRIM$(CHR$(0)) + LTRIM$(CHR$(0)) THEN GOTO EndRec
  156. OUTList$(w) = q$ + InFrame$
  157. PRINT "Frame: "; FRM.Id$; " Frame size: "; FrameSize; " and record in this frame: "; InFrame$
  158.  
  159. InFrame$ = ""
  160. readet = readet + 10 + FrameSize
  161. 'PRINT readet, FrameSize
  162. SLEEP: w = w + 1: GOTO home
  163. EndRec:
  164. PRINT "End of file"
  165. PRINT "List: "; IDENTIT$
  166. FOR z = 1 TO w
  167.     PRINT OUTList$(z)
  168. PRINT "Press any key to end"
  169.  
  170. ID3V11: 'upgraded ID3V1.1 program (upgraded in end file access, many better)
  171. OPEN file$ FOR BINARY AS #1 'upgraded file access style
  172. DIM re AS STRING * 256
  173. DIM re0 AS STRING * 128
  174. DIM re2 AS STRING * 128
  175. GET #1, , re0$: IF LEFT$(re0$, 3) <> "ID3" THEN PRINT "ID3 mark not found! - but i try this" ' its opened, also its posibble to have uncorrect outputs.
  176. PRINT "Wait..."
  177. 'UPGRADE ////
  178. e = LOF(1) - 128
  179. IF e < 128 THEN PRINT "Error: LOF returned file length < 128 bytes!"
  180. SEEK #1, e 'better and many faster access             i  am still learning
  181. 'UPGRADE END
  182. GET #1, e - 1, re0$
  183. GET #1, e, re2$
  184. re$ = re0$ + re2$ '                                   After several attempts, I realized that the begin of the last record may not be the very last recording at the end of the file...    :-D
  185. 'PRINT re$, e: SLEEP
  186. FOR scan = 1 TO LEN(re$) '                            Here this loop byte to byte scanned re$ for text "TAG" - its definition for ID3 tag begin
  187.     id$ = LEFT$(MID$(re$, scan), 3)
  188.     IF id$ = "TAG" THEN sca = scan: found = 1 '       byte position in string, "found" is myself method to prevent uncorrect outputs if file have none or ID3 V.2.2 ID3TAG, but its not usefull at 100%
  189. NEXT scan
  190. IF found = 0 THEN PRINT "Uncorrect record or no ID3V1.1 / ID3V2.3": SLEEP 1: SYSTEM
  191. dal:
  192. 'COLOR , 5                                          i have here tested long text arrays
  193. SongName$ = LEFT$(MID$(re$, 3 + sca), 30)
  194. Autor$ = LEFT$(MID$(re$, 33 + sca), 30) '           filtering strings
  195. Album$ = LEFT$(MID$(re$, 63 + sca), 30) '           This way "integer = ASC(LEFT$(MID$(string$, position), long)) is way how read MP3 HEAD. BUT MP3 HEAD contains MANY tables and recordings.
  196. Rok$ = LEFT$(MID$(re$, 93 + sca), 4)
  197. Coment$ = LEFT$(MID$(re$, 97 + sca), 28)
  198. TrackID$ = LEFT$(MID$(re$, 125 + sca), 1)
  199. IF TrackID$ = CHR$(0) THEN track$ = STR$(ASC(LEFT$(MID$(re$, 126 + sca), 1))) ELSE track$ = "Not writed" '  See to https://en.wikipedia.org/wiki/ID3
  200. Gen$ = LEFT$(MID$(re$, 127 + sca), 1)
  201. IF Gen$ = "" THEN GOTO none
  202. genre = ASC(Gen$) + 1
  203. IF genre = 0 OR genre > 125 THEN GOTO none
  204. FOR gnr = 1 TO genre
  205.     READ genre$
  206. NEXT gnr
  207. none:
  208. 'PRINT re$ '  If you delete this mark "'", you see as its writed in file
  209. PRINT IDENTIT$
  210. PRINT "Song name: "; SongName$ '         Song
  211. PRINT "Author: "; Autor$ '               Author name
  212. PRINT "Album: "; Album$ '                Album
  213. PRINT "Year: "; Rok$ '                   Year
  214. PRINT "Comment: "; Coment$ '             Comment
  215. PRINT "Track: "; track$ '                Track number
  216. PRINT "Genre: "; genre$ '                Genre
  217. genre:
  218.  
  219. DATA Blues,Classic Rock,Country,Dance,Disco,Funk,Grunge,Hip-Hop,Jazz,Metal,New Age,Oldies,Other,Pop,R&B,Rap,Reggae,Rock,Techno,Industrial,Alternative,Ska,Death Metal,Pranks,Soundtrack,Eurotechno,Ambient
  220. DATA Trip-Hop,Vocal,Jazz+Funk,Fusion,Trance,Classical,Instrumental,Acid,House,Game,Sound Clip,Gospel,Noise,Alternative Rock,Bass,Soul,Punk,Space,Meditative,Instrumental Pop,Instrumental Rock,Ethnic
  221. DATA Gothic,Darkwave,Techno-Industrial,Electronic,Jungle,Pop-Folk,Eurodance,Dream,Southern Rock,Comedy,Cult,Gangsta,Top 40,Christian Rap,Pop/Funk,Native American,Cabaret,New Wave,Psychadelic,Rave,Show Tunes
  222. DATA Trailer,Lo-Fi,Tribal,Acid Punk,Acid Jazz,Polka,Retro,Musical,Rock & Roll,Hard Rock,Folk,Folk/Rock,National Folk,Swing,Fast-Fusion,Bebop,Latin,Revival,Celtic,Bluegrass,Avantgarde,Gothic Rock,Progressive Rock
  223. DATA Psychedelic Rock,Symphonic Rock,Slow Rock,Big Band,Chorus,Easy Listening,Acoustic,Humour,Speech,Chanson,Opera,Chamber Music,Sonata,Symphony,Booty Bass,Primus,Porn Groove,Satire,Slow Jam,Club,Tango,Samba
  224. DATA Folklore,Ballad,Power Ballad,Rhytmic Soul,Freestyle,Duet,Punk Rock,Drum Solo,Acapella,Euro-House,Dance Hall,Goa,Drum & Bass,Club-House,Hardcore,Terror,Indie,BritPop,Negerpunk,Polsk Punk,Beat,Christian Gangsta Rap
  225. DATA Heavy Metal,Black Metal,Crossover,Contemporary Christian,Christian Rock
  226.  
  227. '                                                                                       HAPPY CODING!
  228.  
  229. SUB USLTSUB
  230. SHARED InFrame$, a
  231. 'FOR a = 1 TO LEN(InFrame$)
  232. 'r$ = LEFT$(MID$(InFrame$, a), 1)
  233. 'PRINT a, r$, ASC(r$)
  234. 'SLEEP
  235. 'NEXT a
  236.  
  237.  
  238.  
  239.  
  240.  
  241.  
  242.  
  243.  
  244. SUB APICSUB
  245. SHARED InFrame$, a, h ' h is ID3TAGV2.3.0 size
  246. pokusnavelikost = h - SEEK(1)
  247.  
  248.  
  249.  
  250. ' FOR a = 1 TO LEN(InFrame$)
  251. ' r$ = LEFT$(MID$(InFrame$, a), 1)
  252. ' PRINT a, r$, ASC(r$)
  253. ' SLEEP
  254. ' NEXT a                                            ' LADENI  (program setup)
  255.  
  256. textencoding = ASC(LEFT$(InFrame$, 1))
  257. IF textencoding = 0 THEN PRINT "Text encoding NO" ELSE PRINT "Text encoding YES" 'can be 0 or 1
  258. mime$ = LTRIM$(RTRIM$(LEFT$(MID$(InFrame$, 2), 9)))
  259. PRINT "MIME TYPE:"; mime$
  260. PRINT "TEXT:"; te$; "DELKA inframe$ je: (primo bez odecteni): "; LEN(InFrame$); "jsi na pozici:"; SEEK(1)
  261. oll = SEEK(1)
  262. ext$ = ".jpg"
  263. 'GOTO pokus
  264.  
  265.     CASE "image/jpe": ext$ = ".jpg": GOTO POKUS  'k.mp3
  266.     CASE "image/jpg": ext$ = ".jpg" 'c++ source
  267.  
  268.  
  269.  
  270.  
  271.  
  272.  
  273.  
  274.  
  275.  
  276.  
  277.  
  278.  
  279.  
  280.  
  281.  
  282.  
  283.  
  284.  
  285.  
  286.  
  287.  
  288. 'HERE is comming a very BIG problem for me: HOW TO FIND PICTURE SIZE? All sources was i can are bad. Also i use one old method - because record in image head is unussable - bytes with size are bad - or
  289. 'WIKI specification is not correct, author ID3 write about this nothing and frame lenght is bad, i muss scanning file for next idetificator after "APIC". This write to array and the one that is closest
  290. 'to me precisely determines the length of the picture. Although this is smut, but i do not know how else to achieve results - if cant right bytes with size.
  291. 'if you need repairing me, so:  image is MIME format and this is NOT JPEG! its other format with the same name.
  292.  
  293. DIM HYPERBLOCK AS STRING * 20
  294. DIM Identity AS STRING * 4
  295. FOR FATALITY = SEEK(1) TO LOF(1)
  296.     GET #1, , HYPERBLOCK$
  297.     RESTORE IDENTIT
  298.     Identity$ = SPACE$(4)
  299.     FOR iScan = 0 TO 73
  300.         READ IDENTIT$
  301.         FOR D4FF = 1 TO LEN(HYPERBLOCK$)
  302.             Identity$ = LEFT$(MID$(HYPERBLOCK$, D4FF), 4)
  303.             IF Identity$ = IDENTIT$ THEN GOTO VOALA '   First correct detection ends this loop and then with PEEK i calculate correct image size
  304.             ' LOCATE 13, 5: PRINT LOF(1), "/", FATALITY, IDENTIT$, iScan, Identity$
  305.         NEXT D4FF
  306.     NEXT iScan
  307. NEXT FATALITY
  308. EXIT SUB 'next record not found...views outputs and ends
  309.  
  310.  
  311. VOALA: new = SEEK(1)
  312. Size = new - oll
  313. 'PRINT "LOOPSIZE", Size, new, oll, IDENTIT$: BEEP: SLEEP
  314. Size = Size + 64
  315.  
  316. REM ///////////////////////////////
  317. POKUS:
  318. Size = pokusnavelikost
  319. REM //////////////////////////////
  320.  
  321.  
  322. SEEK #1, oll - 77
  323. mega$ = SPACE$(Size) 'if here is correct size, its maked right copy of image to disc.
  324. DIM mega(Size) AS LONG
  325. GET #1, , mega$
  326. IF _FILEEXISTS("swap.???") THEN KILL "swap.???"
  327. swap$ = "swap" + LTRIM$(ext$)
  328. PUT #4, , mega$
  329. SEEK #1, Size + 60
  330. 'InFrame$ = ""
  331.  
  332. j& = _LOADIMAGE(swap$, 32)
  333. IF j& < -1 THEN SCREEN j&
  334.  
  335.  
  336. IDENTIT:
  337.  
  338. DATA AENC,APIC,COMM,COMR,ENCR,EQUA,ETCO,GEOB,GRID,IPLS,LINK,MCDI,MLLT,OWNE,PRIV,PCNT,POPM,POSS,RBUF,RVAD,RVRB,SYLT,SYTC,TALB,TBPM,TCOM,TCON,TCOP,TDAT,TDLY,TENC,TEXT,TFLT,TIME,TIT1,TIT2,TIT3
  339. DATA TKEY,TLAN,TLEN,TMED,TOAL,TOFN,TOLY,TOPE,TORY,TOWN,TPE1,TPE2,TPE3,TPE4,TPOS,TPUB,TRCK,TRDA,TRSN,TRSO,TSIZ,TSRC,TSSE,TYER,TXXX,UFID,USER,USLT,WCOM,WCOP,WOAF,WOAR,WOAS,WORS,WPAY,WPUB,WXXX
  340. 'Although it works, but if no other identifiers after APIC file, so it will end up in long loop. So if this view, then, alone with previous tests of that there is a trailing support, the identifier
  341. 'or simply not show. It's stupid, but a master programmer ID3 tag is inherently inconsistent to man. Head ID3 tag fits binary level, and here size does not deliver. That would put a medal.
  342. 'co taky chtit od svedu. Maximalne ten Hulmiho Ukolen by jeste sel...
  343.  
  344.  
  345.  
  346.  
  347.  
  348.  
  349.  
  350.  
  351. FUNCTION HEAD (b AS STRING) 'BIN to DEC vystup je integer, vstup je string (jmeno FUNKCE je promenna s hodnotou z funkce)
  352. FOR Si = 0 TO LEN(b$)
  353.     e$ = LEFT$(MID$(b$, Si + 1), 1)
  354.     c = VAL(e$)
  355.     Sj = LEN(b$) - Si
  356.     DECtoBI2 = DECtoBI2 + (c * 2 ^ Sj)
  357. NEXT Si
  358. PRINT "Si:"; Si
  359. HEAD = DECtoBI2
  360.  
  361.  
  362. '---- FUNCTIONs TEST ----       if is this copyed alone as new program
  363. 'b$ = DECtoBIN$(255)
  364. 'b = BINtoDEC(b$)
  365. 'c$ = HEX$(b)
  366. 'c = HEXtoDEC(c$)
  367.  
  368. 'PRINT b$, b, c$, c
  369.  
  370. FUNCTION DECtoBIN$ (vstup) 'DEC to BIN ok vystup je string, vstup je integer          decimal to binar number convertor   -   FROM QB64WIKI
  371. '   BINARY$ = ""
  372. FOR rj = 7 TO 0 STEP -1
  373.     IF vstup AND 2 ^ rj THEN BINtoDE$ = BINtoDE$ + "1" ELSE BINtoDE$ = BINtoDE$ + "0"
  374. NEXT rj
  375. DECtoBIN$ = BINtoDE$
  376.  
  377. FUNCTION BINtoDEC (b AS STRING) 'BIN to DEC vystup je integer, vstup je string (jmeno FUNKCE je promenna s hodnotou z funkce)
  378. FOR Si = 0 TO 7
  379.     e$ = LEFT$(MID$(b$, Si + 1), 1)
  380.     c = VAL(e$) '                                                                  binar to decimal number convertor
  381.     Sj = 7 - Si
  382.     DECtoBI = DECtoBI + (c * 2 ^ Sj)
  383. NEXT Si
  384. BINtoDEC = DECtoBI
  385.  
  386. FUNCTION HEXtoDEC (h AS STRING) 'Vystup je integer, vstup je string, opacne k funkci HEX$
  387. HEXtoDEC = VAL("&H" + h$) '                                                        hexadecimal to decimal number convertor
  388.  
  389. FUNCTION BINtoHEX$ (bi$)
  390. c = BINtoDEC(bi$) '                                                                 binar to hexadecimal number convertor (use binar to decimal convertor)
  391. BINtoHEX$ = HEX$(c)
  392.  

i think, it is for first and one new id3 version. The first id3 is always writed in end the file, new versions are in begin file.

Offline Petr

  • Forum Resident
  • Posts: 1720
  • The best code is the DNA of the hops.
    • View Profile
Re: Development of a program for automatic search for music and movies
« Reply #76 on: June 27, 2020, 12:28:17 pm »
List directories using Steve version in QB64 folder duration is 1.5 sec here.

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Development of a program for automatic search for music and movies
« Reply #77 on: June 27, 2020, 12:53:44 pm »
What is interesting is that when I don't double check _DIREXISTS(SearchDirectory + "\" + nam$)
my run time is significantly reduced!!! and I still get the 607 files on my system without the false positive.

I cut about a half second off my build and print time and if I can do this without changing directories (which should be quite possible) will have another big time savings.
Good work, @bplus ! When I get home I'll try your new code if you post it and see how fast it runs on my system.
Shuwatch!

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Development of a program for automatic search for music and movies
« Reply #78 on: June 27, 2020, 12:55:43 pm »
List directories using Steve version in QB64 folder duration is 1.5 sec here.
@Petr   Yes, but his code returns false positives and doesn't sort correctly. Also, he places his timer in a spot differently from where I timed mine and bplus's. So far, @bplus has the most efficient code as his sorts exactly right and doesn't display false positives
« Last Edit: June 27, 2020, 01:01:04 pm by SpriggsySpriggs »
Shuwatch!

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: Development of a program for automatic search for music and movies
« Reply #79 on: June 27, 2020, 01:02:15 pm »
I've spent a little time digging into the flags issue.  The problem arises when we're dealing with the subfolders.  Somehow the path isn't working properly with them, as illustrated with the quick demo here:

Code: [Select]
DEFLNG A-Z
DECLARE CUSTOMTYPE LIBRARY ".\direntry"
    FUNCTION load_dir& (s AS STRING)
    FUNCTION has_next_entry& ()
    SUB close_dir ()
    SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
END DECLARE

REDIM SHARED Tree(0) AS STRING

SCREEN _NEWIMAGE(1280, 740, 32)
_DELAY .25
_SCREENMOVE 80, 0
DIM cd$, i, w$

PRINT "Creating tree."
t# = TIMER(0.001)
MakeTree _CWD$ 'testing in QB64 folder
PRINT "Showing Tree"

FOR i = 0 TO UBOUND(Tree) 'show tree
    PRINT Tree(i)
NEXT
PRINT USING "##.##### seconds creating, printing, and sorting tree."; TIMER - t#


SUB GetSubDirs (SearchDirectory AS STRING)
    DIM flags AS LONG, file_size AS LONG, length, nam$
    IF load_dir(SearchDirectory + CHR$(0)) THEN
        DO
            length = has_next_entry
            IF length > -1 THEN
                nam$ = SPACE$(length)
                get_next_entry nam$, flags, file_size
                IF nam$ <> "." AND nam$ <> ".." THEN
                    t$ = SearchDirectory + "\" + nam$
                    IF flags AND 1 THEN
                        u = UBOUND(Tree) + 1
                        REDIM _PRESERVE Tree(u)
                        Tree(u) = t$
                    END IF
                END IF
            END IF
        LOOP UNTIL length = -1
    ELSE
        PRINT "Dir not loaded"
    END IF
    close_dir
END SUB

SUB MakeTree (Dir$)
    DIM OnDir AS LONG, gap AS LONG, i AS LONG, swapped AS LONG
    REDIM Tree(0) AS STRING
    Tree(0) = Dir$
    DO
        GetSubDirs Tree(OnDir)
        OnDir = OnDir + 1
    LOOP UNTIL OnDir > UBOUND(Tree)
    gap = UBOUND(Tree)
    DO
        gap = 10 * gap \ 13
        IF gap < 1 THEN gap = 1
        i = 0
        swapped = 0
        DO
            IF Tree(i) > Tree(i + gap) THEN
                SWAP Tree(i), Tree(i + gap)
                swapped = -1
            END IF
            i = i + 1
        LOOP UNTIL i + gap > UBOUND(Tree)
    LOOP UNTIL gap = 1 AND swapped = 0
END SUB

This lists the root directories of our folder and flags them properly, but then it fails to properly identify any of the subfolders.  Some digging into the C source will be required to sort out what the heck is going on with the false flag results, and with my current schedule, I can't promise how long that might take me.  For now, I'd go the route Sprigg and bplus suggested:  Remove the IS_DIR flag from consideration completely and just let QB64 use _DIREXISTS and _FILEEXISTS to verify if it's a file or directory.
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: Development of a program for automatic search for music and movies
« Reply #80 on: June 27, 2020, 01:17:14 pm »
@Petr   Yes, but his code returns false positives and doesn't sort correctly. Also, he places his timer in a spot differently from where I timed mine and bplus's. So far, @bplus has the most efficient code as his sorts exactly right and doesn't display false positives

The single false positive can be removed just by taking out the flags AND IS_DIR check.  That's something which POSIX is tossing us, and then it's only tossing the result on some systems.  I imagine debugging it is going to be a PITA.

I'd say as for sorting, that's just a matter of preference and how you decide to order your data.  You can sort by file name, extension, size -- whatever you like.  The only difference is bplus's routine runs recursively -- which expands folders and subfolders as it finds them -- and mine runs a whole level at once before expanding the next level of subfolders.

As for the placement of the timer, you only want to time how quickly you find and build your data tree.  Counting time for printing and stuff is a waste of time.  One could toss a _DISPLAY in there and completely change the result.  If one was using Inform and printing to a form's list box, it'd have a different time.  Printing to console vs a graphical window would give different times.  What you want to know is, "how long does it take to build this data tree" -- not, "How long does everything in my program take from start to finish". 

Heck, on my system, since I'm printing to a RAMdrive, it's faster for me to dump the results to a file than it is to print them and scroll the screen with them...   Time and optimize your tree creation routine first; then worry about timing and optimizing the rest of your program after.  :)
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Development of a program for automatic search for music and movies
« Reply #81 on: June 27, 2020, 02:13:04 pm »
I've spent a little time digging into the flags issue.  The problem arises when we're dealing with the subfolders.  Somehow the path isn't working properly with them, as illustrated with the quick demo here:

Code: [Select]
DEFLNG A-Z
DECLARE CUSTOMTYPE LIBRARY ".\direntry"
    FUNCTION load_dir& (s AS STRING)
    FUNCTION has_next_entry& ()
    SUB close_dir ()
    SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
END DECLARE

REDIM SHARED Tree(0) AS STRING

SCREEN _NEWIMAGE(1280, 740, 32)
_DELAY .25
_SCREENMOVE 80, 0
DIM cd$, i, w$

PRINT "Creating tree."
t# = TIMER(0.001)
MakeTree _CWD$ 'testing in QB64 folder
PRINT "Showing Tree"

FOR i = 0 TO UBOUND(Tree) 'show tree
    PRINT Tree(i)
NEXT
PRINT USING "##.##### seconds creating, printing, and sorting tree."; TIMER - t#


SUB GetSubDirs (SearchDirectory AS STRING)
    DIM flags AS LONG, file_size AS LONG, length, nam$
    IF load_dir(SearchDirectory + CHR$(0)) THEN
        DO
            length = has_next_entry
            IF length > -1 THEN
                nam$ = SPACE$(length)
                get_next_entry nam$, flags, file_size
                IF nam$ <> "." AND nam$ <> ".." THEN
                    t$ = SearchDirectory + "\" + nam$
                    IF flags AND 1 THEN
                        u = UBOUND(Tree) + 1
                        REDIM _PRESERVE Tree(u)
                        Tree(u) = t$
                    END IF
                END IF
            END IF
        LOOP UNTIL length = -1
    ELSE
        PRINT "Dir not loaded"
    END IF
    close_dir
END SUB

SUB MakeTree (Dir$)
    DIM OnDir AS LONG, gap AS LONG, i AS LONG, swapped AS LONG
    REDIM Tree(0) AS STRING
    Tree(0) = Dir$
    DO
        GetSubDirs Tree(OnDir)
        OnDir = OnDir + 1
    LOOP UNTIL OnDir > UBOUND(Tree)
    gap = UBOUND(Tree)
    DO
        gap = 10 * gap \ 13
        IF gap < 1 THEN gap = 1
        i = 0
        swapped = 0
        DO
            IF Tree(i) > Tree(i + gap) THEN
                SWAP Tree(i), Tree(i + gap)
                swapped = -1
            END IF
            i = i + 1
        LOOP UNTIL i + gap > UBOUND(Tree)
    LOOP UNTIL gap = 1 AND swapped = 0
END SUB

This lists the root directories of our folder and flags them properly, but then it fails to properly identify any of the subfolders.  Some digging into the C source will be required to sort out what the heck is going on with the false flag results, and with my current schedule, I can't promise how long that might take me.  For now, I'd go the route Sprigg and bplus suggested:  Remove the IS_DIR flag from consideration completely and just let QB64 use _DIREXISTS and _FILEEXISTS to verify if it's a file or directory.

This explains why my recursive function falls flat when I don't change directory before each call to GetSubDirs. I am beginning to think startDirectory is meaningless in part of the call, it's just flagging the directories directly under the current directory, which is why my recursive only works with changing directories and reading flags.

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Development of a program for automatic search for music and movies
« Reply #82 on: June 27, 2020, 03:10:51 pm »
Here is Tree building by CHDIR and then checking if Directory with Flags which do work in immediate sub dir of current directory, this is fastest time:
Code: QB64: [Select]
  1. 'Tree builder recursive.bas  b+ Petr 2020-06-26  now with SMCNeill Fix!! add CHR$(0) to C string
  2. DEFLNG A-Z
  3.     FUNCTION load_dir& (s AS STRING)
  4.     FUNCTION has_next_entry& ()
  5.     SUB close_dir ()
  6.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
  7.  
  8. REDIM SHARED Tree(0) AS STRING '<<<<<<<<<<<<<<<<<<<<<< G(0) will remain empty
  9. t# = TIMER(0.001)
  10.  
  11.  
  12. MakeTree _CWD$
  13. FOR i = 1 TO UBOUND(Tree) 'show tree
  14.     PRINT _TRIM$(STR$(i)); ": "; Tree(i)
  15. PRINT USING "##.##### seconds creating, printing."; TIMER - t#
  16.  
  17. SUB sAppend (arr() AS STRING, addItem$)
  18.     REDIM _PRESERVE arr(LBOUND(arr) TO UBOUND(arr) + 1) AS STRING
  19.     arr(UBOUND(arr)) = addItem$
  20.  
  21. SUB GetSubDirs (SearchDirectory AS STRING, DirList() AS STRING)
  22.     CONST IS_DIR = 1
  23.     DIM flags AS LONG, file_size AS LONG, length, nam$
  24.     IF load_dir(SearchDirectory + CHR$(0)) THEN 'Steve's fix here with CHR$(0) for C call
  25.         DO
  26.             length = has_next_entry
  27.             IF length > -1 THEN
  28.                 nam$ = SPACE$(length)
  29.                 get_next_entry nam$, flags, file_size
  30.                 IF (flags = IS_DIR) THEN
  31.                     IF nam$ <> "." AND nam$ <> ".." THEN
  32.                         DirCount = DirCount + 1
  33.                         IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
  34.                         DirList(DirCount) = SearchDirectory + "\" + nam$
  35.                     END IF
  36.                 END IF
  37.             END IF
  38.         LOOP UNTIL length = -1
  39.     END IF
  40.     close_dir 'Steve first fix that got navigator working
  41.     REDIM _PRESERVE DirList(DirCount)
  42.  
  43. SUB MakeTree (startDir AS STRING)
  44.     copyStart$ = startDir
  45.     REDIM D(100) AS STRING
  46.     GetSubDirs copyStart$, D()
  47.     FOR i = 1 TO UBOUND(D)
  48.         sAppend Tree(), D(i)
  49.         CHDIR D(i)
  50.         MakeTree D(i)
  51.     NEXT
  52.  
  53.  
  54.  

Recursion will work without changing directories but you have to check 
Code: QB64: [Select]
  1. IF _DIREXISTS(SearchDirectory + "\" + nam$) THEN
which turns out takes longer than just changing directories (~1.69 sec to ~2.93)!

Code: QB64: [Select]
  1. 'Tree builder recursive.bas  b+ Petr 2020-06-26  now with SMCNeill Fix!! add CHR$(0) to C string
  2. DEFLNG A-Z
  3.     FUNCTION load_dir& (s AS STRING)
  4.     FUNCTION has_next_entry& ()
  5.     SUB close_dir ()
  6.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
  7.  
  8. REDIM SHARED Tree(0) AS STRING '<<<<<<<<<<<<<<<<<<<<<< G(0) will remain empty
  9. t# = TIMER(0.001)
  10.  
  11.  
  12. MakeTree _CWD$
  13. FOR i = 1 TO UBOUND(Tree) 'show tree
  14.     PRINT _TRIM$(STR$(i)); ": "; Tree(i)
  15. PRINT USING "##.##### seconds creating, printing."; TIMER - t#
  16.  
  17. SUB sAppend (arr() AS STRING, addItem$)
  18.     REDIM _PRESERVE arr(LBOUND(arr) TO UBOUND(arr) + 1) AS STRING
  19.     arr(UBOUND(arr)) = addItem$
  20.  
  21. SUB GetSubDirs (SearchDirectory AS STRING, DirList() AS STRING)
  22.     CONST IS_DIR = 1
  23.     DIM flags AS LONG, file_size AS LONG, length, nam$
  24.     IF load_dir(SearchDirectory + CHR$(0)) THEN 'Steve's fix here with CHR$(0) for C call
  25.         DO
  26.             length = has_next_entry
  27.             IF length > -1 THEN
  28.                 nam$ = SPACE$(length)
  29.                 get_next_entry nam$, flags, file_size
  30.                 IF nam$ <> "." AND nam$ <> ".." THEN
  31.                     IF _DIREXISTS(SearchDirectory + "\" + nam$) THEN
  32.                         DirCount = DirCount + 1
  33.                         IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
  34.                         DirList(DirCount) = SearchDirectory + "\" + nam$
  35.                     END IF
  36.                 END IF
  37.             END IF
  38.         LOOP UNTIL length = -1
  39.     END IF
  40.     close_dir 'Steve first fix that got navigator working
  41.     REDIM _PRESERVE DirList(DirCount)
  42.  
  43. SUB MakeTree (startDir AS STRING)
  44.     copyStart$ = startDir
  45.     REDIM D(100) AS STRING
  46.     GetSubDirs copyStart$, D()
  47.     FOR i = 1 TO UBOUND(D)
  48.         sAppend Tree(), D(i)
  49.         MakeTree D(i)
  50.     NEXT
  51.  
  52.  
  53.  


Now if you want all files and directories (I get 19279 items in ~15.59 secs) without checking anything do this:
Code: QB64: [Select]
  1. 'Tree builder recursive.bas  b+ Petr 2020-06-26  now with SMCNeill Fix!! add CHR$(0) to C string
  2. DEFLNG A-Z
  3.     FUNCTION load_dir& (s AS STRING)
  4.     FUNCTION has_next_entry& ()
  5.     SUB close_dir ()
  6.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
  7.  
  8. REDIM SHARED Tree(0) AS STRING '<<<<<<<<<<<<<<<<<<<<<< G(0) will remain empty
  9. t# = TIMER(0.001)
  10.  
  11.  
  12. MakeTree _CWD$
  13. FOR i = 1 TO UBOUND(Tree) 'show tree
  14.     PRINT _TRIM$(STR$(i)); ": "; Tree(i)
  15. PRINT USING "##.##### seconds creating, printing."; TIMER - t#
  16.  
  17. SUB sAppend (arr() AS STRING, addItem$)
  18.     REDIM _PRESERVE arr(LBOUND(arr) TO UBOUND(arr) + 1) AS STRING
  19.     arr(UBOUND(arr)) = addItem$
  20.  
  21. SUB GetSubDirs (SearchDirectory AS STRING, DirList() AS STRING)
  22.     CONST IS_DIR = 1
  23.     DIM flags AS LONG, file_size AS LONG, length, nam$
  24.     IF load_dir(SearchDirectory + CHR$(0)) THEN 'Steve's fix here with CHR$(0) for C call
  25.         DO
  26.             length = has_next_entry
  27.             IF length > -1 THEN
  28.                 nam$ = SPACE$(length)
  29.                 get_next_entry nam$, flags, file_size
  30.                 IF nam$ <> "." AND nam$ <> ".." THEN
  31.                     'IF _DIREXISTS(SearchDirectory + "\" + nam$) THEN
  32.                     DirCount = DirCount + 1
  33.                     IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
  34.                     DirList(DirCount) = SearchDirectory + "\" + nam$
  35.                     'END IF
  36.                 END IF
  37.             END IF
  38.         LOOP UNTIL length = -1
  39.     END IF
  40.     close_dir 'Steve first fix that got navigator working
  41.     REDIM _PRESERVE DirList(DirCount)
  42.  
  43. SUB MakeTree (startDir AS STRING)
  44.     copyStart$ = startDir
  45.     REDIM D(100) AS STRING
  46.     GetSubDirs copyStart$, D()
  47.     FOR i = 1 TO UBOUND(D)
  48.         sAppend Tree(), D(i)
  49.         MakeTree D(i)
  50.     NEXT
  51.  
  52.  

It's probably as fast as you can get through QB64.
« Last Edit: June 27, 2020, 03:21:59 pm by bplus »

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Development of a program for automatic search for music and movies
« Reply #83 on: June 27, 2020, 04:04:23 pm »
As for the placement of the timer, you only want to time how quickly you find and build your data tree.  Counting time for printing and stuff is a waste of time.  One could toss a _DISPLAY in there and completely change the result.  If one was using Inform and printing to a form's list box, it'd have a different time.  Printing to console vs a graphical window would give different times.  What you want to know is, "how long does it take to build this data tree" -- not, "How long does everything in my program take from start to finish". 
In that case, for the speed, just checking the amount of time it takes to use @bplus code to build the array it takes 0.8 seconds on my machine and my code only takes 1.29 seconds to build an array that is already presorted. bplus's code seems to be the absolute best. I have no way of beating that with PowerShell calls. Less than one second to build an array containing 659 directories is great. I removed the other 600+ directories. They were backups from my synced folder.
Shuwatch!

Offline SMcNeill

  • QB64 Developer
  • Forum Resident
  • Posts: 3972
    • View Profile
    • Steve’s QB64 Archive Forum
Re: Development of a program for automatic search for music and movies
« Reply #84 on: June 27, 2020, 04:12:45 pm »
In that case, for the speed, just checking the amount of time it takes to use @bplus code to build the array it takes 0.8 seconds on my machine and my code only takes 1.29 seconds to build an array that is already presorted. bplus's code seems to be the absolute best. I have no way of beating that with PowerShell calls. Less than one second to build an array containing 659 directories is great. I removed the other 600+ directories. They were backups from my synced folder.

Aye, and it's cross-platform compatible.  Powershell calls only work in Windows.  ;)
https://github.com/SteveMcNeill/Steve64 — A github collection of all things Steve!

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Development of a program for automatic search for music and movies
« Reply #85 on: June 27, 2020, 04:15:08 pm »
Aye, and it's cross-platform compatible.  Powershell calls only work in Windows.  ;)
@SMcNeill
PowerShell has a cross platform version :) And I know it is available for both Mac and Linux. Found here: https://aka.ms/pscore6
PowerShell advertises it every time I launch it.
« Last Edit: June 27, 2020, 04:43:11 pm by SpriggsySpriggs »
Shuwatch!

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Development of a program for automatic search for music and movies
« Reply #86 on: June 27, 2020, 04:33:20 pm »
@bplus @SMcNeill There seems to be an issue:
I ran your (bplus) code on my "C:\Program Files (x86)\Steam\steamapps\common" folder and returned nothing. Is there some issue with read/write access, perhaps? I gave "Everyone" read/write access and it still failed to list any directories. Hmmm. I get 5,557 directories added to a presorted array in about 3.75 seconds through PowerShell. And running Steve's code, it took over 9 seconds to create and sort an array using strictly his code copy and pasted from the forum on his latest post. It had no trouble going through the directories but listed about 100 extra files as folders due to the false positive. So the false positive problem is much more than just "the one file". It's going to be happening in many places as a lot of programs create files without extensions and those appear to be the ones getting flagged as directories. Not sure why bplus's code fails completely.
« Last Edit: June 27, 2020, 04:54:29 pm by SpriggsySpriggs »
Shuwatch!

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Development of a program for automatic search for music and movies
« Reply #87 on: June 27, 2020, 04:48:46 pm »
@bplus There seems to be an issue:
I ran your code on my "C:\Program Files (x86)\Steam\steamapps\common" folder and returned nothing. Is there some issue with read/write access, perhaps? I gave "Everyone" read/write access and it still failed to list any directories. Hmmm. I get 5,557 directories listed in about 3.75 seconds through PowerShell.

You have to change directory every step to the folder in question because sub-dir Flag only works when you are directly above that.

Here is my Tiny Navigator that steps you through your hard drive hopefully C:\\ is root.
You can see if you can navigate to that folder, sorry it is constant CHDIR if want the fastest Tree code.
Code: QB64: [Select]
  1. ' B+ started 2019-08-22  restart 2020-06-24
  2. ' 2019-08-22 orig post at https://www.qb64.org/forum/index.php?topic=1646.msg108682#msg108682
  3. ' 2019-08-23_13-25 try fix (to nav all directories) with one place for tmpFile, theory can't write files in some dir's
  4. ' For some reason Windows won't write to a fully pathed file in my user folder???from a SHELL command
  5. ' Try testing if dir exists "C:\temp" exists and making one if not, yes! and I can write my temp files there
  6. ' and now I can chDir anywhere!!!!
  7. ' 2019-08-23_14-25 Take Steve's Advice to use users tmp directory for temp files
  8. ' 2019-08-23_23+ Have files window working for file selection
  9.  
  10. ' 2020-06-24 restart using DirEntry.h, wow looks like Steve's fix helped ALLOT! Thanks SMcNeill
  11. ' I am hitting glitches where Windows wont let me access I think. So I am offering a c enter option
  12. ' to restart at the root but this might not work cross platform.
  13.  
  14. ' 2020-06-26 another fix to DirEntry.h usage add CHR$(0) to string when calling a C program. This fixed a snag
  15. ' between marka and next folder up when heading to root with .. calls.
  16.  
  17.  
  18. ' direntry.h needs to be in QB64 folder
  19.     FUNCTION load_dir& (s AS STRING)
  20.     FUNCTION has_next_entry& ()
  21.     SUB close_dir ()
  22.     SUB get_next_entry (s AS STRING, flags AS LONG, file_size AS LONG)
  23.  
  24.  
  25. _TITLE "Tiny Navigator using DirEntry:     press f to Start a Select a File list of current directory"
  26. SCREEN _NEWIMAGE(1000, 600, 32)
  27. _SCREENMOVE 100, 50
  28.  
  29. DIM SHARED selectedFile AS STRING
  30. REDIM SHARED DIRs(0) AS STRING, FILs(0) AS STRING
  31. DIM mySelection&, done$, i, t$
  32.     COLOR _RGB32(180, 180, 255)
  33.     CLS
  34.  
  35.     t$ = "Current Directory: " + _CWD$
  36.     LOCATE 2, (_WIDTH / 8 - LEN(t$)) / 2: PRINT t$
  37.     REDIM DIRs(0) AS STRING, FILs(0) AS STRING
  38.     GetLists _CWD$, DIRs(), FILs()
  39.     FOR i = 0 TO UBOUND(FILs) ' this just offers a sample listing of files
  40.         IF i < 30 THEN LOCATE i + 4, 60: PRINT FILs(i) ELSE EXIT FOR
  41.     NEXT
  42.     mySelection& = getArrayItemNumber&(5, 5, 50, 30, DIRs())
  43.     CLS
  44.     IF selectedFile <> "" THEN
  45.         PRINT "You selected a file: "; selectedFile
  46.         INPUT "Press enter to continue navigator, any + enter to quit... "; done$
  47.         selectedFile = ""
  48.     ELSEIF mySelection& <> -1719 THEN
  49.         IF _TRIM$(DIRs(mySelection&)) <> "" THEN
  50.             CHDIR DIRs(mySelection&)
  51.         END IF
  52.     ELSE
  53.         PRINT "Nothing selected."
  54.         PRINT "There are some places your OS might not allow us to go."
  55.         PRINT "If you get stuck with 0 slections, press c (for root C:\\) enter..."
  56.         INPUT "Press enter to continue navigator, any + enter to quit... "; done$
  57.         IF done$ = "c" THEN CHDIR ("C:\\"): done$ = ""
  58.     END IF
  59.     _LIMIT 60
  60. LOOP UNTIL done$ <> ""
  61.  
  62. SUB GetLists (SearchDirectory AS STRING, DirList() AS STRING, FileList() AS STRING)
  63.  
  64.     CONST IS_DIR = 1
  65.     CONST IS_FILE = 2
  66.     DIM flags AS LONG, file_size AS LONG, DirCount AS INTEGER, FileCount AS INTEGER, length AS LONG
  67.     DIM nam$
  68.     REDIM _PRESERVE DirList(100), FileList(100)
  69.     DirCount = 0: FileCount = 0
  70.  
  71.     IF load_dir(SearchDirectory + CHR$(0)) THEN
  72.         DO
  73.             length = has_next_entry
  74.             IF length > -1 THEN
  75.                 nam$ = SPACE$(length)
  76.                 get_next_entry nam$, flags, file_size
  77.                 IF (flags AND IS_DIR) THEN
  78.                     DirCount = DirCount + 1
  79.                     IF DirCount > UBOUND(DirList) THEN REDIM _PRESERVE DirList(UBOUND(DirList) + 100)
  80.                     DirList(DirCount) = nam$
  81.                 ELSEIF (flags AND IS_FILE) THEN
  82.                     FileCount = FileCount + 1
  83.                     IF FileCount > UBOUND(filelist) THEN REDIM _PRESERVE FileList(UBOUND(filelist) + 100)
  84.                     FileList(FileCount) = nam$
  85.                 END IF
  86.             END IF
  87.         LOOP UNTIL length = -1
  88.         'close_dir 'move to after end if  might correct the multi calls problem
  89.     ELSE
  90.     END IF
  91.     close_dir 'this  might correct the multi calls problem
  92.  
  93.     REDIM _PRESERVE DirList(DirCount)
  94.     REDIM _PRESERVE FileList(FileCount)
  95.  
  96. FUNCTION rightOf$ (source$, of$)
  97.     IF INSTR(source$, of$) > 0 THEN rightOf$ = MID$(source$, INSTR(source$, of$) + LEN(of$))
  98.  
  99. ' "Escape or Red X box returns -1719 to allow a Cancel function and signal no slection."
  100. FUNCTION getArrayItemNumber& (locateRow, locateColumn, boxWidth, boxHeight, arr() AS STRING)
  101.     'Notes: locateRow, locateColumn for top right corner of selection box on screen in characters for LOCATE.
  102.     'boxWidth and boxHeight are in character units, again for locate and print at correct places.
  103.     'All displaying is restricted to inside the box, which has PgUP and PgDn as top and bottom lines in the display.
  104.  
  105.     DIM curRow AS INTEGER, curCol AS INTEGER, fg AS _UNSIGNED LONG, bg AS _UNSIGNED LONG
  106.     DIM maxWidth AS INTEGER, maxHeight AS INTEGER, page AS INTEGER, hlite AS INTEGER, mx AS INTEGER, my AS INTEGER
  107.     DIM lastMX AS INTEGER, lastMY AS INTEGER, row AS INTEGER, mb AS INTEGER
  108.     DIM lba AS LONG, uba AS LONG, choice AS LONG, kh AS LONG, index AS LONG
  109.     DIM clrStr AS STRING, b AS STRING, selNum&
  110.  
  111.     'save old settings to restore at end ofsub
  112.     curRow = CSRLIN
  113.     curCol = POS(0)
  114.     fg = _DEFAULTCOLOR
  115.     bg = _BACKGROUNDCOLOR
  116.     _KEYCLEAR
  117.  
  118.     maxWidth = boxWidth '       number of characters in box
  119.     maxHeight = boxHeight - 2 ' number of lines displayed of array at one time = 1 page
  120.     lba = LBOUND(arr)
  121.     uba = UBOUND(arr)
  122.     page = 0
  123.     hlite = 0 '                 line in display ready for selection by spacebar or if no number is started, enter
  124.     clrStr$ = SPACE$(maxWidth) 'clearing a display line
  125.  
  126.     GOSUB update '              show the beginning of the array items for selection
  127.  
  128.     'signal cancel selection process, exit sub with this unlikely index to signal canel
  129.     choice = -1719 'primes 7 and 8, not likely to be a select index of an array
  130.  
  131.     DO 'until get a selection or demand exit
  132.  
  133.         'handle the key stuff
  134.         kh& = _KEYHIT
  135.         IF kh& THEN
  136.             IF kh& > 0 AND kh& < 255 THEN
  137.                 IF INSTR("0123456789", CHR$(kh&)) > 0 THEN b$ = b$ + CHR$(kh&): GOSUB update
  138.                 IF CHR$(kh&) = "f" THEN
  139.                     'REDIM FILs(0) AS STRING     'hopefully this is already ready
  140.                     'loadFiles FILs()
  141.                     selNum& = getArrayItemNumber&(5, 60, 60, 30, FILs())
  142.                     COLOR _RGB32(180, 180, 255)
  143.                     CLS 'need to signal out of file selection
  144.                     IF selNum& >= LBOUND(FILs) AND selNum& <= UBOUND(FILs) THEN selectedFile = FILs(selNum&)
  145.                     EXIT DO
  146.                     'back to directory select
  147.                 END IF
  148.  
  149.                 IF CHR$(kh&) = "c" THEN b$ = "": GOSUB update
  150.                 IF kh& = 13 THEN 'enter pressed check if number is being entered?
  151.                     IF LEN(b$) THEN
  152.                         IF VAL(b$) >= lba AND VAL(b$) <= uba THEN 'we have number started
  153.                             choice = VAL(b$): EXIT DO
  154.                         ELSE 'clear b$ to show some response to enter
  155.                             b$ = "": GOSUB update 'clear the value that doesn't work
  156.                         END IF
  157.                     ELSE
  158.                         choice = hlite + page * maxHeight + lba 'must mean to select the highlighted item
  159.                     END IF
  160.                 END IF
  161.                 IF kh& = 27 THEN EXIT DO 'escape clause offered to Cancel selection process
  162.                 IF kh& = 32 THEN choice = hlite + page * maxHeight + lba 'best way to choose highlighted selection
  163.                 IF kh& = 8 THEN 'backspace to edit number
  164.                     IF LEN(b$) THEN b$ = LEFT$(b$, LEN(b$) - 1): GOSUB update
  165.                 END IF
  166.             ELSE
  167.                 SELECT CASE kh& 'choosing sections of array to display and highlighted item
  168.                     CASE 20736 'pg dn
  169.                         IF (page + 1) * maxHeight + lba <= uba THEN page = page + 1: GOSUB update
  170.                     CASE 18688 'pg up
  171.                         IF (page - 1) * maxHeight + lba >= lba THEN page = page - 1: GOSUB update
  172.                     CASE 18432 'up
  173.                         IF hlite - 1 < 0 THEN
  174.                             IF page > 0 THEN
  175.                                 page = page - 1: hlite = maxHeight - 1: GOSUB update
  176.                             END IF
  177.                         ELSE
  178.                             hlite = hlite - 1: GOSUB update
  179.                         END IF
  180.                     CASE 20480 'down
  181.                         IF (hlite + 1) + page * maxHeight + lba <= uba THEN 'ok to move up
  182.                             IF hlite + 1 > maxHeight - 1 THEN
  183.                                 page = page + 1: hlite = 0: GOSUB update
  184.                             ELSE
  185.                                 hlite = hlite + 1: GOSUB update
  186.                             END IF
  187.                         END IF
  188.                     CASE 18176 'home
  189.                         page = 0: hlite = 0: GOSUB update
  190.                     CASE 20224 ' end
  191.                         page = INT((uba - lba) / maxHeight): hlite = maxHeight - 1: GOSUB update
  192.                 END SELECT
  193.             END IF
  194.         END IF
  195.  
  196.         'handle the mouse stuff
  197.         WHILE _MOUSEINPUT
  198.             IF _MOUSEWHEEL = -1 THEN 'up?
  199.                 IF hlite - 1 < 0 THEN
  200.                     IF page > 0 THEN
  201.                         page = page - 1: hlite = maxHeight - 1: GOSUB update
  202.                     END IF
  203.                 ELSE
  204.                     hlite = hlite - 1: GOSUB update
  205.                 END IF
  206.             ELSEIF _MOUSEWHEEL = 1 THEN 'down?
  207.                 IF (hlite + 1) + page * maxHeight + lba <= uba THEN 'ok to move up
  208.                     IF hlite + 1 > maxHeight - 1 THEN
  209.                         page = page + 1: hlite = 0: GOSUB update
  210.                     ELSE
  211.                         hlite = hlite + 1: GOSUB update
  212.                     END IF
  213.                 END IF
  214.             END IF
  215.         WEND
  216.         mx = INT((_MOUSEX - locateColumn * 8) / 8) + 2: my = INT((_MOUSEY - locateRow * 16) / 16) + 2
  217.         IF _MOUSEBUTTON(1) THEN 'click contols or select array item
  218.             'clear mouse clicks
  219.             mb = _MOUSEBUTTON(1)
  220.             IF mb THEN 'clear it
  221.                 WHILE mb 'OK!
  222.                     IF _MOUSEINPUT THEN mb = _MOUSEBUTTON(1)
  223.                     _LIMIT 100
  224.                 WEND
  225.             END IF
  226.  
  227.             IF mx >= 1 AND mx <= maxWidth AND my >= 1 AND my <= maxHeight THEN
  228.                 choice = my + page * maxHeight + lba - 1 'select item clicked
  229.             ELSEIF mx >= 1 AND mx <= maxWidth AND my = 0 THEN 'page up or exit
  230.                 IF my = 0 AND (mx <= maxWidth AND mx >= maxWidth - 2) THEN 'exit sign
  231.                     EXIT DO 'escape plan for mouse click top right corner of display box
  232.                 ELSE 'PgUp bar clicked
  233.                     IF (page - 1) * maxHeight + lba >= lba THEN page = page - 1: GOSUB update
  234.                 END IF
  235.             ELSEIF mx >= 1 AND mx <= maxWidth AND my = maxHeight + 1 THEN 'page down bar clicked
  236.                 IF (page + 1) * maxHeight + lba <= uba THEN page = page + 1: GOSUB update
  237.             END IF
  238.         ELSE '   mouse over highlighting, only if mouse has moved!
  239.             IF mx >= 1 AND mx <= maxWidth AND my >= 1 AND my <= maxHeight THEN
  240.                 IF mx <> lastMX OR my <> lastMY THEN
  241.                     IF my - 1 <> hlite AND (my - 1 + page * maxHeight + lba <= uba) THEN
  242.                         hlite = my - 1
  243.                         lastMX = mx: lastMY = my
  244.                         GOSUB update
  245.                     END IF
  246.                 END IF
  247.             END IF
  248.         END IF
  249.         _LIMIT 200
  250.     LOOP UNTIL choice >= lba AND choice <= uba
  251.     getArrayItemNumber& = choice
  252.     COLOR fg, bg
  253.     'clear key presses
  254.     _KEYCLEAR
  255.     LOCATE curRow, curCol
  256.     'clear mouse clicks
  257.     mb = _MOUSEBUTTON(1)
  258.     IF mb THEN 'clear it
  259.         WHILE mb 'OK!
  260.             IF _MOUSEINPUT THEN mb = _MOUSEBUTTON(1)
  261.             _LIMIT 100
  262.         WEND
  263.     END IF
  264.     EXIT SUB
  265.  
  266.  
  267.     update: '--------------- display of array sections and controls on screen
  268.  
  269.     'fix hlite if it has dropped below last array item
  270.     WHILE hlite + page * maxHeight + lba > uba
  271.         hlite = hlite - 1
  272.     WEND
  273.  
  274.     'main display of array items at page * maxHeight (lines high)
  275.     FOR row = 0 TO maxHeight - 1
  276.         IF hlite = row THEN COLOR _RGB(200, 200, 255), _RGB32(0, 0, 88) ELSE COLOR _RGB32(0, 0, 88), _RGB(200, 200, 255)
  277.         LOCATE locateRow + row, locateColumn: PRINT clrStr$
  278.         index = row + page * maxHeight + lba
  279.         IF index >= lba AND index <= uba THEN
  280.             LOCATE locateRow + row, locateColumn
  281.             PRINT LEFT$(LTRIM$(STR$(index)) + ") " + arr(index), maxWidth)
  282.         END IF
  283.     NEXT
  284.  
  285.     'make page up and down bars to click, print PgUp / PgDn if available
  286.     COLOR _RGB32(200, 200, 255), _RGB32(0, 100, 50)
  287.     LOCATE locateRow - 1, locateColumn: PRINT SPACE$(maxWidth)
  288.     IF page <> 0 THEN LOCATE locateRow - 1, locateColumn: PRINT LEFT$(" Pg Up" + SPACE$(maxWidth), maxWidth)
  289.     LOCATE locateRow + maxHeight, locateColumn: PRINT SPACE$(maxWidth)
  290.     IF page <> INT(uba / maxHeight) THEN
  291.         LOCATE locateRow + maxHeight, locateColumn: PRINT LEFT$(" Pg Dn" + SPACE$(maxWidth), maxWidth)
  292.     END IF
  293.     'make exit sign for mouse click
  294.     COLOR _RGB32(255, 255, 255), _RGB32(200, 100, 0)
  295.     LOCATE locateRow - 1, locateColumn + maxWidth - 3
  296.     PRINT " X "
  297.  
  298.     'if a number selection has been started show it's build = b$
  299.     IF LEN(b$) THEN
  300.         COLOR _RGB(255, 255, 0), _RGB32(0, 0, 0)
  301.         LOCATE locateRow + maxHeight, locateColumn + maxWidth - LEN(b$) - 1
  302.         PRINT b$;
  303.     END IF
  304.     _DISPLAY
  305.     _LIMIT 100
  306.     RETURN
  307.  
  308.  


Oh hey! Just CHDIR to that Folder and then Tree code should work.
« Last Edit: June 27, 2020, 04:52:16 pm by bplus »

Offline bplus

  • Global Moderator
  • Forum Resident
  • Posts: 8053
  • b = b + ...
    • View Profile
Re: Development of a program for automatic search for music and movies
« Reply #88 on: June 27, 2020, 04:57:41 pm »
@SMcNeill

Hope you get flags fixed.

We don't have another Type problem between C code and QB64 with Flags and Length... do we?

Ie is a C Long the same as a QB64 Long? or worry about _signed and _unsigned?

« Last Edit: June 27, 2020, 04:59:10 pm by bplus »

Offline SpriggsySpriggs

  • Forum Resident
  • Posts: 1145
  • Larger than life
    • View Profile
    • GitHub
Re: Development of a program for automatic search for music and movies
« Reply #89 on: June 27, 2020, 05:01:29 pm »
Oh hey! Just CHDIR to that Folder and then Tree code should work.
yep! That worked! Good thinking, @bplus ! Your code finished in 4.66 seconds on my machine (5,557 directories), simply building and sorting the array whereas Steve's took over 9 seconds to do it. Nice! See, in this case, with PowerShell, mine works faster because it is a huge directory. Mine wins against yours coming in at 3.15 seconds because the large process can be handled by PowerShell much quicker. Small folders, your code. Big folders, my code. I wish I had a way to figure out whether or not to use yours or mine in a program depending on the number of subdirectories..... Hmmm.... I could do a PowerShell call to return the number of subdirectories and if it was greater than 1000 then run mine. Less than, run yours
« Last Edit: June 27, 2020, 05:09:01 pm by SpriggsySpriggs »
Shuwatch!