Anyone doing website data downloads with QB64?

[Pete]

Specifically, downloads form their own sites. Example, say I want to download a data file I have at my website, Steve-is-awesome dot not. I would need to send my password through my QB64 program to my server, which is the part I don't know how to do, and then do something like Rob coded, which is displayed in the QB64 wiki at: https://www.qb64.org/wiki/Downloading_Files

Anyway, It's just my hunch data files stored in a CGI folder would not be accessible without transmitting the password, but maybe I'm mistaken. In the past, I've always used 3rd party software for this, which does require a password. For non password files, cURL or wGet does the trick.

So, has anyone had success with coding this type of a QB64 app?

Pete

[SMcNeill]

Code: QB64: [Select]

1.  SUB DownloadURL (link$, file$)
2.     f = FREEFILE
3.     OPEN file$ FOR OUTPUT AS #f: CLOSE #f 'erase any old file of the same name
4.     $IF WIN THEN
5.         out$ = "powershell.exe -c " + CHR$(34) + "Invoke-Webrequest '" + link$ + "' -OutFile '" + file$ + "'" + CHR$(34)
6.     $ELSE
7.         out$ = "wget " + chr$(34) + link$ + " -O " + file$ + chr$(34)
8.     $END IF
9.     SHELL _HIDE out$
10. END SUB

[Pete]

So, let me guess. On a Windows system Power Shell does the grunt work to connect and what... asks for a password to be typed at the command prompt? I was hoping for a non-input method, where the app transmits the password, or am I missing something here like Power Shell being used with a modified shell statement to supply the password?

Pete

[Pete]

So today I finished my 2,000 line routine, which now needs a way to upload the results to a server. I tried both Power Shell and wget, but no bueno. So back I go to have a look at one of my old time favorites, cURL. Sure enough, just a few tweaks to figure out how to combine user name, password, and upload actions in a single SHELL, and success!

The format that worked for me:

Code: QB64: [Select]

1. SHELL _HIDE "[cURL Folder\][curl] -T [upload folder][upload file] --user [user name]:[password] ftp://[user name]@[website url][folder like public html]/[sub-folders]/"
2.  

Example: SHELL "C:\mycurl\bin\curl -T C:\myfolder\my.html --user myname:pw1234 ftp://myname@www.mynamewebsite.com/public_html/qb64-files/"

Apparently, Source Forge also has something called wput, which is like wget, but only uploads files.

Pete

[SpriggsySpriggs]

You can use the WinAPI to handle FTP connections. Here is my library. I can't remember if it is fully updated. https://www.qb64.org/forum/index.php?topic=3263.0

It handles both downloading and uploading

[Pete]

Well Indy, When do you think you'll have it optimized to 10-lines of code or less? :D

Wow, impressive! That's what I imagined I'd have to do if a 3rd party app like cURL were not available to me to make shell calls. I also imagined I'd have to do it in another lifetime.

For now, I'll finish up what I'm working on using cURL, but once I get it fully working (I successfully uploaded one file last night) I will have a look and see if I can duplicate my results with your library.

Have you considered releasing your own ftp uploading software for Windows? As far as I could tell, Wget only downloads. Wput was made by a hobbyist to address that issue. I downloaded a copy, but I havn't tried it yet. cURL downloads and uploads, and supports user name and password connections, which I need. If your library does that, and could be used to keep the connection open (only one call) and have multiple files uploaded to multiple folders while the connection stayed open, I think you'd have a winner on your hands. I mean I love products like Filezilla, but apps of that sort are more for navigating to folders and dragging and dropping files from client folders into the server directories. That's slow compared to an automated upload system, which assigns variables to client and server paths, and uploads files to the appropriate folders.

Thanks for the link to your library,

Pete

[SpriggsySpriggs]

There is also another library (using WinAPI) I've got somewhere in my folders for downloading any file from the internet, including HTTPS links. I'd have to find it again if you want it and/or are interested in it.