Text Only | Text with Attachments

QB64.org Forum

Active Forums => QB64 Discussion => Topic started by: FilipeEstima on December 16, 2018, 12:14:09 pm

Title: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 16, 2018, 12:14:09 pm
Hey fellows.

I can work myself around QB64 on doing trivial things, but now I need to learn if it's possible to make online requests to a webservice (and how to do it). The webservice in question is provided by the brazilian mail company Correios and is used by online stores to send information about ZIP origin and destination, weight, dimensions etc; in return the e-commerce gets estimated delivery days and prices for the services asked. The results come in XML format, which I should be able to interpret.

My main problem is, how do go I about connecting my QB64 software to the webservice? I don't know anything about http protocols.
Title: Re: Need help using QB64 to access a webservice
Post by: FellippeHeitor on December 16, 2018, 12:20:24 pm
What web address must you query?
Title: Re: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 16, 2018, 12:32:02 pm
Hi Fellippe, how are you?

This one:

http://ws.correios.com.br/calculador/CalcPrecoPrazo.asmx?op=CalcPrecoPrazo

And this one:

http://ws.correios.com.br/calculador/CalcPrecoPrazo.asmx?op=CalcPrazo

I can only get you an example of the second, as I don't yet understand how to put all the needed arguments on the first one.

http://ws.correios.com.br/calculador/CalcPrecoPrazo.asmx/CalcPrazo?nCdServico=10154&sCepOrigem=69910000&sCepDestino=99920000
Title: Re: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 16, 2018, 12:42:39 pm
Here's the manual on how to make such queries:

https://correios.com.br/solucoes-empresariais/comercio-eletronico/palestras-correios-1/pdf/ManualdeImplementacaodoCalculoRemotodePrecosePrazos.pdf
Title: Re: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 16, 2018, 12:50:15 pm
I got it. To make queries for price and delivery days:

http://ws.correios.com.br/calculador/CalcPrecoPrazo.aspx?sCepOrigem=70002900&sCepDestino=71939360&nVlPeso=1&nCdFormato=1&nVlComprimento=20&nVlAltura=20&nVlLargura=20&sCdMaoPropria=n&nVlValorDeclarado=0&sCdAvisoRecebimento=n&nCdServico=04510,04014&nVlDiametro=0&StrRetorno=xml

Still I don't understand how to make such queries from within a QB64 program, and receive the result in XML, so I can make a routine to gather the useful data.
Title: Re: Need help using QB64 to access a webservice
Post by: FellippeHeitor on December 16, 2018, 01:49:38 pm
You can use the download function Galleon wrote (I'm pasting it here from the wiki, adapted to your case):

Code: QB64: [Select]
  1. server$ = "ws.correios.com.br"
  2. query$ = "/calculador/CalcPrecoPrazo.asmx/CalcPrazo?nCdServico=10154&sCepOrigem=69910000&sCepDestino=99920000"
  3.  
  4. PRINT "Attempting to fetch data..."
  5.  
  6. IF Download(server$ + query$, "correios.xml", 10) THEN ' timelimit = 10 seconds
  7.     COLOR 14: PRINT "Success!": COLOR 15
  8.     OPEN "correios.xml" FOR BINARY AS #1
  9.     DO
  10.         IF EOF(1) THEN EXIT DO
  11.         LINE INPUT #1, dataLine$
  12.         PRINT dataLine$
  13.     LOOP
  14.     CLOSE #1
  16.     PRINT "Download failed."
  18.  
  19. FUNCTION Download (url$, file$, timelimit) ' returns -1 if successful, 0 if not
  20.     url2$ = url$
  21.     x = INSTR(url2$, "/")
  22.     IF x THEN url2$ = LEFT$(url$, x - 1)
  23.     client = _OPENCLIENT("TCP/IP:80:" + url2$)
  24.     IF client = 0 THEN EXIT FUNCTION
  25.     e$ = CHR$(13) + CHR$(10) ' end of line characters
  26.     url3$ = RIGHT$(url$, LEN(url$) - x + 1)
  27.     x$ = "GET " + url3$ + " HTTP/1.1" + e$
  28.     x$ = x$ + "Host: " + url2$ + e$ + e$
  29.     PUT #client, , x$
  30.     t! = TIMER ' start time
  31.     DO
  32.         _DELAY 0.05 ' 50ms delay (20 checks per second)
  33.         GET #client, , a2$
  34.         a$ = a$ + a2$
  35.         i = INSTR(a$, "Content-Length:")
  36.         IF i THEN
  37.             i2 = INSTR(i, a$, e$)
  38.             IF i2 THEN
  39.                 l = VAL(MID$(a$, i + 15, i2 - i - 14))
  40.                 i3 = INSTR(i2, a$, e$ + e$)
  41.                 IF i3 THEN
  42.                     i3 = i3 + 4 'move i3 to start of data
  43.                     IF (LEN(a$) - i3 + 1) = l THEN
  44.                         CLOSE client ' CLOSE CLIENT
  45.                         d$ = MID$(a$, i3, l)
  46.                         fh = FREEFILE
  47.                         OPEN file$ FOR OUTPUT AS #fh: CLOSE #fh 'Warning! Clears data from existing file
  48.                         OPEN file$ FOR BINARY AS #fh
  49.                         PUT #fh, , d$
  50.                         CLOSE #fh
  51.                         Download = -1 'indicates download was successfull
  52.                         EXIT FUNCTION
  53.                     END IF ' availabledata = l
  54.                 END IF ' i3
  55.             END IF ' i2
  56.         END IF ' i
  57.     LOOP UNTIL TIMER > t! + timelimit ' (in seconds)
  58.     CLOSE client

Then all you need to do is build the query$ string and send the download request. The resulting file is your XML to parse.
Title: Re: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 16, 2018, 02:03:59 pm
Thank you, I'll  give it a try and let you know if I got it right.
Title: Re: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 17, 2018, 11:01:00 am
Hi Fellippe, while I waited for a reply here, I found that curl could be used as well. With curl I must direct the result to a file, then I have to open that file and examine it. Since I'm going to make a lot of queries, this approach isn't optimal.

The solution you provided is much better, because if I simply make d$ shared, I can comment the lines that save the query result to a file and work directly with d$, therefore much faster because it doesn't involve the OS saving a new file over and over again. Thanks!
Title: Re: Need help using QB64 to access a webservice
Post by: FellippeHeitor on December 17, 2018, 11:26:54 am
Good to hear you managed to improve the solution. Best of luck with your new program!
Title: Re: Need help using QB64 to access a webservice
Post by: FilipeEstima on December 17, 2018, 03:43:52 pm
I finally made the program I wanted, thanks to your help. When running it, I noticed the time it takes to finish the job is quite long. It has to make about 6.000 queries and it takes almost 4 hours to finish. So I divided the list in smaller parts and made 8 versions of the program, each one responsible for a part of the total list.

That's a hell of an ugly solution; now I'm back to ask for ideas. I suppose there's no way for a single instance of the program to make many parallel queries, right? By many I mean 8 or more in parallel, instead of serial queries.

Another solution would be to launch one program ("master") that would call many copies of another program ("servant"), providing each one with "start" and "end". In this case the "servant" program contains all the necessary information, and each instance would use only the portion between "start" and "end" as determined by the "master". So instead of 6.000 serialized queries, I'd have 750 queries in parallel (8 at the same time).

Maybe I made it sound more confusing that it actually is LOL
Title: Re: Need help using QB64 to access a webservice
Post by: FellippeHeitor on December 17, 2018, 03:48:06 pm
I don't know what the limit for _OPENCLIENTs is, but you surely could tweak the Download function to work with more than one connection at once. I imagine it'd take an array to store client numbers and separate variables to store incoming data, but it sure sounds doable.

If you get the latest development build you can have TYPEs with variable width strings. Something like

TYPE Connection
    Client AS LONG
    ReceivedData AS STRING
END TYPE

DIM SHARED connection(100) AS Connection

Then work the function to keep track of those.
Title: Re: Need help using QB64 to access a webservice
Post by: Cobalt on December 20, 2018, 10:38:20 am
Quote from: FellippeHeitor on December 17, 2018, 03:48:06 pm
If you get the latest development build you can have TYPEs with variable width strings. Something like

Did Luke get that ironed out? last I knew we couldn't save it correctly directly with PUT ##,,UDT_array
or do you mean for him to reassign the UDT string to another string and then save?
Title: Re: Need help using QB64 to access a webservice
Post by: FellippeHeitor on December 20, 2018, 10:40:19 am
This is direct run time manipulation, no file saving involved.

Due to the nature of variable width types, QB64 will prevent you from using them with Get and Put, as previously discussed.
Title: Re: Need help using QB64 to access a webservice
Post by: luke on December 23, 2018, 01:55:12 am
Quote from: Cobalt on December 20, 2018, 10:38:20 am
Quote from: FellippeHeitor on December 17, 2018, 03:48:06 pm
If you get the latest development build you can have TYPEs with variable width strings. Something like

Did Luke get that ironed out? last I knew we couldn't save it correctly directly with PUT ##,,UDT_array
or do you mean for him to reassign the UDT string to another string and then save?
For now you can't use UDT's with variable length strings in GET or PUT operations, but they're perfectly functional otherwise (I just fixed the issue it had with LEN).
Text Only | Text with Attachments