put / get buffer binary mode tcp

[Parkland]

For EG,
This "beavis.avi" file, is 240mb
If I leave the "_delay" command commented out so it runs full speed, the receive computer ends up with a ~75mb file.

If I use the delay command, the received file is larger.

[Parkland]

Code: QB64: [Select]

1. w1:
2. c = _OPENCLIENT("tcp/ip:59999:192.168.1.105")
3. IF c = 0 THEN GOTO w1
4. PRINT "Client filehandle : "; c
5.  
6.  
7. w2:
8. GET #c, , i$
9. fl$ = fl$ + i$
10. IF LEN(fl$) > 31 THEN GOTO dfl
11. GOTO w2
12.  
13.  
14. dfl:
15. fl& = VAL(fl$)
16. PRINT "File length:"; fl&
17. e$ = "!"
18. PUT #c, , e$
19. PRINT "Sent the send code, waiting on data..."
20. OPEN "beavis.avi" FOR BINARY AS #1
21. x& = 1
22.  
23.  
24. giterdone:
25. GET #c, , q$
26. PUT #1, , q$
27. x& = x& + LEN(q$)
28. IF x& > fl& THEN END
29. GOTO giterdone
30.  
31.  
32.  
33.  
34.  
35.  
36.  
37.  

  [ You are not allowed to view this attachment ]  



This is a receive program I just made, still same issue, not recording all data to disk, as if not all data from send program is making it over the network.

[Parkland]

I made 2 more little programs,
The send one sends a 10 byte string 100,000 times,

The receive one uses GET to read it, in the form;

X:
Get connection,,d$
data$=data$+d$
locate 1,1 :  print len(data$)
goto x

and I ran it 4 times, all 4 times the receive end string stopped around half what was sent, but never the exact same.

What the heck is going on? Is there something incompatible? I have linux mint on desktop and laptop, is there some kind of buffer issue with qb64?

[Parkland]

I'm going to take a wild guess and it seems like the buffer is 64kb.
In the screenshots, I used just plain string data, ignore message about file, that data isn't used.

Sending 64 bytes 1000 times works perfect.
sending 64 bytes 2000 times data is lost.
Sending 64 bytes 2000 times but with a time delay on the sending computer, perfect again.

Does anyone know if this is a bug? Or just the way it's supposed to work?
I guess I can make the file transfer be 64kb at a time, and the receiving computer asks for more data whenever it saves the last 64kb.

[Parkland]

There must be a buffer / stack overflow issue.
I modified the programs.

Now, the computer that's downloading data and saving it on disk, sends "total received data" to the sending computer whenever it receives data.
The sending computer, keeps the record of the total received data on the receiving computer, and won't send data exceeding that number plus 64000.

I'm not sure what size the buffer could be, I'm guessing 64kb, but things happen so fast that it's hard to say.
I can say for this set up / speed etc, when I keep the sending computer from sending more than 64kb that the receiving one hasn't received, I get the file 100%.
If I change my buffer size to 80kb, I only got 99% of my file.

So it really seems that using the PUT command, you can PUT up to 64kb (?) of data and it will make it to the other computer, but over 64kb(?) it just disappears.

[Parkland]

This definitely complicates things.
With such a small buffer with the TCP/IP, the sending program has to only keep sending data while receiving data from receiving computer of the received data, to insure a 64kb gap isn't exceeded. This isn't too terrible in the house, latency is low.
If latency became high, like over the internet, speed would drop way off.

Or-

The sending computer sends data as packets made in QB64, the receiving computer sends a message back to indicate if it was received, then the sending computer if it stops getting received reports, re sends the lost information, and slows down sending rate, so that hopefully the buffer doesn't get exceeded.

[Parkland]

  [ You are not allowed to view this attachment ]    [ You are not allowed to view this attachment ]  

This literature seems confusing 🤔
So get and put read and write "raw" data.... so IS that udp protocol then? Or just a tcp/ip stream?

From reading that, it doesn't seem like get and put are a reliable way to transfer data without designing some kind of packet programming 🤢

[Parkland]

Upon further reading, the linux UDP transmit buffer is 65536 bytes, exactly the limit I thought I was hitting.
So, I'm pretty convinced that get and put are using UDP stream not tcp/ip