put / get buffer binary mode tcp

[Parkland]

Ok,
I made a program which is a transfer program for what I'm making, a peer to peer ap.
so the program so far (not 100% done) can copy files disk to disk, download from network to disk, upload from disk to network.
It has 1000 transfer command queue capacity, but the program only runs 32 at once and advances the queue as they complete.

So far, I am using binary mode, and get and put, 1 byte at a time.

I am curious; the beginning of the file I transfer sometimes is missing data in the beginning,
I made the program wait for 1 second before the data starts transferring.
Other than the beginning, it's all intact.

I am wondering though, is this data protected by tcp/ip for errors and buffering?
If the connection drops or slows down, does the sending computer pause at the put command until a value is read?
Or does it just keep sending bytes and they're lost?

[Parkland]

 I guess I'm not even sure what I'm asking.
I'm curious about how the qb64 interfaces with the network protocol I guess.
I have 1 computer with qb64 and a program running, sending a file, byte by byte using put #a, ,x$

and another computer with qb64 and another program using get #a,, x$    ; and adding x$ to a file in binary mode.

So, what happens when the one computer sends faster than the other recieves? does qb64 automatically slow down the sending program to not overload the tcp/ip stack?
Or does data start getting lost?

And is this the right way to send data? Is the output bytes filling a buffer and sending as a stream of packets? Or is it triggering a packet for every single byte and wasting tons of network bandwidth?

[Unseen Machine]

TCP/IP gurantees the data and the data order but doesnt work very fast. UDP is WAYYYYYYY faster bu you have to manage the data as the order an actual data transfer is not guranteed (hence it's nickname being the Usless Datagram Protocol!)

[Parkland]

OK interesting.

so using TCP/IP in qb64 (the only internet connectivity method available), a person shouldn't have to worry about error correction etc.

[Parkland]

  [ You are not allowed to view this attachment ]    [ You are not allowed to view this attachment ]  

Ok did some experiments over a wireless connection to my laptop, and a desktop computer sending the data.
I tried sending 1 byte at a time, 4 bytes, then 8.
Sending 1 byte per PUT instruction, you can see that the sending computer is slower than the wifi speed.
Sending 4 bytes per instruction is faster.
Over 8 bytes per PUT on the sending computer makes no difference.

Interesting I was wondering if sending 1 byte at a time was wasteful and inefficient, but it seems to have rather low network overhead regardless.

Unfortunately, qb64 really uses a lot of CPU to move data at just 3 megabytes per second.
Maybe because of the PUT and GET instructions as opposed to using INPUT and PRINT with larger strings.


** the first pic is 1 byte at a time, 2nd pic is 4 bytes at a time.
*** I used 1 kb=1024 bytes, 1 MB=1024kb

Sending computer just sends a character using out. Recieving computer adds the byte to a string, then displays string length every second. (and erases string)

[Parkland]

Ok,
I found some things out sending and receiving data in binary modes over tcpip.

The string size can be increased to raise data rate and lower CPU usage.
However, there seems to be data lost sometimes :(

When i add delay in the sending computer program, the file is the right size (240mb)
When I remove the delay instruction , the file on the receive computer ends up being a random (smaller) size.

So, I'm not sure.
Could have 2 connections open? 1 to transfer data and the other one to link back whats received?
In the system task monitor, it is uploading data while downloading, so it *seems* like qb64 should already be doing this, but maybe not?

[FellippeHeitor]

Connections are bidirectional, you can get and put simultaneously.

Amount of data is controlled by the system, just send away. The receiving end should get the data sequentially until an agreed end marker is received.

[Parkland]

.... How come by slowing the sending computer down , the receiving computer is getting all the information,
and if I let the sending computer run full speed, information is lost :(

...I'm still experimenting, maybe I'll figure it out yet

[FellippeHeitor]

I suspect your GET side is not storing things properly in a buffer:

Code: QB64: [Select]

1. 'this should be done until no more newData$ or until an agreed end marker is received
2. GET #Handle, , newData$
3. receivedTotal$ = receivedTotal$ + newData$

Do that in a loop.

[Parkland]

This gets repeated until lpc = length of file, which is exchanged before the file is transffered.

    GET fn1(x), , i$  (reading from network)
    PUT fn2(x), , i$  (saving to file)
    lpc= lpc + LEN(i$)

The sending computer is a desktop with ethernet to switch. Laptop is the reciever on WIFI.
The problem seems to be that this works at low data rate (1 byte per PUT) it works as it should.
I can increase bytes per PUT until the network backs up, then I seem to recieve less than is being sent.
Adding delay in sending computer program fixes it.

Also the send program is very simple, recieve program is huge and slower.
I checked, and the recieve string is sometimes up over 1000 bytes, so lots slower than the sending program.

[Parkland]

Could I be doing something wrong on the sending computer?
It looks something like:

DIM a as STRING * (2 works, playing with higher numbers)
bt=2 (or same as A above)

open "file" for binary as #1
open "net" for binary as #2

for x = 0 to 10000 step bt
get #1,x,a
put #2,,a
next

[FellippeHeitor]

What are the actual commands you're using for this bit of pseudocode?

Code: QB64: [Select]

1.  open "net" for binary as #2
2.  

[SMcNeill]

DIM a as STRING, L AS LONG
bt=2 (or same as A above)

open "file" for binary as #1
open "net" for binary as #2

L = LOF(1)
a$ = SPACE$(L)
get #1,1,a$
PUT #2,,L
put #2,,a$


Then read the file size as a long, and DO... GET.... LOOP UNTIL the info received = that size.

[Parkland]

  [ You are not allowed to view this attachment ]  

SCREEN 12
DIM i AS STRING * 32 ' set length of string to get from file/ send on net
bl = 32 ' for program calculation


server& = _OPENHOST("TCP/IP:59999")
PRINT "Waiting for connection..."


DO
    connection& = _OPENCONNECTION(server&)
LOOP UNTIL connection& <> 0

PRINT "recieved connection"

GAME:
PRINT "sending bytes"
_DELAY .5
OPEN "beavis.avi" FOR BINARY AS #1


' get file length, make it be 32 bytes
p$ = STR$(LOF(1))
PRINT p$
x = LEN(p$)
q = 32 - x
FOR z = 1 TO q
    o$ = o$ + " "
NEXT
p$ = o$ + p$
PRINT LEN(p$)
PRINT p$
FOR z = 1 TO 32
    a$ = MID$(p$, z, 1)
    PUT #connection&, , a$
NEXT


PRINT "Waiting for send instruction..." ' wait for "!" from client
wfi:
GET #connection&, , a$
IF a$ <> "!" THEN GOTO wfi
PRINT "Sending data ..."
_DELAY 0.2


' send file data

lf& = LOF(1)
PRINT lf&
PRINT "Sending bulk data..."; bl; "bbyte legnth"
OPEN "beavis2.avi" FOR BINARY AS #3 ' for testing, write to local disk the data being sent over net



FOR x& = 1 TO lf& STEP bl ' repeat until end of file, increment of bytes sent each time
    GET #1, x&, i
    PUT #connection&, , i
    PUT #3, , i ' to local disk, for testing
    '    _DELAY (.0001)  ' Playing around with this number changes the data recieved on other end
NEXT






dtft:



CLOSE
CLEAR
END

[Parkland]

The program works;
the source file data is read, sent to other computer proper, then data starts transferring, and saving.
Everything works, just that at a certain speed, it seems data starts getting lost.

The more I slow down the sending computer/ program, the more data is recieved on the recieving computer / program.