Streaming over TCP

[helios]

Suppose an abstract stream class has the following definition:

class ClientStream{
public:
    size_t read(void *buffer, size_t size);
    void seek(size_t position);
};

And suppose this implementation in a TCP client:

size_t ClientStream::read(void *buffer, size_t size){
    return socket.receive(buffer, size);
}

void ClientStream::seek(size_t position){
    //???[1]
    socket.send(position);
}

And the server:

while (true){
    char buffer[4096];
    server_stream.read(buffer, sizeof(buffer));
    socket.send(buffer, sizeof(buffer));
    if (/*???[2]*/){
        size_t position;
        socket.receive(position);
        server_stream.seek(position);
    }
}

The idea is that, as long as the client continues reading sequentially, the server can just send on the socket with the client only responding with ACK, but if the client does need to seek, it can reply something other than ACK (or whatever) and then send the new position. Are there any values of ???[1] and ???[2] that allow doing this, without having the server do a receive after every send?

Example communication:

(sequential reading)
C <--- DATA ---  S
C <--- DATA ---  S
C <--- DATA ---  S
C  --- ACK  ---> S
C <--- DATA ---  S
C <--- DATA ---  S
C <--- DATA ---  S
C  --- ACK  ---> S
C <--- DATA ---  S
(seek requested)
C  --- ???? ---> S
C  --- DATA ---> S
(sequential reading resumes at new position)
C <--- DATA ---  S
C <--- DATA ---  S
C <--- DATA ---  S
C  --- ACK  ---> S

[helios]

I figured out a way, although it's a major PITA.
1. Client opens a main socket S1. S1 will be used by the server to send data.
2. Client opens a secondary socket S2. S2 will be used by the client to send seek requests.
3. Server must somehow link these two sockets for every communication stream. I elected to send a 16 character random string on both sockets.
4. The server uses the following procedure:

while (true){
    S2.async_receive();
    while (true){
        //populate buffer
        S1.send(buffer);
        if (S2.receive_completed()){
            //get data from S2
            //use data
        }
    }
}

5. Care must be taken if the sockets implementation has internal buffering. In my case it did; after the client sent on S2, I had to make it receive and discard in a loop until a receive timed out. This forced the client and server to get synced.
6. Additionally, if the sockets implementation supports send coalescing (a.k.a. Nagle's algorithm), which causes delays when sending, S2 should be opened with this setting disabled.

[toad]

Why not use something a little more high level, e.g. web sockets [1, 2]?
I believe something like this would be trivial to implement:

C <---   N bytes   ---  S
C <--- wanna seek? ---  S
C  ---     no      ---> S
C <---   N bytes   ---  S
C <--- wanna seek? ---  S
C  ---  yes, 1337  ---> S
C <---   N bytes   ---  S
C <--- wanna seek? ---  S
C  ---     no      ---> S

[...]

And if you choose the value of N to be big enough,
I suppose it should be OK bandwidth-usage-wise.

[1] https://en.wikipedia.org/wiki/WebSocket
[2] https://github.com/uWebSockets/uWebSockets

[helios]

The point is to minimize latency. Adding anything on top of the socket just increases it.

[toad]

I see. Is this live streaming? (where, I presume, seeking allows the client to go back to the past or return to 'now') Also, how many clients are you targeting to serve simultaneously?

[helios]

It's a protocol for a file server, so there's no concept of "now". Streams have a finite, fixed, and known length. Think SMB or NFS.
The number of clients has no bearing on the latency, at the network level. Possibly at the storage level.

[toad]

Good. Are there existing protocol P and functionality X for which you can say "I want to implement something almost exactly like P except it also supports X / does X more efficiently (than other implementations of P do)"?

The number of clients [...]

I'm just trying to understand how much more desirable a solution
with one connection per client would be to the one you have now.

[helios]

No, I can't use an existing protocol. The metadata (e.g. directory browsing, etc.) protocol is already in place and has specific requirements. I was just deciding on a bulk transfer protocol. And it's already done as well, so...

[toad]

Alright then.