Architecture Question regarding Threading In Low Latency Network Application

[cowboy2474]

Hi,

I am building a low latency network application where I have to forward certain packets immediately upon receipt (I'm currently doing this using a kernel bypass, low latency API provided by the manufacturer of my network card, and in the same thread as the one in which I handle the received packet, so that part is airtight).

However, in some scenarios, I must hold up and queue the packet which I am forwarding and/or send a (non-latency critical) packet back to the sender. This packet I'm sending back to the sender doesn't have to be sent in a fast manner at all, however, it should be done in a way where it has minimal impact on my ability to process subsequent packets coming.

I can provide a bit more as far as code samples, but basically I have a single thread which is FIFO scheduled, 100% on a single core, busy waiting to receive packets.

Busy waiting thread:
while(waitforpacket()) {
if(received(p)) {
decidetosendpacketorqueue(p);
send_acknowledgementbacktosender(p, sender(p));
}
}

Basically, how should I structure 'send_acknowledgementbacktosender()' so that there is minimal delay in handling the next packet (subsequent p, possibly waiting to be death with)?

Should I hand it off to another thread? Create a new thread there on the fly or have a pool of worker threads I should rotate between? What is the proper design pattern/technique for such a scenario given my requirements?

Furthermore, is this easily doable using pthreads (or is there some other library more amenable for what I want to do)? Does anyone have any sample code for handling such a 'slow, asynchronous ack on other thread, but minimize delay to next instruction' operation?

Thank you all!

[kooth]

Depending on the size of the packet of the "slow, asynchronous ack", you could use out-of-band data. If the packet is "smallish", OOB data might be enough. You would have to calculate where you start to see degradation of the turnaround time to know if this is a viable design.

In your algorithm you have the "busy waiting thread." Are these threads started per connection, or does one thread handle all of the traffic? If you create a thread per connection, then your problem goes away: The thread handling the current request is not affected by another connection.

Using pthreads should be fine.