I see, so your suggestion is a single data in/out mutex per connection.
This would only work if data could not be written from multiple connections however. For example, if the PLC and a separate machine were both to have the ability to control the Battery switch state, the two threads could receive messages at the same time to set the same data. Those locks on the connections would only work if they were shared, and starts reposing the question of having locks on systems instead.
Also, I should point out that the only reason why a single master mutex idea is applicable is the requirements of the system. Data/logic needs to respond exceptionally fast to any input, but the inputs are not necessarily happening quickly or often. We receive a packet once every 50 ms on our fastest port, but need to respond in less than 10 ms. Some of our slower connections are only communicating at 10 Hz. The amount of time spent in a write lock is minimal and not often done, so the chances of threads waiting to write is severely reduced.
|Why not using the safe TCP/IP connection?|
Ha, I asked this same question multiple times when I first started this job. In the end, there are simply too many external devices that unreliably broadcast their data (GPS are notorious for this), and essentially only do so through UDP. Also, our real-time operating system isn't a huge fan of TCP/IP for speed sake. It simply has a fairly crappy custom card which I assume was due to no expectations of customer's using it.
|Don't use mutexes... Use asynchronous message passing.|
As far as I've seen in C++ (which we can't avoid due to the nature of the VxWorks OS), asynchronous message passing still requires a critical section. This is due to the queue itself. You can't increase and decrease the size of the message queue across threads without thread safety. The queue.size variable could become corrupted. Most asynchronous message passing methods are in other languages that circumvent this behind the scenes. Certainly correct me if I'm wrong on this.
|Use immutable data structures to store state.|
I don't see how that's possible to always use immutable types in a C++ environment in a simulation with many dynamically changing variables. Perhaps I misunderstand you on this one?
|Use a language that is suited for these kind of things.|
Stuck in C++, it's the only thing our kernel will run. Also, language should never be a pitfall if you know how to design.
Anyway, I read through those and they don't say anything useful at all that applies to my problem. Yes you should avoid threading where possible, but being that we have many real-time systems trying to communicate and some of them have VERY particular timing requirements, CPU concurrency is a requirement.
I do appreciate the Akka suggestion. It's a fine model, but our OS won't support it's libraries. This executable is practically running on an embedded system, but an oddly powerful one. C++ and stdlib only unfortunately.