The first architecture design for a multi-client system is to:
- Open 2 threads for a single connection to client (One for reading, and another for writing)
- When a job come in, we fork a thread to do some jobs.
The problem with this architecture is that if there are 2 jobs that should not be done concurrently, then there is a big problem.
We can use the synchronized statement, but the synchronized statement cannot guarantee the order of jobs.
An example from POS system that breaks this architecture is when a client opens a table and the very same client make orders immediately after.
There is a chance that the make-order job will be processed before the open-table job.
----------------
The second design is:
- Open a single thread for a single connection. Therefore, this implies that when a request is sent, it expects a result. (Because there is only one thread for read and write.
- We use that same thread to process a job (since it has to give back the response anyway.)
This one is the current architecture. The jobs are nicely put in order.
But it loses performance in some cases.
The example that breaks it is when I open table 5 and make orders on it, then I open table 6.
It turns out that the open-table-6 request has to wait because the requests must be executed in order.
And there is a performance penalty when pushing an update. Because it has to wait until there is no request to be processed, then the thread will be available for writing update to the client.
-------------------
The third architecture is superior to those two.
- There are 2 threads to read and write socket. The thread of reading socket also translate request and add job to the queue.
- Queue is divided into multiple queue. Ideally, one queue for one table. This way we can ensure the order of execution.
- A job is dequeued by a worker. When a worker finishes, it posts a response to the writing thread.
With this design, everything is asynchronous. This means that each message must be labelled with a number. Otherwise, when we post a response, we won't know which request the response belongs to.
The implementation is a little bit harder though, but the abstractions are the same. We just process jobs differently. Therefore, it is just difficult on the architecture level, not the code itself.
I think this justifies the third architecture. We cannot just discard this architecture because it is difficult to implement...
