How can I enable the middle layer of the middleware, such as the middleware and datasnap, to support a large number of concurrent users and control the number of connections?

I recently wrote an intermediate layer based on xe2 datasnap. I have also written a DCOM-based middleware layer. I can see some colleagues on the Internet asking questions about the middleware layer. Here are two typical questions:

First question:

How can the three-tier program of the "das" system support simultaneous connection requests from a large number of users?
System Structure: tclientdataset-> tsocketserver-> scktsrvr.exe-> remotedatamodule-> ado
　　
Requirement: Allow-clients to "request" connections at the same time, and neither server nor client can die.
Status quo: in an existing program, the application server stops responding as long as the number of clients connected to the request reaches a large value, for example, 80 or 100. When the client tries to connect, it cannot establish a connection or even obtain

An error is returned, but the response is stopped together.

Expectation:-clients can send connection requests and try to connect at the same time. Of course, you can limit the number of concurrent jobs, even if it is limited to a small number; however, the number of clients that exceed this limit is being executed

Tsocketconnection. when opening, you must be able to immediately receive a response message "the number of connections is full and connection is denied"; or enter a waiting queue, retry a specified number of times or retry a specified time; or, even if only one exception is thrown

. However, it is not allowed to stop the response and to die silently.

Second question:

How do I limit the number of connections in connection with a three-tier program based on MIDAS/socket/scktsrvr?
　
Architecture: tsocketconnection-> scktsrvr.exe-> remotedatamodule
　
Status quo: a large number of clients (such as 200) Initiate connection requests at the same time. The middle layer or scktsrvr obviously cannot process so many connections, but does not give up, resulting in the death of the client, middle layer and scktsrvr.
　
Expectation: Let scktsrvr limit the number of sockets. A client that exceeds the limit (for example, 20) can send a connection request to scktsrvr, but scktsrvr is not allocated to the socket and rejects the connection. After the client times out, the dialog box prompts you.

Questions about the intermediate layer on the Internet may seem strange. In fact, the above two major questions are nothing more than the one mentioned above (although the above questions are all about Midas, they are also suitable for datasnap ):

1. Enable the middle layer to support a large number of concurrent tasks without killing them;

2. Control the maximum number of connections to prevent servers from being killed because hardware resources are exhausted. In fact, these two problems can be said to be the same issue.

Can these two problems be solved? The answer is yes.

First, describe how many colleagues write the intermediate layer:

1. Select "session" for the middle-layer thread service mode, which is also the default thread mode of DAS/datasnap;

2. Put some controls such as database connection controls and dataset controls on the remote data module ......

It is undeniable that it is the easiest way to write the middle layer, which is also described in many textbooks. However, this causes the above problems.

This method consumes resources on the server. When a client is connected, the server opens a thread to serve the server until the client terminates the connection. Why is it heavyweight? When the thread is enabled, a remote data module is created, and a large number of dataset objects and data connection objects in the remote data module are created, so many objects exist throughout the client connection period and can only provide services for one customer connection. Because server resource consumption is heavyweight, once the number of customer connections is a little large, the server's hardware resources are likely to be exhausted immediately. In this case, if the maximum number of connections is not controlled, the server can only be used up because of resource depletion. This is the disadvantage of writing the middle layer in this way. That is, This method determines that the number of concurrent clients supported by the intermediate layer is very limited, because each client occupies heavy resources on the server: first, it takes a long time (as long as the client still maintains the connection, the server resources will be occupied), and second, the server needs to create a large number of objects for each customer to connect to the service.

How can this problem be solved?

1. The occupation of heavyweight objects is lightweight; 2. The occupation of long time is real-time occupation.

As a result, objects in the object pool can connect to all customers, and not some object resources can only connect to a single customer.

The most important thing is to change the thread service mode of the server: the default "session" thread mode cannot be used, and the thread pool mode should be changed. For the operating system, the number of threads that can be created is always limited. For example, in windows, about 2000 empty thread objects can be created. After being changed to the thread pool mode, the number of threads is not limited.

Transforming to a thread pool is the first step, and then transforming the database connection object pool and the DataSet object pool ......

What are the benefits of this implementation?

Objects in the object pool are only occupied in real time. This occupies only the moment when one event action of a customer occurs. When this event is completed, objects are returned to the object pool (only returned to the pool, and no objects are released ). Objects can be created and released frequently, reducing the time required to create many objects (Object creation is time-consuming, not to mention the creation of so many objects, the amount of time consumed ). Reduces memory fragments generated by frequent object creation and release (each creation and release of an object generates a small amount of memory fragments, which seems insignificant. But you have never thought about how much memory fragmentation will be generated when the server runs and the number of objects frequently created and released is so large. Memory fragmentation 1 affects the server's execution performance, and 2 may enable the server to act as a server ).

How to control the maximum number of customer connections?

Set the upper limit (poolsize) of the server thread pool to control the maximum number of threads created in the thread pool and the maximum number of client connections.

How many suitable are the upper limit settings?

It depends on the hardware configuration of the server. The actual stress test is available.