Dora-RPC-Dragon Boat Festival upgrade PHP microservice development kit Dora-RPC-Dragon Boat Festival upgrade. Currently, the master has added a large number of features in beta.
Several companies have been using the project online for more than a year. of course, they have made some changes based on this prototype. Recently, the upgrade has improved a lot. for example, if unique function is removed, the performance will double. it is found that some functions on the server will be repeatedly executed and the problem has been fixed. In addition, the asynchronous task is added to get the result. Currently, groupclient is being transformed. later, the groupclient function will be integrated into the client, instead of the two clients.
I have not provided any in-depth introduction before. here I will briefly introduce some of the current features:
This open-source package uses Swoole extensions for PHP frontend and backend separation. when our projects become large, we need to isolate some functions as public services, we hope to use this method to make our complex systems simple and easy to maintain through API services, so as to better assess complex projects and make cross-project group calls clearer.
Framework thinking:
- Only a basic set of servers and clients are provided, which can be used in any development framework through simple integration.
- The returned data structure is divided into three layers. each layer has msg and code. The first layer (bottom layer) indicates the current client communication, the second layer indicates whether the current api is executed on the server or whether there is an exception or delivery status. The third layer indicates the code execution status. if the business code is abnormal, it is automatically recorded here. This method is used to determine the true state of the system due to the complexity of RPC.
- To support later retrieval of asynchronous tasks, the underlying layer of the development kit has been upgraded, mainly because the server will send the processing result to the client after processing the asynchronous task, if the current client sends a new task to the server and executes the Recive, the client cannot determine whether the returned content is the request. due to the particularity of the long link, if the last request is interrupted due to an exception and does not execute recive, the result of the previous request will be obtained by this request. In order to prevent the above problems, The guid generation function is used to complete the collection of asynchronous task results and the identification of returned data from this request.
- Due to the special nature of our business code, we cannot guarantee that our business code will not fail due to some special reasons, and many failures will lead to process termination, therefore, our business code is executed on the task rather than the worker. In this way, we can do many special functions. However, the disadvantage of tasks is that many swoole asynchronous features are not supported.
- Service Group. in many cases, we are a public pool. since we are in the same day, we will receive an impact. In addition, different services are deployed on different servers. Therefore, we provide the service grouping function. Specifies which groups of servers a server belongs to, and the correct server configuration can be found through the group for connection.
Task delivery mode:
The number of tasks assigned can be divided:
- Concurrent Task delivery
- Single task delivery
There are multiple modes for sending task wait results:
- Issue Task blocking the current process waiting for processing to complete and get the returned data
- If an asynchronous task is issued without waiting for the processing result of the task, the task is successfully delivered, and no result is returned after the server completes running.
- Asynchronous task issuance does not wait for the task to be processed on the server. if the task is successfully delivered, the system returns the successful delivery. after the service is complete, the server returns the processing result to the client, the client returns all the previous asynchronous task results to the current business process through a function.
Through the above method, we can multiply the response speed of our interface and the server's exploitation capability. We can achieve our goal even without complicated multithreading.
Network Communication:
- Clients can work in PHP-FPM and CLI to maintain long links
- The network time consumed for each round-trip request is 0.002 ~ About 0.004 (also depends on the intranet)
- After the request is processed, the link is retained but not closed to reduce the handshake time consumption and the number of ports consumed by the client server.
- If the data size is large, gzip supports package compression. if you have special requirements, you can simply change it to another compression method.
- Serialize of PHP for serialization can be replaced directly if needed
- By default, the connection and receiving results time out for 3 seconds. it is set in the const file. if a timeout occurs, the client will automatically use other configurations to retry a specified number of times and return a json description of the error cause. If necessary, you can add retry monitoring in the retry logic to monitor the working status of each server.
- If an ip address in this request has failed once, it is automatically blocked in this request. if you find that all configurations have been tried or the number of retries has exceeded, a failure is returned. When the API is issued again after the next failure in the cli, all the previously blocked configurations will be canceled and the retry will continue.
- Previously, the QPS of RPC single-threaded clients and virtual machines was approximately 200 ~ 500 communications per second, but each task can have a lot of QPS on a 2 k single application server
- The current communication architecture does not involve the reverse proxy service, because it takes some time for the client to connect to the reverse proxy. In addition, although the reverse proxy server has high performance, it is also a "single point". for distributed services, direct connection is more convenient.
Service discovery:
- When our servers need to increase or decrease, our normal process is to update the client configuration to increase the number of new servers. However, this method still has many disadvantages if a server fails, in a short period of time, we cannot find out. after the discovery, we still need to manually modify the configuration or remove it from the O & M configuration center. through service discovery, we can automatically remove and add servers, as long as our control logic is improved, we can make the process completely automatic later.
- This framework provides the simplest implementation method for service discovery. redis is used as storage to record the ip addresses and ports of all servers and their groups.
- After you start the application server and tell him the list of redis instances discovered by the service, the server automatically reports the ip addresses, ports, and groups of the current service to multiple redis instances for registration, and regularly refresh the timeout value. If some nodes are not reported after the time-out period is exceeded, they are automatically removed.
- The client regularly obtains the latest configuration from multiple redis instances and updates the local configuration file.
Later support:
The next step will support distributed logs, configuration synchronization, etc.
Finally, Dora-RPC is a very open source and intended to provide excellent RPC and microservices for Swoole community users as a reference.
At the same time, we hope that you will participate in the development of this project. any valid participants will be recorded in the wiki.
Project address: https://github.com/xcl3721/Dora-RPC
Recent Activities: http://www.huodongxing.com/event/5337738177600 share SOA experience, free tickets
