Simple text and binary protocols

Simple text protocol and binary protocol write network programs cannot hide the Protocol. The Protocol actually defines the Message format and how messages are exchanged. The protocol can be simple, complex, and sophisticated, such as the TCP protocol, and simple and unrestrained, such as the HTTP protocol. Here, I will summarize the protocols I have come into use, and finally throw out simple and common text protocols designed by individuals. Designing a protocol is not easy, especially when the design requirements contain good descriptive and scalable requirements. If efficiency is taken into consideration, it is even more mental-intensive. Before continuing the discussion, let's take a look at some existing protocols. Here we mainly discuss the application-layer protocols. Most of the application-layer protocols are request response modes (except zeromq, this abnormal guy, will be discussed later), so we will focus on message formats here. HTTP may be the most widely used protocol. HTTP is a simple text-based protocol. The message format is text-based. line breaks are used to separate key-value strings. Keys and values are separated by colons. Standard keys and values are also defined. This Protocol is highly descriptive-human readable. Strong scalability-it is easy to add custom headers, and there are almost no side effects (except for a little increase in the message volume ). However, the disadvantage is that there are too many things defined by the standard and too many details. Parsing is complicated. Memcached has two Protocols: text protocol and binary protocol. The text protocol ends a request with a line break. The request parameters are separated by spaces. To ensure binary security, add the length parameter before binary data, for example, set x 3 abc \ r \ n. The text protocol is simple, but when the request is very small, too much data of the Protocol itself is wasted (for example, each HTTP request only sends one letter, but the HTTP header may have hundreds of characters, it is a waste of time). In addition, the server still needs to perform text parsing after receiving the request, which also consumes cpu usage. Therefore, memcached launched a binary protocol to improve the efficiency of memcached. The advantage of the binary protocol is that it is highly efficient because all information is expressed with the least amount of data, and the server performs mathematical comparison rather than string comparison when parsing requests. The memcached protocol is much lighter than HTTP protocol (of course, the two are not applicable in different scenarios, so this is of little significance ). However, there are also disadvantages, that is, poor scalability. The protocol is too dead, where there is something, and what is the meaning of it. Therefore, it is unrealistic to directly use it in different scenarios. Redis protocol is also a text protocol, but it is designed to be more careful and universal (the protocols I have defined later are greatly affected), while ensuring descriptive data while minimizing the data volume of the Protocol itself, for example, in the memcached text protocol, errors are represented by "ERROR \ r \ n", while redis uses "-", and ":. This design combines the simplicity of the text protocol and the efficiency of some binary protocols. It is suitable for redis. The exchange method is also a request response. One disadvantage of the redis protocol is that the number of parameters of a message must be specified at the beginning. Unlike the HTTP protocol, empty lines are used to indicate the end. This brings about a disadvantage. It is difficult to operate in a flow. That is to say, the message must be completely determined before the first byte is sent, because the first thing is the number of parameters :-(. Most of the above are text protocols. Of course there are many services using binary protocols, such as gearman and mysql. binary protocols are equivalent to compressed text protocols, which improves the transmission efficiency, but it reduces descriptive and flexibility (in case the reserved bits for that location are not enough ). In general, this is a balance process. If the message body is not very small (for example, the message that transmits only one byte at a time is very small), it is worthwhile to use the text protocol. After all, the text protocol is simple and scalable, which is conducive to debugging (telnet can be used as a client). Although messages are read by computers, sometimes they can be viewed by others. Therefore, you can use text. Simple Text Protocol I finally defined a Simple Text-based Protocol called simpletp (Simple Text based Protocol ). http://simpletp.org This protocol is designed for simple RPC. It combines the flexibility of the HTTP protocol (ended with blank lines) and the lightweight redis protocol (size + data ). For the sake of simplicity and universality, there is no redis single character representation type design, because this will introduce additional complexity, such as the expression of integers with ":", then floating point, plural, and so on, if no data type is available, the size + data string is transmitted simply (a bit similar to zeromq, but less than zeromq), and the specific type is handled by other mechanisms, for example, protocol buffer.