Switch: easy to understand the iso8583 packet Protocol

When I first entered the financial industry, I knew the is08583 Message Protocol. I think I have heard of the is08583 protocol before entering this industry. I know how influential iso8583 is. When I first came into contact with it, I was confused about some of the details. In view of this, I think many of my peers will inevitably go through the same stage, so I write this article so that you can take less detours. At the same time, I wrote on the Internet (http://blog.csdn.net/lysheng/archive/2005/03/03/309914.aspx) I want to write "comprehensive understanding of iso8583 packets" and "comply with the CEN/XFS (that is, wosa/XFS) specification of SP writing" two articlesArticleMany people ask me when I can write it out. I know that many people need to understand this knowledge, even if I have less time, we also need to try to write these two articles to provide some reference for those who need them.

I think it is meaningless to simply describe the definitions of the is08583 fields. The standard has already explained each field in great detail. If you think it is difficult to understand the iso8583 standard in English, there are also similar behaviors on the Internet. We have translated the Chinese version of iso8583, so my goal is to learn about iso8583 after reading this article, so that people who have never been in touch with it can comply with the iso8583 packet specification.

Now, we should turn into the subject.

At the very beginning, financial systems were provided only by big companies such as IBM, such as various hosts and terminals. Data needs to be exchanged between various computer devices. We know that the data is transmitted over the network, and the data transmitted over the network is based on binary data such as 0 or 1. If the data is not encoded, no one can understand the data, which is useless. At first, X.25, SDLC, and the popular TCP/IP network protocols all provide underlying communication encoding protocols, which solve the underlying communication problems, it can transmit a string of characters from one place to another. However, it does not make much sense to transmit strings only. It is very important to parse what the strings represent. Otherwise, it is useless to transmit some strings of "0123abcd.

Let's go back to a time point a few decades ago. If we are pushed to the stage of history, we will design a universal packet protocol to solve the packet exchange between financial systems, currently, this protocol is called iso8583. At this time, the technology is constantly moving forward. At the beginning, it seems that IBM was not doing well in a standalone situation. Companies of different sizes have entered the financial industry in order to gain some benefits, and they are in a wild situation. How can we design a packet Protocol to include all the companies that have sprung up? In fact, it is not very easy.

Let's take a step-by-step approach. In fact, the financial industry does not involve thousands of data and cannot be counted. On the contrary, the data is relatively small. We can get it from the bottom, such as the transaction type, account, Account type, password, transaction amount, transaction service fee, date and time, and merchantCode, 2 magnetic 3 magnetic data, transaction serial number, etc., all the data that can be summarized up to about 100 pieces of data. First, we can simply design iso8583, define 128 fields, and sort all financial data types that can be considered, such as the "Account" mentioned above, in one order, corresponds to one of the 128 fields. Each data type occupies a fixed length, which is defined in advance. This is simple. When you want to send a packet, connect the 128 fields in order and then send the entire string of packets.

After receiving the iso8583 package, any financial software can unpackage it directly according to our defined specifications, because the 128 fields of the entire message, from which one to which one represents what everyone knows, as long as you know that your data packet is an iso8583 package, we have already defined it. For example, the 1st fields are "transaction type", the length is 4 bits, the 2nd Field bits are "Account" and 19 BITs. The receiver can take 4 digits, 19 digits, and so on until the 128 fields of the entire data packet are completely parsed.

In fact, this approach is really simple and straightforward, basically enough to meet the needs. However, we have several questions to consider:
1. How do I know the Data Type of each field? Is it a number or a character?
2. Each packet is sent with 128 fields. The network bandwidth is sufficient. Sometimes I only need 5 fields, the result contains 123 useless fields.
3. What should I do if the length of some of my fields is not fixed? Because the unpackage is fixed as each field of the data packet, when C language is used for unpacking, a string of characters with a fixed length is taken as a field by the pointer.

Let's solve these problems one by one.

The first problem is simple. When I define iso8583, in addition to defining what each field represents, I also define its content as numbers or characters. The possible types include letters, numbers, special characters, years, months, days, and other time and binary data. For example, the length of the "Merchant type" field in the 128 fields is 15 and the type is a letter. What if the "Merchant type" contains numbers and letters at the same time? Then we can define the type as letter or number, that is, a field can belong to multiple types at the same time.

The second problem is a little complicated. In essence, if I only pass 5 fields of 128 fields, how does the receiver know which fields I have passed to it. If we fill in all the remaining 123 as 0 or other special identifiers, do we need to mark this field as unnecessary? This method is useless and does not solve the essential problem of network bandwidth. It is still necessary to transmit 128 fields.

In another way, I add a packet header in front of the packet. The information contained in the packet header can let others know that only five fields are passed. How can we design this header? In this way, we use 2 bytes, that is, 128 bits (one byte equals 8 bits) to indicate whether a field exists in the 128 fields. Each bit is either 1 or 0 in the binary of the computer. If it is 1, it indicates that the corresponding field exists in this message. If it is 0, it does not exist. In this case, if someone else receives the iso8583 packet, You can first identify the fields and fields of the packet following the header Based on the header. For example, I want to send five fields that belong to the 128, 3, 6, 8, and 9 fields of the 2nd fields, I can fill in the 011001011000000000-bit packet header ..........., A total of 128 bits, followed by 0. Note that 2nd, 3, 6, 8, and 9 are 1, and the others are 0.

With this-bit packet header, we can send only five required fields. How to organize packets? Put the 128bit header, that is, the header of two bytes, and then put the 2, 3, 6, 8, and 9 Fields Behind the header. These fields are placed together, you do not need to fill in the fields 4 and 5 between 3 and 6. When receiving the packet, the receiver unpacks the packet based on the 3rd-bit header. It naturally knows that after taking out the 3rd fields, the receiver directly retrieves 6th fields after the fields, the length of each field is defined in iso8583, and the data packet is easily parsed.

This is good. To solve the second problem above, we just added two bytes of data in the message, so we can easily solve it. We call these two bytes bit map, a bitmap is used to indicate whether a bit exists. However, let's make a little optimization. Considering that 128 fields are not required in most cases, half of the 64 fields may not be used up. So I can reduce the packet header from 128bit to 64bit, and only put the remaining 64bit into the packet when necessary. Is the packet length less than one byte?

this is a good idea. We put the most common 128 fields in iso8583 into the first 64 fields, so we can double the processing. In this way, I generally only need to send 64 bit packets, that is, the header of a byte, plus the required fields. What if some packets use fields between 64-128? In this case, I use the first bit of the 64bit packet header to represent a special meaning. If this bit is 1, it indicates that the 64bit is followed by the remaining 64bit packet header; if the first bit is 0, it indicates that there is no other 64-bit packet header after the 64-bit, which is directly the packet in the 128 fields. Then, the receiver will determine whether the first bit in the header is 1 or 0, so as to know whether the packet header is 64bit or 128bit, and then the corresponding processing can be done. Because the second 64bit of the packet header sometimes exists, we call it extended bit map extended bitmap. The first 64bit of the corresponding packet header is called the primary bit map primary bitmap. We put the extended bitmap directly to the first field of the 128 fields, and each data packet of the main bitmap is mandatory before all the 128 fields, it is not included in the 128 fields.

The third problem can be solved in this way. For example, if the 2nd fields are "accounts", they are not long. Some bank accounts may be 19 characters, and some may be 17 characters. When we set the iso8583 specification, we can specify that 2nd fields are 25 bits. This is enough to include both 19 and 17 bits. But what if there are 30 bits in the future? Now we set the field to 100 bits. What should we do if we have more than 100 bits in the future? Moreover, if you have only 19 BITs in account and we have defined 100 bits, the 81 bits in data will not waste the network bandwidth. It seems that it is not good to define a large number of digits in advance.

In this way, for the 2nd fields "Account", add the length of "Account" at the beginning of the field. For example, if the account number is 0123456789 and there are 10 digits in total, we will change it to 100123456789. Note that there are 10 more digits in front, indicating that the 10 digits below will be the account number. If you have been in contact with BSTR in COM, you should be familiar with this processing. After receiving this field, the receiver knows that iso8583 requires that the "Account" of the 2nd fields become longer. Therefore, the receiver obtains the value from the first two digits, which is the length, then, based on the length value, it is the real account to know which data is next to the field to be copied. If you think that only two characters in length can only represent 99 characters in length and are not enough, we can also define a variable-length field that allows the first three digits to be in length, so that there are 999 characters in length, it should be enough. In the specification, If I define the attribute of a field as "llvar", note that ll indicates the length, VAR indicates the subsequent data, and two ll indicates the length, the maximum value is 99. If it is three digits, it is "lllvar", and the maximum value is 999. In this way, when we look at the iso8583 specification document, we can simply understand the meaning of a variable length field based on these letters.

The several problems solved here have been solved. Let's review our self-designed iso8583 specifications. In fact, there is nothing more than dividing the data that may appear in the financial industry into different categories, arranging the order, and then connecting them to form a packet to be sent. Some optimizations were made to the design of the paper, and the bit map concept was introduced, which is also a good idea.

The rest of the work is simple. We will directly collect the possible data field types in the financial industry and divide them into 128 field types. If there are not so many fields, we will first save them, in addition, considering some people have special requirements, we stipulate that you can define the content of several fields in the 128 fields by yourself, which is also an extension.

In this way, we finally get the field description table of the iso8583 specification. If you want to know the meaning of each field in detail, you can directly view the table. This is relatively simple.