Message ISO8583 Protocol

I just contact the financial IT industry, the message ISO8583 agreement is just understand, read an article, personally feel well written, hereby share the following:

If simply speaking IS08583 those field definition, I think there is no meaning, the standard has been explained to each field very detailed, if you feel that understanding the English version of the ISO8583 specification is a bit difficult, online also has a peer for our translation of the Chinese version of the ISO8583 specification, So my goal is to reach the reading of this article can be known to ISO8583, but also know the reason why, so that the basic non-contact with it before the people can also achieve the mastery of ISO8583 message specification.



Well, it's time for us to turn to the chase.



At first, the financial system was only IBM's big companies to provide equipment, such as various hosts and terminals. There is a need to exchange data between individual computer devices. We know that the data is transmitted over the network, and that the data that is transmitted over the network is based on binary data of 0 or 1, and if the data is not encoded, then no one can understand that data is useless. The initial x., SDLC, and now popular TCP/IP network protocols all provide the underlying communication coding protocol, which solves the most basic communication problem and is capable of transmitting a string of characters from one place to another. However, it is not significant to simply transfer strings, it is very important to parse what the string represents, otherwise it is useless to send some "0123ABCD" strings.



Let's go back to some time a few decades ago, assuming we were pushed to the stage of history, we designed a generic message protocol to address the exchange of messages between financial systems, called the ISO8583 protocol. At this time, the technology is in the continuous forward, the original IBM a single show situation seems to have been bad, various sizes of companies have entered the financial industry in order to be able to gain, a piece of blossoming situation. How we design a message protocol that can incorporate all of these springing up companies is not really easy.



Let's take a step at it. The financial industry actually involves the data content is not thousands, cannot statistic, on the contrary, is relatively few. We can count in the bottom of our hearts, such as the type of transaction, account number, account type, password, transaction amount, transaction fee, date and time, merchant code, 2 Magnetic 3 magnetic data, transaction serial number, etc., summed up all can be summed up only about 100 data. Then we can first simple design ISO8583, define 128 fields, all can be considered similar to the above mentioned "account number" and other financial data types, in one order, corresponding to 128 fields in one field. Each data type occupies a fixed length, and this order and length are defined beforehand. This is simple, to send a message, the 128 fields are connected in sequence, and then the entire string of data packets sent out.



Any financial software received ISO8583 package, directly according to our definition of the specification to unpack, because the entire message of the 128 fields from which to which a representative what, we all know, just know that your packet is ISO8583 package, we have already defined. For example, the 1th field is "transaction type", the length is 4 bits, the 2nd is "account number", 19 bits and so on. The receiver can take 4 bits first, then take the next 19 bits, and so on, until the entire 128 fields of the packet have been solved.



In fact, this kind of practice is very simple and direct, basically can satisfy the need. But we have a few questions to think about:

1. How do I know the data type of each field, a number or a character?

2, each transmitted message all 128 fields are passed, the network bandwidth can withstand, and sometimes I may only need 5 of the fields, the results received more than 123 useless fields.

3, if the length of some of my fields is not fixed, is a variable length of what to do, because you are now unpacking as a packet of each field is fixed, the C language when unpacking directly rely on the pointer to a fixed length of a string of characters as a field.



We come to one by one to solve these problems.



The first question is simple, I define ISO8583 in addition to defining what each field represents, but also specify that the content is a number or character, etc. Consider the types of possible occurrences, but there are several: letters, numbers, special characters, date and time, and binary data. For example, I define the "Merchant type" field in 128 fields with a length of 15, and define its type as a letter. Fine point, if the "merchant type" inside the data include both numbers and letters? Then we can define its type as a letter, or as a number, that is, a field can belong to more than one type at a time.



The second question is a little more complicated. The essence is that if I only pass 5 fields of 128 fields, how does the receiver know which fields I have passed to it? What if we fill out the rest of the 123 in 0 or other special identifiers, indicating that the field does not need to be used? There is no use in this approach, there is no solution to the intrinsic problem of network bandwidth, or 128 fields.



Another idea, I put a packet in front of Baotou, Baotou contains information to let others know that only 5 fields passed. How to design this Baotou, so, we use 16 bytes, that is, 128 bit (one byte equals 8bit) to represent the existence of a field in 128 fields. Each bit in the computer's binary is not 1 is 0, if 1 means that the corresponding field is present in this message, if 0 is not present. In this case, if someone else receives the ISO8583 message, it is possible to start with the first message header and then know what fields are in the message immediately after the message header and what fields are not available. For example, I would like to send 5 fields, which belong to the 2nd, 3, 6, 8, 9 fields in 128 fields, and I can fill the header of the 128bit headers into 011001011000000000 ..., 128 bits in total, and 0 at the back. Note that 2nd, 3, 6, 8, 9 bits are 1, and the others are 0.



With this 128bit header, we can send only the required 5 fields. How to organize messages? First put the 128bit, that is, 16 bytes of the head, and then put 2, 3, 6, 8, 9 fields on the back of the head, these fields close together, 3 and 6 do not need to fill in the two fields 4, 5. The receiver receives this message, it will be based on the packet of 128bit packets, it is natural to know that the 3rd field is removed, directly after the 3rd field to take the 6th field, the length of each field is defined in ISO8583, it is easy to solve the packet.



Well, in order to solve the second problem above, we just added 16 bytes of data in the message, it's easy to fix, we call these 16 bytes bit
Map, the ascended image, used to indicate whether a bit exists. But let's just tweak it a little bit, considering that a lot of times the message doesn't need 128 fields so much, and half of its 64 fields don't necessarily run out. Then I can reduce the header from 128bit to 64bit, only when needed to put the remaining 64bit into the message, so that the length of the message is not less than 8 bytes?



It's a good idea. We put the most common of the 128 fields in the ISO8583 into the first 64 fields, so we can reduce the processing by a factor of two. So I usually send a message only to send 64bit, that is, a byte of the message header, plus the required number of fields to be able. What if some messages use a field between 64 and 128? This is good, I use the first bit of the 64bit message header to represent a special meaning, if the bit is 1, it means that 64bit followed by the remaining 64bit headers; If the first bit is 0, then 64bit is not followed by the remaining 64bit headers, is directly the message in the 128 fields. That, the receiver will determine the first bit of the header is 1 or 0, so that the message header is 64bit or 128bit, you can do the corresponding processing. Because the second 64bit of the message header is sometimes there, so we call it extended.
Bit map extension bitmap, the corresponding message header at the beginning of the 64bit we call it primary bit map bitmap. We fixed the extended bitmap directly into the first field of the 128 fields, and each packet of the bitmap had a mandatory place in front of all 128 fields, and not into 128 fields.



The third question can be considered for such a solution. For example, the 2nd field is "Account", is not long, there may be a bank account number is 19, there are 17-bit and so on. When we set ISO8583 specification, we can stipulate that the 2nd field is 25 bits, which is enough to include 19 and 17 cases, but what if there are 30 bits in the future? Then we will now set the field to 100 bits. After more than 100 people what to do, and if you only have 19 account, we define 100 bits, that 81 bits of data is not a waste of network bandwidth. It seems that it is not very good to define a predetermined number of digits that we consider to be larger.



In this case, for the 2nd field "Account", add the length of "account" at the beginning of the field. For example, the account number is 0123456789, altogether 10, we become 100123456789, pay attention to the front more than 10, indicating that the following 10 digits for the account. If you have been exposed to a BSTR inside COM, you should be familiar with this process. When the receiver receives the field, it knows that ISO8583 specifies that the 2nd field "account" is longer, so it takes the first 2 bits out of it, gets its value, is the length, and then, based on the length value, knows which data should be copied after the field, which is the real account. If you think the length if only two bits can only represent 99 bits long, not enough, we also define can allow the front 3 bits are length of the variable length field, so there are 999 bits long, should be enough. In the specification, if I define a field that has a property of "Llvar", you notice that the LL represents the length, VAR indicates the following data, two ll represents two bits long, the maximum is 99, if three bits is "Lllvar", the maximum is 999. This way we can see the meaning of a variable-length field directly based on these letters when we define the ISO8583 specification document.



Some of the problems solved here, we have to review their own design of the ISO8583 specifications. In fact, there is nothing, nothing more than the financial industry may appear in the breakdown of data, order, and then connect them together to form a message sent out. Some optimizations are made for the design of the message, and a bit is introduced.
The concept of map bitmap is also a good idea.



The rest of the work is simple, we will directly collect the financial industry may appear in the data field type, divided into 128 field types, if not to 128 so much to keep some of the first, and considering that some people have special requirements, we have a few fields in 128 fields you can define their own content, It's kind of an extension.



So, in the end we get the field description table for the ISO8583 specification. It is easy to know the meaning of each field in detail and look directly at the table.

Message ISO8583 Protocol