Distributed Computing (2) inter-process communication

Every day, we accumulate a little bit. One day, we will see more and more things, but we will understand distributed computing more deeply from the bottom layer.

What is inter-process communication? What is the relationship with distributed systems? How to define it? The heart is entangled, not in its place

1. Basic principles of inter-Communication

In network and distributed programming, inter-process communication (IPC: inter-process communication) is an important development method. Inter-process communication can be roughly divided into two categories: one is the inter-process communication method applied on the same node, mainly including pipelines, FIFO, signals, message pair columns and shared memory; the other is the inter-process communication method applied on different nodes. socket-based communication is a typical method.

Java can be understood as a JVM process, and inter-process communication relies on system calls. To implement inter-process communication in Java, you can use RMI or CORBA. In fact, Java corab is also implemented through RMI, and RMI is ultimately implemented using socket. Therefore, socket is the most basic means of inter-process communication in Java.

The inprocess routine implements inter-process communication by means of shared memory.

Simple diagram of inprocess communication between processes

Import Java. io. ioexception; import Java. io. randomaccessfile; import Java. NIO. mappedbytebuffer; import Java. NIO. channels. filechannel; import Java. NIO. channels. filelock; public class producer extends thread {private string mfilename; private filechannel mfilechannel; private mappedbytebuffer mmappedbytebuffer; Public producer (string FN) {try {mfilename = FN; // obtain a read/write Random Access File object randomaccessfile rafile = new randomaccessfile (mfilename, "RW"); // obtain the corresponding file channel mfilechannel = rafile. getchannel (); // sets the file size so that the image can be mapped to the shared memory int size = 10000; // gets the shared memory buffer, Which is readable by mmappedbytebuffer = mfilechannel. map (filechannel. mapmode. read_write, 0, size ). load ();} catch (ioexception ex) {system. out. println (Ex) ;}} public void run () {int I = 0; while (true) {try {filelock lock = NULL; lock = mfilechannel. trylock (); If (Lock = NULL) {system. err. println ("Producer: Lock failed"); continue;} mmappedbytebuffer. putint (0, ++ I); mmappedbytebuffer. putint (4, ++ I); mmappedbytebuffer. putint (8, ++ I); system. out. println ("Producer:" + (I-3) + ":" + (I-2) + ":" + (I-1); thread. sleep (200); lock. release (); thread. sleep (500);} catch (ioexception ex) {system. out. println (Ex);} catch (interruptedexception ex) {system. out. println (Ex) ;}} public static void main (string [] ARGs) {producer = new producer ("sharedmemory"); producer. run ();}}

This program defines a simple buffer implemented by the Java thread mechanism. This buffer zone obtains a random file object that can be read and written. The producer class uses the derivation of the thread class to create a new thread and directly creates an instance of the thread class. After the thread starts, the thread writes data to the producer thread in the buffer. The thread continuously puts a new data into the buffer at intervals.

Import Java. io. ioexception; import Java. io. randomaccessfile; import Java. NIO. mappedbytebuffer; import Java. NIO. channels. filechannel; import Java. NIO. channels. filelock; public class Consumer extends thread {private string mfilename; private filechannel mfilechannel; private mappedbytebuffer mmappedbytebuffer; public consumer (string FN) {try {mfilename = FN; // obtain a read/write Random Access File object randomaccessfile rafile = new randomaccessfile (mfilename, "R"); // obtain the corresponding file channel mfilechannel = rafile. getchannel (); // sets the file size so that the image can be mapped to the shared memory int size = 10000; // gets the shared memory buffer, Which is readable by mmappedbytebuffer = mfilechannel. map (filechannel. mapmode. read_only, 0, size ). load ();} catch (ioexception ex) {system. out. println (Ex) ;}} public void run () {While (true) {try {thread. sleep (300); filelock lock = NULL; lock = mfilechannel. trylock (0, 10, true); If (Lock = NULL) {system. err. println ("Consumer: Lock failed"); continue;} thread. sleep (1, 200); system. out. println ("Consumer:" + mmappedbytebuffer. getint (0) + ":" + mmappedbytebuffer. getint (4) + ":" + mmappedbytebuffer. getint (8); lock. release ();} catch (ioexception ex) {system. out. println (Ex);} catch (interruptedexception ex) {system. out. println (Ex) ;}} public static void main (string [] ARGs) {consumer = new consumer ("sharedmemory"); consumer. start ();}}

This program also uses the Java thread mechanism to implement a simple buffer. This buffer zone obtains a read/write Random Access File object, obtains the corresponding file channel, obtains the actual size of the file, and then maps it to the shared memory to obtain the shared memory buffer zone. The consumer class uses the derived class of the thread class to create a new thread and directly creates an instance of the thread class. After the thread starts, the thread reads data from the buffer. The thread continuously extracts data from the buffer and displays the data on the screen.

2. Interface and Interface Definition Language

The Interface Definition Language (IDL) is the language specification used to describe the interfaces of software components. IDL is described in a neutral language to enable communication between software components (written in different languages. IDL provides the ability to separate object interfaces from their implementations, and separates transactions from their specific implementations.

3. Data Representation and encoding

Inter-communication is the primary problem in distributed systems. In an object-oriented distributed system, inter-process communication usually involves message transmission between remote objects. The processes on both sides of the Communication maintain the message queue. The sending process first places the message in the sender's buffer zone and uses the communication channel to reach the receiving buffer zone of the target host. The receiving process extracts data from the buffer zone. To transmit abstract data types or objects on a binary stream-based network, the data must be encoded in a format that can be understood by dual-producer communication.

Data Representation standards for different abstraction layers

External Data Representation (XDR: external data representation) is an architecture-independent data representation method provided by sunsoft, this solution solves the differences in Data byte sorting, Data byte size, data representation, and data ing. XDR applications can exchange data on heterogeneous hardware systems.

Abstract Syntax mark ASN.1 (symbol at Syntax Notation One) is a data format used to represent, encode, transmit, and decode data at a higher abstraction layer. ASN.1 and Its Encoding Rules transmit structured data and describe the data structure in a way independent of the computer architecture and language.

XML: extensible markup language is also widely used at a higher abstraction level ). XML inherits the scalability of the General Markup Language Standard (SGML: Standard for General Markup Language) (users can customize new identifiers and attribute names as needed), structure (XML file structure nesting can be complex to any extent, can represent the object-oriented hierarchical) and verifiable (XML file can include a syntax description, so that the application can confirm the structure of the file ). Therefore, XML makes data exchange between distributed systems easier and allows data to be shared among application systems.

4. event synchronization mechanism

Because processes related to inter-process communication are executed independently, and processes do not know the execution status of the processes of the other party, the inter-process communication facilities usually adopt the event synchronization mechanism. The simplest way to synchronize events is to block the execution of a process until the execution of an operation initiated by the process ends.

Based on the Synchronization Methods between processes, you can divide inter-process communication methods into three types.

1) synchronous communication: The Releaser waits until the receiver returns the confirmation message.

2) asynchronous communication: the sender continues to execute the message after sending the message, and uses the idle time to round the receiver to check whether the request has been processed.

3) one-way communication: the sender continues to execute the message after sending the message and does not ask whether the request is processed completely.

 

Refer:

Distributed Computing tutorial

Zookeeper

Distributed Computing (2) inter-process communication