Java pipeline stream

Java pipeline stream

Even if the graphic user interface is dominant today, console output still plays an important role in Java programs. The console is not only the default stack trace and error message output window for Java programs, but also a practical debugging tool (especially for those who are used to using println ). However, console windows have many limitations. For example, on Windows 9x, the DOS console can only accommodate 50 rows of output. If a Java program outputs a large amount of content to the console at a time, it is very difficult to view the content.

The console window is especially valuable for developers who use the javaw Startup Program. Because when you start a Java program using Java W, there is no console window. If the program encounters a problem and throws an exception, you cannot view the call stack trace information written to system. out or system. err in the Java Runtime Environment. To capture stack information, some people use try/catch () block to encapsulate main (), but this method is not always effective. At some time points during Java runtime, some descriptive error messages will be written to system before an exception is thrown. out and system. err; this information is invisible unless the two console streams can be monitored.

Therefore, it is necessary to check the data written to the console stream in the Java Runtime Environment (or a third-party Program) and take appropriate operations. One of the topics discussed in this article is to create an input stream from which you can read data previously written into the Java console stream (or any other program's output stream. We can imagine that the data written to the output stream is immediately "reflux" to the Java program in the form of input.

The goal of this article is to design a swing-based text window display console output. During this period, we will also discuss important considerations related to Java pipeline streams (pipedinputstream and pipedoutputstream. Figure 1 shows the Java program used to intercept and Display Console text output. The core of the user interface is a jtextarea. Finally, we need to create a simple program that can capture and display other programs (which can be non-Java programs) output on the console.

Figure 1: multi-threaded console output interceptor

1. Java pipeline stream

To display console output in the text box, we must "intercept" the console stream in some way. In other words, we need a method to efficiently read all the content written to system. Out and system. Err. If you are familiar with the Java pipeline stream pipedinputstream and pipedoutputstream, we believe that we already have the most effective tool.

Data written to the pipedoutputstream output stream can be read from the corresponding pipedinputstream input stream. The Java pipeline stream greatly facilitates the capture of console output. Listing 1 shows a very simple screenshot console output solution.

[Listing 1: Console output with pipeline stream capture] pipedinputstream pipedis = new pipedinputstream (); pipedoutputstream pipedos = new pipedoutputstream (); try {pipedos. connect (pipedis);} catch (ioexception e) {system. err. println ("connection failed"); system. exit (1);} printstream PS = new printstream (pipedos); system. setout (PS); system. seterr (PS );

As you can see, the code here is extremely simple. We just created a pipedinputstream and set it as the final destination of all data written to the console stream. All data written to the console stream is transferred to pipedoutputstream. In this way, data written to the console stream can be quickly intercepted by reading from the corresponding pipedinputstream. The next thing seems to be that only the data read from the pipedis stream is displayed in swing jtextarea to get a program that can display the console output in the text box. Unfortunately, there are some important notes when using Java pipeline streams. Listing can be guaranteed only when all these precautions are taken seriously.
1. The code runs stably. Next let's take a look at the first note.

1.1 Note 1
Pipedinputstream uses a circular buffer with a fixed size of 1024 bytes. The data written to pipedoutputstream is actually saved to the internal buffer of the corresponding pipedinputstream. When a read operation is performed from pipedinputstream, the data read actually comes from this internal buffer zone. If the corresponding pipedinputstream input buffer is full, any thread attempting to write pipedoutputstream will be blocked. The write operation thread will be blocked until the operation to read pipedinputstream is executed to delete data from the buffer zone.

This means that the thread writing data to pipedoutputstream should not be the only thread responsible for reading data from the corresponding pipedinputstream. As shown in figure 2, we can clearly see the problem here: Assume that thread t is the only thread responsible for reading data from pipedinputstream; in addition, assume that t attempts to write () to pipedoutputstream at a time () the method is called to write 2000 bytes of data to the corresponding pipedoutputstream. Before the T thread is congested, it can write up to 1024 bytes of data (the size of the internal buffer of pipedinputstream ). However, once t is blocked, read
Pipedinputstream operations will no longer appear, because T is the only thread to read pipedinputstream. In this way, the T thread is completely blocked. At the same time, all other threads trying to write data to pipedoutputstream will also encounter the same situation.

Figure 2: Pipeline Workflow

This does not mean that more than 1024 bytes of data cannot be written in a write () call. However, it should be ensured that while writing data, another thread reads data from pipedinputstream.

Listing 2 demonstrates this problem. This program uses a thread to alternately read pipedinputstream and write pipedoutputstream. Each call to write () writes 20 bytes to the buffer zone of pipedinputstream. Each call to read () only reads and deletes 10 bytes from the buffer zone. The internal buffer is eventually full, leading to write blocking. Since we use the same thread to perform read and write operations, once the write operation is blocked, we cannot read data from pipedinputstream.

[Listing 2: using the same thread to execute read/write operations causes thread blocking] import Java. io. *; public class listing2 {static pipedinputstream pipedis = new pipedinputstream (); static pipedoutputstream pipedos = new pipedoutputstream (); public static void main (string [] A) {try {pipedis. connect (pipedos);} catch (ioexception e) {system. err. println ("connection failed"); system. exit (1);} byte [] inarray = new byte [10]; byte [] outarray = new byte [20]; int byte Sread = 0; try {// send 20 bytes of data pipedos to pipedos. write (outarray, 0, 20); system. out. println ("20 bytes sent... "); // read 10 bytes in each loop iteration // send 20 bytes bytesread = pipedis. read (inarray, 0, 10); int I = 0; while (bytesread! =-1) {pipedos. write (outarray, 0, 20); system. out. println ("20 bytes sent... "+ I); I ++; bytesread = pipedis. read (inarray, 0, 10) ;}} catch (ioexception e) {system. err. println ("error when reading pipedis:" + E); system. exit (1) ;}// main ()}

As long as the read/write operations are separated to different threads, The Listing 2 problem can be easily solved. Listing 3 is the modified version of Listing 2. It writes pipedoutputstream in a separate thread (different from the reading thread ). To prove that the data written at a time can exceed 1024 bytes, the write operation thread writes 2000 bytes each time it calls the pipedoutputstream's write () method. Is the thread created in the startwriterthread () method blocked? According to the thread scheduling mechanism during Java runtime, it will certainly be blocked. Before blocking, write operations can write a maximum of 1024 bytes of payload (the size of the pipedinputstream buffer ). But this will not be a problem, because the main thread will soon
The circular buffer of pipedinputstream reads data, leaving the buffer space empty. In the end, the write operation thread starts from the last stop and writes the remaining part of the 2000-byte payload.

[Listing 3: Separate read/write operations to different threads] import Java. io. *; public class listing3 {static pipedinputstream pipedis = new pipedinputstream (); static pipedoutputstream pipedos = new pipedoutputstream (); public static void main (string [] ARGs) {try {pipedis. connect (pipedos);} catch (ioexception e) {system. err. println ("connection failed"); system. exit (1);} byte [] inarray = new byte [10]; int bytesread = 0; // start the write operation thread startwriter Thread (); try {bytesread = pipedis. Read (inarray, 0, 10); While (bytesread! =-1) {system. out. println ("read" + bytesread + "bytes... "); bytesread = pipedis. read (inarray, 0, 10) ;}} catch (ioexception e) {system. err. println ("An error occurred while reading the input. "); system. exit (1) ;}// main () // create an independent thread // execute the operation to write pipedoutputstream Private Static void startwriterthread () {New thread (New runnable () {public void run () {byte [] outarray = new byte [2000]; while (true) {// loop without termination conditions try {// before the thread is blocked, A maximum of 1024 bytes of data are written to pipedos. write (outarray, 0, 2000);} catch (ioexception e) {system. err. println ("write operation error"); system. exit (1);} system. out. println ("2000 bytes have been sent... ");}}}). start ();} // startwriterthread ()} // listing3

Maybe we cannot say that this problem is a defect in Java pipeline stream design, but it must be paid close attention to when applying pipeline stream. Next let's take a look at the second more important (more dangerous) problem.

1.2 Note 2
When reading data from pipedinputstream, if the following three conditions are met, an ioexception occurs:

1. Trying to read data from pipedinputstream,
2. The buffer zone of pipedinputstream is "null" (that is, there is no readable data ),
3. The last thread that writes data to pipedoutputstream is no longer active (detected by thread. isalive ).
   

This is a very delicate moment and an extremely important moment. Assume that one thread W writes data to pipedoutputstream, And the other thread R reads data from the corresponding pipedinputstream. The following events cause the r thread to encounter an ioexception when attempting to read pipedinputstream:

1. W writes data to pipedoutputstream.
2. W (W. isalive () returns false ).
3. R reads data written by W from pipedinputstream and clears the buffer zone of pipedinputstream.
   
4. R tries to read data from pipedinputstream again. At this time, the buffer of pipedinputstream is empty and W is over, which leads to an ioexception during read operation execution.
   

It is not difficult to construct a program to demonstrate this problem. You only need to delete the while (true) condition from the startwriterthread () method of listing 3. This change prevents the write operation method from being executed cyclically, so that the write operation execution method ends after a write operation. As mentioned above, when the main thread tries to read pipedinputstraem, it will encounter an ioexception.

This is a rare case and there is no direct correction method. Please do not use the method of generating a Child class from the pipeline genre-it is totally inappropriate to use inheritance here. In addition, if sun later changes the pipeline stream implementation method, the modifications made now will not be valid.

The last problem is very similar to the second one. The difference is that it generates an ioexception when the reading thread ends (rather than the writing thread.

1.3 Note 3
If a write operation is executed on pipedoutputstream and the thread recently read from the corresponding pipedinputstream is no longer active (detected by thread. isalive (), the write operation throws an ioexception. Assume that there are two threads W and R, W writes data to pipedoutputstream, and r reads data from the corresponding pipedinputstream. The following series of events will cause the W thread to encounter an ioexception when attempting to write pipedoutputstream:

1. The write operation thread W has been created, but the r thread does not exist.
2. W writes data to pipedoutputstream.
3. The read thread R is created and reads data from pipedinputstream.
4. The r thread ends.
5. W attempted to write data to pipedoutputstream and found that R has ended, throwing an ioexception.
   

In fact, this problem is not as tricky as the second one. Compared with multiple read threads/single write threads, it may be more common to have one read thread (as the server responding to the request) and multiple write threads (sending requests) in the application.

1.4 solve the problem
To prevent problems caused by the first two limitations of the pipeline stream, one method is to use a bytearrayoutputstream as a proxy or replace pipedoutputstream. Listing 4 shows a loopedstreams class. It uses a bytearrayoutputstream to provide similar functions as a Java pipeline stream, but does not encounter deadlocks or ioexception exceptions. This class still uses pipeline streams internally, but isolates the first two problems described in this article. Let's take a look at the public methods of this class (see figure 3 ). The constructor is simple. It connects to the pipeline stream and then calls the startbytearrayreaderthread () method (this method will be discussed later ). The getoutputstream () method returns an outputstream (specifically, a bytearrayoutputstream) to replace pipedoutputstream. Write to this
Data of outputstream will eventually appear as input in the stream returned by the getinputstream () method. Unlike pipedoutputstream, activation, data writing, and termination of the thread that writes data to bytearrayoutputstream do not bring negative effects.

Figure 3: bytearrayoutputstream Principle

[Listing 4: Preventing common problems in pipeline stream applications] import Java. io. *; public class loopedstreams {private pipedoutputstream pipedos = new pipedoutputstream (); Private Boolean keeprunning = true; private bytes bytearrayos = new topology () {public void close () {keeprunning = false; try {super. close (); pipedos. close ();} catch (ioexception e) {// record errors or other processing // This is a simple calculation. Here we end system directly. exit (1) ;}}; private pipedinputstream pipedis = new pipedinputstream () {public void close () {keeprunning = false; try {super. close ();} catch (ioexception e) {// record errors or other processing // This is a simple calculation. Here we end system directly. exit (1) ;}}; public loopedstreams () throws ioexception {pipedos. connect (pipedis); extract () ;}// loopedstreams () Public inputstream getinputstream () {return pipedis ;}// getinputstream () Public outputstream getoutputstream () {return bytearrayos ;} // getoutputstream () Private void startbytearrayreaderthread () {New thread (New runnable () {public void run () {While (keeprunning) {// check the number of bytes in the stream if (bytearrayos. size ()> 0) {byte [] buffer = NULL; synchronized (bytearrayos) {buffer = bytearrayos. tobytearray (); bytearrayos. reset (); // clear the buffer} Try {// send the extracted data to pipedoutputstream pipedos. write (buffer, 0, buffer. length);} catch (ioexception e) {// record errors or other processing // is a simple calculation. Here we end system directly. exit (1) ;}} else // No data is available. The thread enters the sleep state. Try {// check bytearrayoutputstream every 1 second to check the new data thread. sleep (1000);} catch (interruptedexception e ){}}}}). start ();} // startbytearrayreaderthread ()} // loopedstreams

The startbytearrayreaderthread () method is the real key of the entire class. The goal of this method is to create a thread that regularly checks the bytearrayoutputstream buffer. All data found in the buffer is extracted to a byte array and then written to pipedoutputstream. Because the pipedinputstream corresponding to pipedoutputstream is returned by getinputstream (), the threads that read data from the input stream will read the data originally sent to bytearrayoutputstream. As mentioned above, the loopedstreams class solves the first two problems in the pipeline stream. Let's take a look at how this works.

Bytearrayoutputstream can expand its internal buffer as needed. Because there is a "full buffer", the thread will not be blocked when writing data to the stream returned by getoutputstream. Therefore, the first problem will not cause us any more trouble. In addition, bytearrayoutputstream's buffer will never be reduced. For example, if a piece of 500 k Data is written to the stream before data can be extracted, the buffer will always maintain a capacity of at least 500 K. If this class has a method that can modify the buffer size after the data is extracted, it will be improved.

The second problem is solved because only one thread writes data to pipedoutputstream at any time. This thread is created by startbytearrayreaderthread. Because this thread is completely controlled by the loopedstreams class, we don't have to worry about it will generate an ioexception.

The loopedstreams class has some details worth mentioning. First, we can see that bytearrayos and pipedis are actually instances of the derived classes of bytearrayoutputstream and pipedinputstream, that is, they add special actions in their close () method. If the user of a loopedstreams object disables the input or output stream, the thread created in startbytearrayreaderthread () must be disabled. The overwrite close () method sets the keeprunning flag to false to close the thread. In addition, please note that startbytearrayreaderthread
. Ensure that the bytearrayoutputstream buffer between the tobytearray () call and the reset () call is not modified by the thread that writes the stream. This is essential. Because all versions of the write () method of bytearrayoutputstream are synchronized on this stream, we ensure that the internal buffer of bytearrayoutputstream is not accidentally modified.

Note that the loopedstreams class does not involve the third issue of MPs queue streams. The getinputstream () method of this class returns pipedinputstream. If a thread reads data from this stream and terminates after a period of time, the ioexception will occur the next time data is transmitted from the bytearrayoutputstream buffer to pipedoutputstream.

Back to Top

Ii. Capture Java console output

The consoletextarea class of Listing 5 extends swing jtextarea to capture console output. Don't be surprised that this class has so much code. It must be noted that the leletextarea class has more than 50% of the Code for testing.

[Listing 5: intercepting Java console output] import Java. io. *; import Java. util. *; import javax. swing. *; import javax. swing. text. *; public class consoletextarea extends jtextarea {public consoletextarea (inputstream [] instreams) {for (INT I = 0; I <instreams. length; ++ I) startconsolereaderthread (instreams [I]);} // leletextarea () Public consoletextarea () throws ioexception {final loopedstreams ls = new loopedstreams (); // Redirect system. out and system. err printstream PS = new printstream (LS. getoutputstream (); system. setout (PS); system. seterr (PS); startconsolereaderthread (LS. getinputstream ();} // leletextarea () Private void Merge (inputstream instream) {final bufferedreader BR = new bufferedreader (New inputstreamreader (instream); new thread (New runnable () {public void run () {stringbuffer sb = New stringbuffer (); try {string s; document DOC = getdocument (); While (S = Br. Readline ())! = NULL) {Boolean caretatend = false; caretatend = getcaretposition () = Doc. getlength ()? True: false; sb. setlength (0); append (sb. append (s ). append ('\ n '). tostring (); If (caretatend) setcaretposition (Doc. getlength () ;}} catch (ioexception e) {joptionpane. showmessagedialog (null, "read error from bufferedreader:" + E); system. exit (1 );}}}). start () ;}// startconsolereaderthread () // test the public static void main (string [] ARGs) function) {jframe F = new jframe ("leletextarea test"); consoletextare A consoletextarea = NULL; try {consoletextarea = new consoletextarea ();} catch (ioexception e) {system. err. println ("loopedstreams cannot be created:" + E); system. exit (1);} leletextarea. setfont (Java. AWT. font. decode ("monospaced"); F. getcontentpane (). add (New jscrollpane (consoletextarea), Java. AWT. borderlayout. center); F. setbounds (50, 50,300,300); F. setvisible (true); F. addwindowlistener (New Java. AWT. even T. windowadapter () {public void windowclosing (Java. AWT. event. invalid wevent EVT) {system. exit (0) ;}}); // start several write operation threads to // system. out and system. err outputs startwritertestthread ("write operation thread #1", system. err, 920, 50); startwritertestthread ("write operation thread #2", system. out, 500, 50); startwritertestthread ("write operation thread #3", system. out, 200, 50); startwritertestthread ("write operation thread #4", system. out, 1000, 50); startwritertestthread ("write operation Thread #5 ", system. err, 850, 50);} // main () Private Static void startwritertestthread (final string name, final printstream ps, final int delay, final int count) {New thread (New runnable () {public void run () {for (INT I = 1; I <= count; ++ I) {ps. println ("***" + name + ", hello !, I = "+ I); try {thread. sleep (Delay);} catch (interruptedexception e ){}}}}). start () ;}// startwritertestthread ()} // leletextarea

The main () method creates a jframe. The jframe contains an consoletextarea instance. There is nothing special about the code. After the frame is displayed, the main () method starts a series of write operation threads, and the write operation thread outputs a large amount of information to the console stream. Leletextarea captures and displays this information, as shown in 1.

Consoletextarea provides two constructors. The constructor without parameters is used to capture and display all data written to the console stream. An inputstream [] constructor forwards all data read from each array element to jtextarea. An example will be provided later to show the usefulness of this constructor. First, let's take a look at the leletextarea constructor without parameters. This function first creates a loopedstreams object, and requests the Java Runtime Environment to forward the console output to the outputstream provided by loopedstreams. Finally, the constructor calls startconsolereaderthread (), create a thread that constantly appends text lines to jtextarea. Note: After the text is appended to jtextarea, the program carefully ensures the correct position of the insertion point.

In general, the update of swing parts should not be performed outside the AWT event dispatch thread (aedt. In this example, this means that all operations to append text to jtextarea should be performed in aedt, rather than in the thread created by startconsolereaderthread () method. However, in fact, appending text to jtextarea in swing is a thread-safe operation. After reading a line of text, you only need to call jtext. append () to append the text to the end of jtextarea.