Deep understanding of streaming in Java---Combined with Hadoop for a detailed explanation __ streaming

In Javase's basic course, flow is a very important concept, and has been widely used in Hadoop, this blog will be focused on the flow of in-depth detailed.
A The related concepts of javase midstream
1, the definition of flow
① in Java, if a class is dedicated to data transfer, this class is called a stream
② flow is one of the channels used for data transmission since the grafting between programs and devices, this device can be a local hard disk, can be a memory bar, can also be the network associated with another computer, and so on, where different pipes have different buttons, press a different button is equivalent to calling different methods, The channel with the button for data transmission is the flow, that is, the flow is a pipe

The ③ stream must be a class, but a class is not necessarily a stream
2, the classification of the flow
According to the direction of the data flow, the input flow and the output stream are divided into the byte stream and the character stream according to the different processing data units, and the original flow and the parcel flow are divided according to the function.
Input and output streams: the input stream is the input pipeline that reads data from a specified device, such as a keyboard. The output stream is the output pipe that writes data to a specified device, such as a monitor.
Stream of bytes and character streams: the byte stream processes the data in units of a single character, in which a character streams the data is a char, which in Java is equivalent to two bytes.
Raw flow: A stream that reads and writes data from a particular device, a data source, like a single pipe to the faucet and starts to water.
Parcel Flow: The so-called package flow is because the original flow of the function is too simple, the original flow of a certain processing--on the basis of the original flow of a more powerful pipe, this pipe is called the flow of the package. The flow of a parcel is similar to a pipe on an already existing tube.
The key difference between the original flow and the parcel flow is the ability to connect directly to the device.
3, the four basic abstract flow
The so-called four basic abstract flow is the four basic abstract class, the four basic abstract flow including byte stream and character stream, where the byte stream includes InputStream and OutputStream, character stream including reader and writer, and usually the end of stream is byte flow.
The stream of bytes is exactly the same as the method of the stream of characters; InputStream is the parent class of all byte input streams, and the OutputStream is the parent of all byte output streams; Reader is the parent of all character input streams, Writer is the parent class for all character output streams.
4, IO package used in the flow
File stream (Raw stream):
FileInputStream FileReader
FileOutputStream FileWriter
Output stream (Raw stream):
PrintStream
Buffer flow (parcel flow):
Bufferedinputstream BufferedReader
Bufferedoutputstream BufferedWriter
Conversion flow (parcel flow):
InputStreamReader
OutputStreamWriter
Data flow (parcel flow):
DataInputStream
DataOutputStream
5, file flow, output flow, buffer flow, conversion flow, data flow related role
File flow: A file stream in Java can read, write, and copy the contents of a file in bytes or characters
Output streams: PrintStream provides enhanced functionality on a outputstream basis to output a string of basic type data lattices
Buffer Flow: Buffer flow is the input and output stream with buffer, the flow with buffer is faster than the flow without buffer, because this time is not read a write one, but read it after the buffer, in a one-time write to the specified device
Conversion flow: The role of the InputStreamReader is to convert the input byte stream into a character streams
The role of OutputStreamWriter is to convert the output byte stream into a character stream
Data flow: The data stream in Java can read the basic type data directly from the underlying byte input stream or write the basic type data directly into the byte output stream in a machine-independent way, that is, the data stream can read or write the binary of the basic type data directly.
6, briefly describe the difference between the stream of words and streams
1> byte stream can handle all the files in the format, but the text with Chinese characters through the output of the word stream will appear garbled phenomenon, but to complete the copy of the text file will not appear garbled phenomenon-that is, the byte stream can complete the copy of the text file
But the Ioutils tool class in Hadoop solves this flaw in the byte stream.
A 2> character stream can only work with text-formatted files, not text-formatted files, because non-text-formatted files themselves are not made up of one character, so it's definitely going to be an error to read as a character.
3> in the actual project the word throttling is widely used
Two Specific application of javase--a combination of APIs in Hadoop (ioutils)
Ioutils is a tool class provided by Hadoop, which is very convenient ——— import org.apache.hadoop.io.IOUtils in the process of programming, which will be illustrated with concrete examples.
The actual development process advocated 2 points:
1> try to deal with the problem with the word throttling
2> try to use the Org.apache.hadoop.io.IOUtils tool class in Hadoop to solve problems, because simple
Example 1: Read the contents of a file and output it to the monitor
Method ①

Package javase;

Import Java.io.BufferedReader;
Import Java.io.FileReader;

public class App1
{public
      static void Main (string[] args) throws Exception
     {
         BufferedReader fr = new Buff Eredreader (New FileReader ("C:\\file.txt"));
         int i = Fr.read ();
         while ( -1!=i)//-1 represents reading to the end of the file
         {
             System.out.print ((char) i);
             i = Fr.read ();
}} Run Result:/
*
Sometimes your plans don ' t work out because God has better.
Perseverance and patience in some way determine what a person will become.
 */

Method ②

Package javase;

Import Java.io.BufferedReader;
Import Java.io.FileReader;

public class App1
{public
      static void Main (string[] args) throws Exception
     {
         BufferedReader fr = new Buff Eredreader (New FileReader ("C:\\file.txt"));
         char[] buf = new char[1024];
         int len = Fr.read (BUF);//Read data from the file associated with Fr and store it in an array buf
         System.out.println (New String (Buf,0,len))
     ;
}
Run Result:/
*
Sometimes your plans don ' t work out because God has better.
Perseverance and patience in some way determine what a person will become.
 */

Methods ③ using Hadoop's Own Tools class: Ioutils (the word stream to resolve the text output to the display of Chinese characters in the garbled error), and compared to the Javase Io method is simpler

Package javase;

Import Java.io.FileInputStream;

Import Org.apache.hadoop.io.IOUtils;

public class App1
{public
      static void Main (string[] args) throws Exception
     {
          FileInputStream fr = new Fil Einputstream ("C:\\file.txt");
          Ioutils.copybytes (fr,system.out,1024,true);
     }
Run Result:/
*
Sometimes your plans don ' t work out because God has better.
Perseverance and patience in some way determine what a person will become.
 */

Example 2: Programmatic implementation of File replication-Copy the file.txt text file under C disk to the File2.txt text file under D disk
Method ①

Package javase;

Import Java.io.BufferedInputStream;
Import Java.io.BufferedOutputStream;
Import Java.io.FileInputStream;
Import Java.io.FileOutputStream;

public class App2
{public
      static void Main (string[] args) throws Exception
     {
           Bufferedinputstream fr = NE W Bufferedinputstream (New FileInputStream ("C:\\file.txt"));
           Bufferedoutputstream FW = new Bufferedoutputstream (New FileOutputStream ("D:\\file2.txt"));
           int i = Fr.read ();
           while ( -1!=i)
           {
               fw.write (i);
               i = Fr.read ();
           }
           Fw.flush ();
           Fr.close ();
           Fw.close ();
     }

Method ②

Package javase;


Import Java.io.BufferedInputStream;
Import Java.io.BufferedOutputStream;
Import Java.io.FileInputStream;
Import Java.io.FileOutputStream;


public class App2
{public
      static void Main (string[] args) throws Exception
     {
           Bufferedinputstream fr = NE W Bufferedinputstream (New FileInputStream ("C:\\file.txt"));
           Bufferedoutputstream FW = new Bufferedoutputstream (New FileOutputStream ("D:\\file2.txt"));
           byte[] buf = new byte[1024];
           int i = Fr.read (BUF);
           Fw.write (buf,0,i);
           Fw.flush ();
           Fw.close ();
           Fr.close ();
     }

Method ③ using Hadoop's Own tool class: Org.apache.hadoop.io.IOUtils, simpler than the IO method in Javase

Package javase;



Import Java.io.BufferedInputStream;
Import Java.io.FileInputStream;
Import Java.io.PrintStream;

Import Org.apache.hadoop.io.IOUtils;


public class App2
{public
      static void Main (string[] args) throws Exception
     {
           Bufferedinputstream fr = NE W Bufferedinputstream (New FileInputStream ("C:\\file.txt"));
           System.setout (The New PrintStream ("D:\\file2.txt"));//redirect
           ioutils.copybytes (fr,system.out,1024,true) for standard output streams;
     }
}

Example 3: Use the ReadLine () method in Bufferreader and the WriteLine () method in BufferedWriter to complete the copying of text files--often done in a project.

Package javase;



Import Java.io.BufferedReader;
Import Java.io.BufferedWriter;
Import Java.io.FileReader;
Import Java.io.FileWriter;


public class APP3
{public
      static void Main (string[] args) throws Exception
     {
           BufferedReader fr = new Buff Eredreader (New FileReader ("C:\\file.txt"));
           BufferedWriter FW = new BufferedWriter (New FileWriter ("D:\\file2.txt"));
           String str = fr.readline ();
           while (Str!=null)
           {
               fw.write (str);
               Fw.newline ();
               str = Fr.readline ();
           }
           Fw.flush ();
           Fr.close ();
           Fw.close ();
     }

Example 4: Programmatic implementation writes a long base type of data to a byte array and then reads the data from the byte array--a task that is often accomplished in socket programming
Method ①

Package javase;


Import Java.io.ByteArrayInputStream;
Import Java.io.ByteArrayOutputStream;
Import Java.io.DataInputStream;
Import Java.io.DataOutputStream;
Import Java.util.Scanner;


public class APP4
{public
      static void Main (string[] args) throws Exception
     {Scanner
           Scanner = new Scanne R (system.in);
           Long  i = Scanner.nextlong ();
           Bytearrayoutputstream array = new Bytearrayoutputstream ();//kernel is byte array
           dataoutputstream FW = new DataOutputStream ( Array);
           Fw.writelong (i);//writes the long integer variable i to the byte array
           byte[] ByteArray = Array.tobytearray ()//Copy the contents
           of the array kernel byte arrays Bytearrayinputstream ByteArray = new Bytearrayinputstream (ByteArray);
           DataInputStream FR = new DataInputStream (ByteArray);
           Long j = Fr.readlong ();
           System.out.println (j);
     }
Run Result:/*
 *

Method ②

Package javase;

Import Java.util.Scanner;

public class APP4
{public
      static void Main (string[] args) throws Exception
     {Scanner
           Scanner = new Scanne R (system.in);
           Long  i = Scanner.nextlong ();
           String str = i + "";
           byte[] bytes = str.getbytes ()//To save the long integer variable i indirectly to the array
           System.out.println (new String (bytes));
     }
Run Result:/*
 *

Note: This example is not suitable for Hadoop's own tool class: Org.apache.hadoop.io.IOUtils to Solve
Example 5: Using Data flow to complete a basic type of data serialization and deserialization

package javase;
Import Java.io.DataInputStream;
Import Java.io.DataOutputStream;
Import Java.io.FileInputStream;
Import Java.io.FileOutputStream;


Import Java.util.Scanner; public class App5 {public static void main (string[] args) throws Exception {Scanner Scanner = new
           Scanner (system.in);
           int i = Scanner.nextint ();

           Int J = Scanner.nextint ();
           DataOutputStream FW = new DataOutputStream (New FileOutputStream ("C:\\file2.txt"));
           Fw.writeint (i);

           Fw.writeint (j);
           DataInputStream FR = new DataInputStream (New FileInputStream ("C:\\file2.txt"));
           int i2 = Fr.readint ();

           int j2 = Fr.readint ();
           System.out.println (i);
     System.out.println (j); }//Run Result://///

Considerations for serialization and deserialization:
1> the so-called serialization is to write the object in binary form into the byte output stream, the so-called deserialization is the object from the byte input stream read out
The order of 2> serialization and deserialization should be the same: which data is first written into the stream pipeline and which data is first read from the stream pipe
The data stream in 3>java writes the binary code of the base type data into a text file, while the output stream in Java PrintStream writes a string of the base type data to a text file/FONT>
The following is the sample result:
When we open the File2.txt file in the above program, the result is as follows:

which
00 00 00 41 65 for 16 in 00 00 00 for 38 56
Example 5: Programmatic implementation assigns a keyboard input character string directly to a string object
Method ①

Package javase;

Import Java.io.BufferedReader;
Import Java.io.InputStreamReader;

public class APP5
{public
      static void Main (string[] args) throws Exception
     {
            BufferedReader fr = new Buff Eredreader (New InputStreamReader (system.in));//Converts the keyboard input bytes into character stream
            string str = Fr.readline ();
            System.out.println (str);
     }
Run Result:/
*
data analysis player Shujufenxiwanjia
data analysis player Shujufenxiwanjia
* *

Method ②

Package javase;

Import Java.util.Scanner;



public class APP5
{public
      static void Main (string[] args) throws Exception
     {Scanner
             Scanner = new Scanne R (system.in);
             String str = scanner.nextline ();
             System.out.println (str);

     }
Run Result:/
*
data analysis player Shujufenxiwanjia
data analysis player Shujufenxiwanjia
* *

Example 6: Redirection of standard input and output: programmatic implementation to enter the input data into the File1.txt file, and if the input is incorrect, output the error message to the B file
The Knowledge points used:
1> related APIs are shown in the following illustration:


The E.printstacktrace () method in the 2>java exception is to export the error message to the device associated with System.err

package javase;
Import Java.io.PrintStream;

Import Java.util.Scanner; public class App6 {public static void main (string[] args) throws Exception {system.setout (new Pr Intstream ("C:\\file1.txt"))//redirect Output System.seterr (new PrintStream ("C:\\file2.txt"));/redirect Error output wh
                 Ile (True) {Scanner Scanner = new Scanner (system.in);
                     try {int i = Scanner.nextint ();
                 System.out.println (i);
                 catch (Exception e) {e.printstacktrace (); } 
             }

     }
}

After the operation is complete:
The contents of the File1.txt text file are:
23
56
The contents of the File2.txt text file are:
Java.util.InputMismatchException
At Java.util.Scanner.throwFor (scanner.java:864)
At Java.util.Scanner.next (scanner.java:1485)
At Java.util.Scanner.nextInt (scanner.java:2117)
At Java.util.Scanner.nextInt (scanner.java:2076)
At JavaSe.App1.main (app1.java:19)

For the Javase basic programming in the flow of usage is written here, if there is a problem, please comment.