Java uses ByteArrayOutputStream and ByteArrayInputStream to avoid repeated reading of the configuration file. bytearrayinputstream

Java uses ByteArrayOutputStream and ByteArrayInputStream to avoid repeated reading of the configuration file. bytearrayinputstream

Recently, Mycat, an open-source project on github, is a mysql database/table sharding middleware. I found that the code used to read the configuration file was frequently opened, read, and closed multiple times. The code was very preliminary and I have read some framework source code, it will not make such a mistake. So it was optimized.

The code before optimization is as follows:

    private static Element loadRoot() {        InputStream dtd = null;        InputStream xml = null;        Element root = null;        try {            dtd = ConfigFactory.class.getResourceAsStream("/mycat.dtd");            xml = ConfigFactory.class.getResourceAsStream("/mycat.xml");            root = ConfigUtil.getDocument(dtd, xml).getDocumentElement();        } catch (ConfigException e) {            throw e;        } catch (Exception e) {            throw new ConfigException(e);        } finally {            if (dtd != null) {                try {                    dtd.close();                } catch (IOException e) { }            }            if (xml != null) {                try {                    xml.close();                } catch (IOException e) { }            }        }        return root;    }

LoadRoot () is frequently called by other methods ():

    @Override    public UserConfig getUserConfig(String user) {    Element root = loadRoot();        loadUsers(root);        return this.users.get(user);    }    @Override    public Map<String, UserConfig> getUserConfigs() {    Element root = loadRoot();        loadUsers(root);        return users;    }    @Override    public SystemConfig getSystemConfig() {    Element root = loadRoot();        loadSystem(root);        return system;    }    // ... ...

The ConfigUtil. getDocument (dtd, xml) method is as follows:

    public static Document getDocument(final InputStream dtd, InputStream xml) throws ParserConfigurationException,            SAXException, IOException {        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();//factory.setValidating(false);        factory.setNamespaceAware(false);        DocumentBuilder builder = factory.newDocumentBuilder();        builder.setEntityResolver(new EntityResolver() {            @Override            public InputSource resolveEntity(String publicId, String systemId) {                return new InputSource(dtd);            }        });        builder.setErrorHandler(new ErrorHandler() {            @Override            public void warning(SAXParseException e) {            }            @Override            public void error(SAXParseException e) throws SAXException {                throw e;            }            @Override            public void fatalError(SAXParseException e) throws SAXException {                throw e;            }        });        return builder.parse(xml);    }

Obviously, this is not a good solution. Because the configuration file will be repeated multiple times.

1. First optimization:

Why not read it once and then cache it? Then, when other methods call loadRoot (), they can directly use the cache. But there is a problem,InputStream cannot be cached and then read repeatedly, because once the InputStream is read, its pos pointer and so on will change, so it cannot be read repeatedly.. So onlyRead and process the content of the configuration file, cache it in byte [], and then use ByteArrayOutputStream to read the content in the cache again. Then, ByteArrayOutputStream is used to construct InputStream to read the configuration file once.And then re-construct InputStream to read the data again. The related code is as follows:

// To avoid frequent calls to loadRoot in the original code to read/mycat. dtd and/mycat. xml, so the two files are cached. // note that the files are not always cached in the memory. With the collection of LocalLoader objects, the memory occupied by the cache will naturally be recycled. Private static byte [] xmlBuffer = null; private static byte [] dtdBuffer = null; private static ByteArrayOutputStream xmlBaos = null; private static ByteArrayOutputStream dtdBaos = null; static {InputStream input = ConfigFactory. class. getResourceAsStream ("/mycat. dtd "); if (input! = Null) {dtdBuffer = new byte [1024*512]; dtdBaos = new ByteArrayOutputStream (); bufferFileStream (input, dtdBuffer, dtdBaos);} input = ConfigFactory. class. getResourceAsStream ("/mycat. xml "); if (input! = Null) {xmlBuffer = new byte [1024*512]; xmlBaos = new ByteArrayOutputStream (); bufferFileStream (input, xmlBuffer, xmlBaos );}}

BufferFileStream method:

    private static void bufferFileStream(InputStream input, byte[] buffer, ByteArrayOutputStream baos){        int len = -1;        try {    while ((len = input.read(buffer)) > -1 ) {    baos.write(buffer, 0, len);}    baos.flush();} catch (IOException e) {e.printStackTrace();logger.error(" bufferFileStream error: " + e.getMessage());}    }

LoadRoat is optimized as follows:

    private static Element loadRoot() {        Element root = null;        InputStream mycatXml = null;        InputStream mycatDtd = null;                if(xmlBaos != null)        mycatXml = new ByteArrayInputStream(xmlBaos.toByteArray());        if(dtdBaos != null)        mycatDtd = new ByteArrayInputStream(dtdBaos.toByteArray());                try {root = ConfigUtil.getDocument(mycatDtd, mycatXml).getDocumentElement();} catch (ParserConfigurationException | SAXException | IOException e1) {e1.printStackTrace();logger.error("loadRoot error: " + e1.getMessage());}finally{if(mycatXml != null){try { mycatXml.close(); } catch (IOException e) {}}if(mycatDtd != null){try { mycatDtd.close(); } catch (IOException e) {}}}                return root;    }

After this optimization, even if many methods frequently call the loadRoot () method, the configuration file will not be read repeatedly. Instead, the byte [] content will be used to repeatedly construct the InputStream.

In fact, its principle,Byte [] is used as an intermediate container to cache byte. ByteArrayOutputStream stores the byte read by InputStream as the byte [] container, and then uses ByteArrayInputStream to read content from the byte [] container, construct an InputStream. As long as the byte [] cache container exists, the InputStream can be constructed multiple times.. As a result, the configuration file is read once, And the InputStream is repeatedly constructed to avoid reading the configuration file every time the InputStream is constructed.

2. Second optimization:

You may think of better methods, such:

Why do we not cache the private static Element root = null as a class attribute so that you do not need to open or close the configuration file again? The modifications are as follows:

Public class LocalLoader implements ConfigLoader {private static final Logger logger = LoggerFactory. getLogger ("LocalLoader ");//..... private static Element root = null; // change the loadRoot method to private static Element loadRoot () {InputStream dtd = null; InputStream xml = null; // Element root = null; if (root = null) {try {dtd = ConfigFactory. class. getResourceAsStream ("/mycat. dtd "); xml = ConfigFa Ctory. class. getResourceAsStream ("/mycat. xml "); root = ConfigUtil. getDocument (dtd, xml ). getDocumentElement ();} catch (ConfigException e) {throw e;} catch (Exception e) {throw new ConfigException (e);} finally {if (dtd! = Null) {try {dtd. close ();} catch (IOException e) {}} if (xml! = Null) {try {xml. close () ;}catch (IOException e) {}}} return root ;}

In this way, you do not need to open or close the configuration file again. As long as the root property is not recycled, the Document Object introduced by root will also be cached. This is obviously much better than the first optimization, because the first optimization, we need to repeat the construction of InputStream from byte [], and then rebuild the Document Object.

3. The third Optimization

In the preceding example, private static Element root = null is cached as an attribute to avoid repeated reading. So why don't we cache the Document object as an attribute directly. It has better semantics and better understanding of the Code. The Code is as follows:

Public class LocalLoader implements ConfigLoader {private static final Logger logger = LoggerFactory. getLogger ("LocalLoader ");//...... // to avoid frequent calls to loadRoot in the original code to read/mycat. dtd and/mycat. xml, so the Document is cached. private static Document document = null; private static Element loadRoot () {InputStream dtd = null; InputStream xml = null; if (Document = null) {try {dtd = ConfigFactory. class. getResourc EAsStream ("/mycat. dtd "); xml = ConfigFactory. class. getResourceAsStream ("/mycat. xml "); document = ConfigUtil. getDocument (dtd, xml); return document. getDocumentElement ();} catch (Exception e) {logger. error ("loadRoot error:" + e. getMessage (); throw new ConfigException (e);} finally {if (dtd! = Null) {try {dtd. close ();} catch (IOException e) {}} if (xml! = Null) {try {xml. close ();} catch (IOException e) {}}} return document. getDocumentElement ();}

This is a qualified implementation. Anyway, the first optimization, learned how to use ByteArrayOutputStream and ByteArrayInputStream together with byte.

--------------------- Split line ------------------------------------

Reference: http://blog.csdn.net/it_magician/article/details/9240727 is as follows:

Sometimes we need to use the same InputStream object multiple times. For example, if the client obtains data from the server and uses the getInputStream () method of HttpURLConnection to obtain the Stream object, the data must be displayed to the foreground (read for the first time ), I want to write the data into the file and cache it locally (for the second read ).

However, after reading the InputStream object for the first time, it may have reached the end of the Stream (EOFException) or the Stream has been closed.

The InputStream object itself cannot be copied because it does not implement the Cloneable interface. In this case, you can first convert InputStream to ByteArrayOutputStream. When you want to use an InputStream object, you can then convert it back from ByteArrayOutputStream. The code is implemented as follows:

InputStream input = httpconn. getInputStream (); ByteArrayOutputStream baos = new ByteArrayOutputStream (); byte [] buffer = new byte [1024]; int len; while (len = input. read (buffer)>-1) {baos. write (buffer, 0, len);} baos. flush (); InputStream stream1 = new ByteArrayInputStream (baos. toByteArray (); // TODO: display to the front-end InputStream stream2 = new ByteArrayInputStream (baos. toByteArray (); // TODO: local cache