Java --- reading and writing large binary files; java binary

Java --- reading and writing large binary files; java binary

As required by the project, you need to read, write, and convert binary files.

File Description: a binary file obtained by other programs. The file content is: 23543 sets of flow rate vectors (u, v) files corresponding to the triangular network with 13270 triangles and 721 vertices, generally speaking, a piece of data contains two double-precision values. Each array contains 23543 pieces of data. If a double-precision value is used as the unit, then there are a total of 23543*721*2 = 33,949,006 data records. The Fortran program stores a numeric binary file every 8 bytes. The final file size is shown in:

　　　　　　　　　　　　　　

　　

Test: After Reading data from this file, convert it to decimal and store it in another file.

　　

/*** For large file storage, call beginSave, AddSave, and endSave in sequence. ** @ Author CK **/public class DataUtil {DataOutputStream BinaryOut = null; BufferedWriter TextOut = null; String FilePath = null; enum SaveFileType {Text, Binary}; SaveFileType; /*** convert double to byte [] ** @ param d * @ return */public static byte [] double2Bytes (double d) {long value = Double. doubleToRawLongBits (d); byte [] byteRet = new byte [8]; for (int I = 0; I <8; I ++) {byteRet [I] = (byte) (Value> 8 * I) & 0xff) ;}return byteRet ;} /*** convert byte [] to double ** @ param arr * @ return */public static double bytes2Double (byte [] arr) {long value = 0; for (int I = 0; I <8; I ++) {value | = (long) (arr [I] & 0xff )) <(8 * I);} return Double. longBitsToDouble (value);}/*** start of large-scale data storage * @ param FilePath file path * @ param saveFileType, text Files and binary files with Double Precision * @ return * @ throws IOException */Public boolean BeginSave (String FilePath, SaveFileType saveFileType) throws IOException {if (FilePath = "" | FilePath = null) {System. out. println ("the SavePath is null. "); return false;} this. filePath = FilePath; this. saveFileType = saveFileType; File dataFile = new File (FilePath); if (! DataFile. getParentFile (). exists () {dataFile. getParentFile (). mkdirs ();} if (dataFile. exists () {dataFile. delete ();} dataFile. createNewFile (); switch (this. saveFileType) {case Text: TextOut = new BufferedWriter (new FileWriter (dataFile, true); break; case Binary: BinaryOut = new DataOutputStream (new FileOutputStream (dataFile, true )); break; default: break;} return true;}/*** append storage for large file storage * @ param DataStr is not required for text storage. If it is a binary file of Double precision, it is separated by several spaces * @ return * @ throws IOException */public boolean AddSave (String DataStr) throws IOException {switch (this. saveFileType) {case Text: this. textOut. append (DataStr); break; case Binary: DataStr = DataStr. trim (); String [] dataArray = DataStr. split ("\ s +"); for (int I = 0; I <dataArray. length; I ++) {this. binaryOut. write (double2Bytes (Double. parseDouble (dataArray [I]);} brea K; default: break;} return true;}/*** end storage of large file storage, clear the cache, and close the file. * @ Return * @ throws IOException */public boolean EndSave () throws IOException {switch (this. saveFileType) {case Text: this. textOut. flush (); this. textOut. close (); break; case Binary: this. binaryOut. flush (); this. binaryOut. close (); break; default: break;} return true;}/*** Save the string as a text file (completed at one time) ** @ param DataStr * file content * @ param SavePath * file path, including the file name and suffix * @ return * @ throws IOException */public bool Ean saveTextFile (String DataStr, String SavePath) throws IOException {if (DataStr = "" | DataStr = null) {System. out. println ("the dataStr is null. "); return false;} if (SavePath =" "| SavePath = null) {System. out. println ("the SavePath is null. "); return false;} File dataFile = new File (SavePath); if (! DataFile. getParentFile (). exists () {dataFile. getParentFile (). mkdirs ();} if (dataFile. exists () {dataFile. delete ();} dataFile. createNewFile (); BufferedWriter out; out = new BufferedWriter (new FileWriter (dataFile); out. append (DataStr); out. flush (); out. close (); return true;}/*** double-precision storage is a string consisting of binary data (one-time storage) ** @ param DataStr double-precision data, separated by several spaces * @ param OutputPath * @ return * @ throws IOException */publ Ic boolean saveBinaryFile (String DataStr, String OutputPath) throws IOException {if (DataStr = "" | DataStr = null) {System. out. println ("the dataStr is null. "); return false;} if (OutputPath =" "| OutputPath = null) {System. out. println ("the OutputPath is null. "); return false;} File dataFile = new File (OutputPath); if (! DataFile. getParentFile (). exists () {dataFile. getParentFile (). mkdirs ();} if (dataFile. exists () {dataFile. delete ();} dataFile. createNewFile (); DataOutputStream out; out = new DataOutputStream (new FileOutputStream (dataFile); // data processing DataStr = DataStr. trim (); String [] dataArray = DataStr. split ("\ s +"); for (int I = 0; I <dataArray. length; I ++) {out. write (double2Bytes (Double. parseDouble (dataArray [I]);} out. flush (); out. close (); return true ;}}

Code Description, the code above contains a method to store all the content in the cache when processing small files, and then write the content into a text file or binary file at a time. It also contains a method to read and write large files, the following is your own read/write test.

/*** Test the reading and writing of a large binary file (about MB) * @ author ck **/public class FileTest {static String inputFilePath = ""; // input file path, contains the file name suffix static String outputFilePath = ""; // output file name, including the file name suffix public static void file2file () throws IOException {DataUtil dataUtil = new DataUtil (); dataInputStream br = new DataInputStream (new BufferedInputStream (new FileInputStream (inputFilePath); dataUtil. beginSave (outputFilePath, SaveFileType. T Ext); // initialization, creating a file, using the idea of append storage of a file byte [] oneData = new byte [8]; int I = 0, count = 0; while (br. read (oneData, 0, 8 )! =-1) {I = I + 1; dataUtil. addSave (String. valueOf (DataUtil. bytes2Double (oneData); if (I/23543 = 0) {count ++; System. out. println (count + "\ n") ;}} dataUtil. endSave (); // write data in the cache to the file to close the file. }}

The test code soon ran out, but it took about nearly half a minute to generate the output file (deliberately timing the stopwatch once). I tried to use a one-time read/write method and got stuck for a long time, no results. The resulting decimal text file is of the following size:

I think the original intention of the Fortran Program author should be that binary storage saves space than decimal storage. In fact, it actually saves more than half of the space.

Well, this record is complete.