Golang file Operations Collation

This is a creation in Article, where the information may have evolved or changed.

Recently done a little thing, with a lot of Golang file operation of the relevant content, create, delete, traverse, compression and so on, here to tidy up, hoping to master the system a little, the vague place to clarify.

Basic operations

File creation

When creating a file, be sure to pay attention to the permissions issues, the general default file permissions are 0666 about the permissions of the relevant content, specifically can refer to the bird Uncle p141 here or again, the file properties R W x r W x r W x, the first bit is the file attributes, the general commonly used "-" represents the ordinary file, "D" represents Is the directory, Golang inside the use of os.Create creating files when it seems to use only 0xxx form. For example, 0666 means that a normal file is created, the permissions of the file owner, the permissions of the user group to which the file belongs, and the other person's permissions on the file are 110 readable and writable, not executable.

File deletion

When the file is deleted, whether it is a normal file or a directory file, you can do err:=os.Remove(filename) so with this operation. Of course, if you want to remove the entire folder, use the RemoveAll(path string) action directly. Can take a look at the internal implementation of the RemoveAll function, the overall is the traversal, recursive operation process, the other similar file operations can be implemented with similar templates, the following RemoveAll function as a template for a specific analysis, pay attention to the various situations:

Func RemoveAll (Path string) error {//simple case:if Remove works, we ' re done.//first try remove if it is a normal file directly delete the error may be the directory and the child files ER r: = Remove (path)//Yes or the path does not exist directly return nilif err = Nil | | Isnotexist (Err) {return nil}//Otherwise, is this a directory where we need to recurse into?//directory there are files that need to be recursively processed//note LSTAT and STAT functions The difference, two is the status information of the returned file//lstat more the function of processing link file, will return the information of linked file, and state directly returns the information of the file pointed by the link file dir, serr: = Lstat (path) if serr! = Nil {if serr, OK: = Serr. ( *PATHERROR); Ok && (Isnotexist (SERR). ERR) | | Serr. ERR = = Syscall. Enotdir) {return Nil}return serr}//is not a directory if!dir. Isdir () {//not a directory; return the error from Remove.return err}//directory.fd, err: = Open (path) if err! = Nil {if I Snotexist (ERR) {//Race. It was deleted between the Lstat and open.//Return nil per RemoveAll ' s Docs.return nil}return err}//Remove Contents &amp ; return First error.err = nil//recursively traverse a file in the directory if the parameter n<=0 the entire information into a slice return//If the parameter n>0 the information of at most n elements is deposited into the slice// A similar function is Readdir, which returns the FileInfo information for the contents of the directory for {names, err1: = FD. Readdirnames (+) for _, Name: = range names {err1: = RemoveAll (Path + string (pathseparator) + name) If Err = = Nil {err = ER r1}}//Traverse to the last position if err1 = = Io. EOF {break}//If readdirnames returned an error, use it.if err = = Nil {err = Err1}if len (names) = = 0 {break}}//Close dire Ctory, because Windows won ' t remove opened directory.fd.Close ()//recursion end current directory next empty Delete current directory//Remove DIRECTORY.ERR1: = Remove (Pat h) If err1 = = Nil | |     Isnotexist (ERR1) {return nil}if err = = Nil {err = Err1}return err}

File status

Write content from a file

This part of the more involved in the I/O operation, the introduction of the system in the I/O part of the collation, in general to read and write content in the file when there are three ways:

1. Use a f, err := os.Open(file_path) f.read() f.write() custom buffer to read/read fixed content from a file directly after using open file

2, the use of Ioutl ReadFile and WriteFile method

3. Use Bufio to read and write using a cache, for example, by info:=bufio.NewReader(f) implementing IO. After an instance of reader's interface is loaded, you can use info. ReadLine () to implement a full line of reads every time until the Err information is IO. EOF, read end

This blog to three kinds of file operation of the read speed comparison, seemingly read large files when the use of ioutil when the efficiency is higher.

Each method has a different application, the following is an example of three different ways to read the operation, for writing to the file, you can refer to the read operation to do:

Package Mainimport ("Bufio" "FMT" "io" "io/ioutil" "OS") func check (e error) {if E! = nil {panic (e)}}func main () {//view current working directory path To get the absolute path of the test file Current_dir, _: = OS. GETWD () fmt. Println (current_dir) File_path: = Current_dir + "/temp.txt"//Mode One://through the Ioutil directly through the file name to load files//Once the entire file loaded in a large size Err returns to nil when the file will be successfully loaded DAT, err: = Ioutil. ReadFile (File_path)//flocculent contains a directory that will return []os. FileInfo's information//ioutil. ReadDir () Check (ERR)//the type of data is []UINTFMT. PRINTLN (DAT)//Convert the file contents to a string output FMT. Println (string (DAT))//Mode two://via OS. The open way to get a variable of type *file//seems to be a pointer to this file through which the pointer can do a finer-grained operation on the file F, err: = OS. Open (file_path) Check (ERR)//Manually specify a fixed size buffer each time through buffer to do the corresponding operation buffer1: = Make ([]byte, 5)//Read len from File F (buffer1) The message to Buffer1 returns a value N1 is the length of the read byte n1, err: = F.read (buffer1) Check (err) fmt. Printf ("%d bytes:%s\n", N1, String (buffer1))//by F.seek for finer operations The first parameter indicates that offset is 6 The second parameter indicates the relative position of the start of the file// Then read the information from the O2 position and read the message O2, err: = F.seek (6, 0) Check (err) Buffer2: = Make ([]byte, 2)//read into N2 length information to Buffer2, err: = N2 ( BUFFER2) Check (err) fmt. Printf ("%d bytes after%dPosition:%s\n ", N2, O2, String (buffer2))//function with IO Package can also implement similar functions O3, err: = F.seek (6, 0) Check (err) Buffer3: = Make ([]byte, 2 ) N3, err: = Io. Readatleast (f, Buffer3, Len (buffer3)) Check (err) fmt. Printf ("%d bytes after%d position:%s\n", N3, O3, String (BUFFER3))//mode three//by Bufio package to read Bufio There are many more useful functions such as reading a whole line of content at a time//tune The starting position of the entire file pointer is to the beginning of the place _, err = F.seek (0) Check (err) R4: = Bufio. Newreader (f)//read out from the beginning of the 5 byte b4, err: = R4. Peek (5) Check (err)//fmt. Println (String (B4)) fmt. Printf ("5 bytes:%s\n", String (B4))//Adjust the file to another place _, err = F.seek (0, 0) Check (err) R5: = Bufio. Newreader (f)//read out the 5 byte B5 starting from the position indicated by the pointer, err: = R5. Peek (5) Check (err)//fmt. Println (String (B4)) fmt. Printf ("5 bytes:%s\n", String (B5))//The other functions of the test Bufio for {//read-out contents are saved as string each time read to ' \ n ' is the location marked by line, err: = R5. ReadString (' \ n ') fmt. Print (line) If Err = = Io. The use of EOF {break}}//readline () ReadByte () is similar generally when err is IO. EOF when//read the content on the end//feel the actual use of time or through the way three better granularity is appropriate there are many ways to handle the input F. Close ()}

Advanced operations

File packaging, file decompression, file traversal, these related operations can basically refer to the RemoveAll way to do, is recursive plus traversal way.
The following is an implementation of file compression:

Packages the contents of a folder into a. gz.tar file package Mainimport ("Archive/tar" "Compress/gzip" "FMT" "IO" "OS")//writes the contents of the fi file to the dir directory Compress into a tar file in the Func filecompress (TW *tar. Writer, dir string, fi os. FileInfo) {//Open file is a combination of directory name/file name composition filename: = dir + "/" + FI. Name () fmt. Println ("The Last One:", filename) fr, err: = OS. Open (filename) fmt. Println (Fr. Name ()) if err! = Nil {panic (err)}defer fr. Close () HDR, err: = Tar. Fileinfoheader (FI, "") Hdr. Name = Fr. Name () If Err = tw. Writeheader (HDR); Err! = Nil {panic (err)}//bad way////The information header when the tar file is generated, first write to the TAR structure//h: = new (tar. Header)////fmt. Println (reflect. TypeOf (h))//h.name = fi. Name ()//h.size = fi. Size ()//h.mode = Int64 (FI. Mode ())//h.modtime = fi. Modtime ()////writes the contents of the information header to//err = tw. Writeheader (h)//if err! = Nil {//panic (err)//}//copy (DST Writer,src Reader) _, err = Io. Copy (TW, FR) if err! = Nil {panic (err)}//print file name FMT. Println ("Add the file:" + fi.) Name ())}//the contents of the directory recursively through the write to the tar file in the Func dircompress (TW *tar. Writer, dir string) {fmt. Println (dir)//Open folder Dirhandle, err: = OS. Open (dir + "/")//fmt. Println (dir. Name ())//fmt. Println (reflect. TypeOf (dir)) if err! = Nil {panic (err)}defer Dirhandle. Close () FIS, err: = Dirhandle. Readdir (0) The type of//fis is []os. Fileinfo//can also read the names of all the sub-files through the Readdirnames//But in this way, it is necessary to determine whether the file is to get the file information by STAT//The OS is returned. The type of file if err! = Nil {panic (err)}//traverses the file list for each file to be written to a new *tar. Header//var fi os. Fileinfofor _, Fi: = Range fis {fmt. Println (FI. Name ()) if Fi. Isdir () {newname: = dir + "/" + FI. Name () fmt. Println ("Using dir") fmt. PRINTLN (newname)//This kind of direct continue wrote all the files together without hierarchy//filecompress (tw, dir, FI) dircompress (TW, NewName)} else {// If the normal file is written directly behind Dir already has the/filecompress (tw, dir, FI)}}}//in the Tardir directory to create a. tar.gz file that holds the compressed file after the Func Dirtotar (SourceDir string, Tardir string, tarname string) {//file write creates a FW in the Tardir directory, err: = OS. Create (Tardir + "/" + Tarname + ". tar.gz")//type of FW is *os. File//fmt. Println (reflect. TypeOf (FW)) If err! = Nil {panic (err)}defer FW. Close ()//gzip WRITERGW: = gzip. Newwriter (FW) defer GW. Close ()//tar writetw: = tar. Newwriter (GW) FMT. Println ("Source directory:", SOurcedir) dircompress (TW, SourceDir)//By controlling the write stream you can also control the directory structure such as the Dockerfile file under the current directory is written separately in the outermost fileinfo, err: = OS. Stat ("Tarrepo" + "/" + "Testdockerfile") fmt. Println ("The file name:", FileInfo. Name ()) if err! = Nil {panic (err)}//For example, the outermost layer placed dockerfile in the TAR package is registered to the/tarrepo/testdockerfile in the TAR package filecompress (TW, " Tarrepo ", FileInfo)//filecompress (TW," systempdir/test_testwar_tar/", FileInfo) fmt. Println ("tar.gz packaging OK")}func Main () {//workdir, _: = OS. GETWD ()//fmt. Println (Workdir) Dirtotar ("TestDir", "Tarrepo", "Testtar")}