Combine Go to read apue-basic file I/O

This is a creation in Article, where the information may have evolved or changed.

In the public "do not catch the urgency" on the synchronization of the article, and can click the original link to read: Portal

Basic file I/O

I think open, read, write, Lseek, close this few operations will meet the basic requirements for file operations. Of course, I also read the book is so written.

Each language basically has a corresponding function or method, we call on the line, in this case, we can understand that 语言就是个工具 . I prefer go style, so here I take the function library of go as an example, but before introducing it, understand a concept: file descriptor.

The focus of the painting:

For the kernel, all open files are referenced by a file descriptor. The file descriptor is a non-negative integer.

Is the description of the above still a little vague?

When an existing file is opened or a new file is created, the kernel returns one to the process 文件描述符 .

When a file is read and written, the file that is returned by using open or create 文件描述符 is 文件描述符 passed as a parameter to read or write.

Typically, a variable is used fd to represent a file descriptor (Descripter)

function open and Openat & function Create

You can open or create a file by calling the Open or Openat function.

#include <fcntl.h>int open(const char *path, int oflag, ... /* mode_t mode */);int openat(int fd, const char *path, int oflag, ... /* mode_t mode */);

Call the CREATE function to make a new file.

#include <fcntl.h>int create(const char *path, mode_t mode);

Parameters in the above function:

• Path is the name of the file to open or create
• Oflag is the flag that is performed on the file, for example: o_rdwr| o_create| O_trunc
• mode specifies the access permission bit for the file
• FD represents a file descriptor

Here is a list of flags for what to do with the file in Go:

// Flags to OpenFile wrapping those of the underlying system. Not all// flags may be implemented on a given system.const (    O_RDONLY int = syscall.O_RDONLY // open the file read-only.    O_WRONLY int = syscall.O_WRONLY // open the file write-only.    O_RDWR   int = syscall.O_RDWR   // open the file read-write.    O_APPEND int = syscall.O_APPEND // append data to the file when writing.    O_CREATE int = syscall.O_CREAT  // create a new file if none exists.    O_EXCL   int = syscall.O_EXCL   // used with O_CREATE, file must not exist    O_SYNC   int = syscall.O_SYNC   // open for synchronous I/O.    O_TRUNC  int = syscall.O_TRUNC  // if possible, truncate file when opened.)

How to open or create a file with Go:

// Open file func Open(name string) (*File, error) {    return OpenFile(name, O_RDONLY, 0)}// Create file func Create(name string) (*File, error) {    return OpenFile(name, O_RDWR|O_CREATE|O_TRUNC, 0666)}

By observing the source code, we know that both are called OpenFile functions, just flag, mode is different.

OpenFile is the generalized open call; Most users would use open//or Create instead. It opens the named file with specified flag//(o_rdonly etc) and perm, (0666 etc) if applicable. If successful,//methods on the returned File can being used for i/o.//If There are an error, it would be of type *PATHERROR.F UNC OpenFile (name string, flag int, perm FileMode) (*file, error) {chmod: = False if!supportscreatewithstickybit & amp;& flag&o_create! = 0 && Perm&modesticky! = 0 {If _, err: = Stat (name); Isnotexist (err) {chmod = true}} var r int for {var e error r, E = Syscall. Open (name, Flag|syscall. O_cloexec, Syscallmode (perm)) if E = = nil {break}//On OS X, sigaction (2) doesn ' t Guara Ntee that Sa_restart would cause//open (2) to being restarted for regular files.        This was easy-to-reproduce on//fuse file systems (see HTTP://GOLANG.ORG/ISSUE/11180). If runtime. GOOS = = "Darwin" && e = = Syscall. EINTR {Continue} return nil, &patherror{"open", Name, E}}//Open (2) itself won ' t ha Ndle the sticky bit on *BSD and Solaris if chmod {chmod (name, PERM)}//There ' s a race here with Fork/ex EC, which we is//content to live with. See..    /syscall/exec_unix.go. If!supportscloseonexec {syscall. Closeonexec (R)} return NewFile (UIntPtr (r), name), nil}

When reading the above code, this is supportsCreatedWithStickyBit stuck, the Knowledge point is StickyBit (sticky position)

See below StickyBit (sticky bit):

In earlier versions of UNIX that did not use the request paging technique, if 可执行文件 set StickyBit , a copy of the body part of the program is still saved in the swap area at the end of the execution of the file, so that it can be loaded into memory quickly the next time it executes. However, most of today's UNIX is configured with virtual storage systems and fast file systems, so it is no longer necessary to use this technology.

In the OpenFile function source code, the constant supportsCreatedWithStickyBit value in Ubuntu 16.04 environment is true, so that part of the code will not be executed. So developers in the Ubuntu 16.04 environment don't have to know the if !supportsCreatedWithStickyBit ... code block. Because of the use of Ubuntu 16.04, OpenFile functions can be simplified as follows:

// OpenFile is the generalized open call; most users will use Open// or Create instead. It opens the named file with specified flag// (O_RDONLY etc.) and perm, (0666 etc.) if applicable. If successful,// methods on the returned File can be used for I/O.// If there is an error, it will be of type *PathError.func OpenFile(name string, flag int, perm FileMode) (*File, error) {    var r int    for {        var e error        r, e = syscall.Open(name, flag|syscall.O_CLOEXEC, syscallMode(perm))        if e == nil {            break        }        return nil, &PathError{"open", name, e}    }    return newFile(uintptr(r), name), nil}

The simplified code shows that the core code is: syscall.Open(name, flag|syscall.O_CLOEXEC, syscallMode(perm)) trigger the system call. Before we get to know each other, we'll take them out syscallMode(prem) and remove the obstacles.

// syscallMode returns the syscall-specific mode bits from Go's portable mode bits.func syscallMode(i FileMode) (o uint32) {    o |= uint32(i.Perm())    if i&ModeSetuid != 0 {        o |= syscall.S_ISUID    }    if i&ModeSetgid != 0 {        o |= syscall.S_ISGID    }    if i&ModeSticky != 0 {        o |= syscall.S_ISVTX    }    // No mapping for Go's ModeTemporary (plan9 only).    return}

Let us understand FileMode that the source code is defined in this way, type FileMode uint32 and by viewing the source value i.Perm() equivalent to i & 0777 , and through Open the understanding of mode 0, syscallmode (0) = = 0; Create The mode is 0666, Syscallmode (0666) = = 438

Tips: At first because POSIX end of the file is "POSIX system" (non-existent) down, after checking, only to know that the UNIX system is reduced.

So let's focus on switching to syscall.Open(name, mode, prem) , like C in the method! Deep words first dig into this place.

Let's go back to the OpenFile remaining knowledge points after simplification: PathError , NewFile(uintptr(r), name) .

PathErrorThe source code is as follows:

// PathError records an error and the operation and file path that caused it.type PathError struct {    Op   string    Path string    Err  error}func (e *PathError) Error() string { return e.Op + " " + e.Path + ": " + e.Err.Error() }

erroris an interface, as long as the implementation of the Error method is OK.

uintptr(r)is uintptr defined in the following:

// uintptr is an integer type that is large enough to hold the bit pattern of// any pointer.type uintptr uintptr

uintptr(r)ris a int type in.

Look at NewFile how this function is defined, the source code is as follows:

// NewFile returns a new File with the given file descriptor and name.func NewFile(fd uintptr, name string) *File {    fdi := int(fd)    if fdi < 0 {        return nil    }    f := &File{&file{fd: fdi, name: name}}    runtime.SetFinalizer(f.file, (*file).close)    return f}

The above function fd returns to the type after one cycle int . Fileis the file type of encapsulation, the source code is as follows:

// File represents an open file descriptor.type File struct {    *file // os specific}// file is the real representation of *File.// The extra level of indirection ensures that no clients of os// can overwrite this data, which could cause the finalizer// to close the wrong file descriptor.type file struct {    fd      int    name    string    dirinfo *dirInfo // nil unless directory being read}

In the above function runtime.SetFinalizer(f.file, (*file).close) , type C + + in the 析构函数 bar! (Dig, let's do It)

function close

Call the close function to close an open file.

#include <unistd.h>int close(int fd);

How do I use Go to close a file?

// Close closes the File, rendering it unusable for I/O.// It returns an error, if any.func (f *File) Close() error {    if f == nil {        return ErrInvalid    }    return f.file.close()}func (file *file) close() error {    if file == nil || file.fd == badFd {        return syscall.EINVAL    }    var err error    if e := syscall.Close(file.fd); e != nil {        err = &PathError{"close", file.name, e}    }    file.fd = -1 // so it can't be closed again    // no need for a finalizer anymore    runtime.SetFinalizer(file, nil)    return err}

Visible from the code above, syscall.Close(file.fd) similar c in close , plays a key role. Its source code is as follows:

// THIS FILE IS GENERATED BY THE COMMAND AT THE TOP; DO NOT EDITfunc Close(fd int) (err error) {    _, _, e1 := Syscall(SYS_CLOSE, uintptr(fd), 0, 0)    if e1 != 0 {        err = errnoErr(e1)    }    return}

Syscall(SYS_CLOSE, uintptr(fd), 0, 0)It is estimated that the lower level of the call, no longer digging.

function Lseek

Call Lseek to explicitly set an offset for an open file.

#include <unistd.h>off_t lseek(int fd, off_t offset, int whence);

Parameters in the above function:

• FD represents a file descriptor
• If whence is seek_set, set the offset of the file to 距文件开始处 offset a byte
• If whence is seek_cur, set the offset of the file to a 其当前值加 offset, offset positive negative
• If whence is seek_end, set the offset of the file to a 文件长度加 offset, offset positive negative

These parameters are also applicable to go, but this method has been deprecated in go, as detailed below:

// Seek whence values.//// Deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.const (    SEEK_SET int = 0 // seek relative to the origin of the file    SEEK_CUR int = 1 // seek relative to the current offset    SEEK_END int = 2 // seek relative to the end)

How do I use Go to set the offset of a file?

// Seek sets the offset for the next Read or Write on file to offset, interpreted// according to whence: 0 means relative to the origin of the file, 1 means// relative to the current offset, and 2 means relative to the end.// It returns the new offset and an error, if any.// The behavior of Seek on a file opened with O_APPEND is not specified.func (f *File) Seek(offset int64, whence int) (ret int64, err error) {    if err := f.checkValid("seek"); err != nil {        return 0, err    }    r, e := f.seek(offset, whence)    if e == nil && f.dirinfo != nil && r != 0 {        e = syscall.EISDIR    }    if e != nil {        return 0, f.wrapErr("seek", e)    }    return r, nil}

Visible f.seek(offset, whence) plays a key role.

// seek sets the offset for the next Read or Write on file to offset, interpreted// according to whence: 0 means relative to the origin of the file, 1 means// relative to the current offset, and 2 means relative to the end.// It returns the new offset and an error, if any.func (f *File) seek(offset int64, whence int) (ret int64, err error) {    return syscall.Seek(f.fd, offset, whence)}

syscall.Seek(f.fd, offset, whence)Initiated a system call, and then dug it up to the bottom and the assembly.

function read

Call the Read function to read the data from the open file.

#include <unistd.h>ssize_t read(int fd, void *buf, size_t nbytes);

Parameters in the above function:

• FD represents a file descriptor
• BUF the file to be read, the type is a generic pointer
• Nbytes indicates the number of bytes read

If read succeeds, it returns the number of bytes read, such as the end of the file that was reached, 0.

Tips: There are a number of situations where the number of bytes that are actually read is less than what is required to be read.

How do I read data from an open file with Go?

// Read reads up to len(b) bytes from the File.// It returns the number of bytes read and any error encountered.// At end of file, Read returns 0, io.EOF.func (f *File) Read(b []byte) (n int, err error) {    if err := f.checkValid("read"); err != nil {        return 0, err    }    n, e := f.read(b)    return n, f.wrapErr("read", e)}

Its underlying code, like above, is recursively viewed go package .

Function write

Call the Write function to write data to the open file.

#include <unistd.h>ssize_t write(int fd, const void *buf, size_t nbytes);

Parameters of the above function:

• FD represents a file descriptor
• BUF the file to be written, the type is a generic pointer
• Nbytes indicates the number of bytes read

If write succeeds, the number of bytes read is returned, such as the end of the file reached, 0 is returned.

How do I write data to an open file with Go?

func (f *File) Write(b []byte) (n int, err error)

End

If the light to see Apue, the first few pages can also, slowly look down, go to the Lib basically with the Unix interface similar to, combined with the source of the go to see together, as long as there is a general framework OK, with the future slowly in depth, there will be a deeper understanding.