Go language concurrent programming mutual exclusion lock, read-write lock detailed _golang

In this section, we describe the lock-related APIs provided in the go language. This includes mutexes and read-write locks. We described the mutex in the 6th chapter, but did not mention the read-write lock. Both of these locks are very common and important for traditional concurrent programs.

One, mutual exclusion lock

Mutual exclusion Lock is the main means of accessing and controlling shared resources by traditional concurrent programs. It is represented by the mutex structure body type in the standard library code bundle sync. Sync. Type of mutex (specifically, *sync.) Mutex type) has only two public methods--lock and unlock. As the name implies, the former is used to lock the current mutex, while the latter is used to unlock the current mutex amount.

Type sync. The 0 value of a mutex represents a mutex that is not locked. In other words, it is an out-of-the-box tool. We just need to make a simple statement about it and we can use it normally, like this:

Copy Code code as follows:

var mutex sync. Mutex

Mutex. Lock ()

One of the low-level mistakes we can make when we use lock tools in other programming languages, such as C or Java, is to forget to unlock locked locks in time, causing a series of problems such as process execution exceptions, thread execution stagnation, and even program deadlock. In the go language, however, this low-level error is extremely low. The main reason is the existence of defer statements.

We typically lock the mutex and then use the defer statement to ensure that the mutex is unlocked in a timely manner. Take a look at the following function:

Copy Code code as follows:

var mutex sync. Mutex

Func Write () {

Mutex. Lock ()

Defer mutex. Unlock ()

Omit several statements

}

This defer statement in function write guarantees that the mutex mutex must be unlocked before the function is executed. This eliminates the task of additional unlock operations that we repeat before all return statements and when the exception occurs. In the case where the internal execution process of a function is relatively complex, this workload is not negligible and is prone to omission and error. Therefore, the defer statement here is always necessary. In the go language, this is a very important idiom. We should develop this kind of good habit.

The lock operation and unlock operation for the same mutex should always appear as a pair. If we lock a locked mutex, the goroutine of the repeat lock will be blocked until the mutex returns to the unlocked state. Take a look at the following example:

Copy Code code as follows:

Func Repeatedlylock () {

var mutex sync. Mutex

Fmt. PRINTLN ("Lock" lock. (G0) ")

Mutex. Lock ()

Fmt. PRINTLN ("The lock is locked. (G0) ")

For i: = 1; I <= 3; i++ {

Go func (i int) {

Fmt. Printf ("Lock" lock. (g%d) \ n ", i)

Mutex. Lock ()

Fmt. PRINTF ("The lock is locked. (g%d) \ n ", i)

} (i)

}

Time. Sleep (time. Second)

Fmt. Println ("Unlock the lock.") (G0) ")

Mutex. Unlock ()

Fmt. PRINTLN ("The lock is unlocked. (G0) ")

Time. Sleep (time. Second)

}

We refer to the goroutine that executes the Repeatedlylock function as G0. In the Repeatedlylock function, we also enabled 3 Goroutine and named them G1, G2, and G3 respectively. As you can see, we locked the mutex mutex before enabling the 3 Goroutine, and also added a lock on the mutex at the beginning of the go function that the 3 goroutine will execute. The meaning of this is to simulate the case where the same mutex is locked concurrently. After the For statement is executed, let G0 take a nap for 1 seconds so that the runtime system has enough time to start running G1, G2, and G3. After that, unlock the mutex. In order to give readers a clearer idea of how the Repeatedlylock function was executed, we added several print statements before and after these locking and unlocking operations, and added the names we have for these goroutine in the printed content. Also for this reason, we wrote a "sleep" statement at the end of the Repeatedlylock function to wait a little while for other printed content that might appear.

After a short execution, the following will appear on the standard output:

Copy Code code as follows:

Lock the lock. (G0)

The lock is locked. (G0)

Lock the lock. (G1)

Lock the lock. (G2)

Lock the lock. (G3)

Unlock the lock. (G0)

The lock is unlocked. (G0)

The lock is locked. (G1)

From these eight lines of print content, we can clearly see the above four Goroutine implementation. First, at the beginning of the Repeatedlylock function, the first locking operation on the mutex is performed and successfully completed. This can be seen from the first and second lines of the printed content. Then, the three goroutine that are enabled in the Repeatedlylock function begin to run during the first "sleep" of G0. When the lock operation on the mutex is carried out in the corresponding go function, they are blocked. The reason is that the mutex is already locked. This is where we see only three consecutive lock-lock. (g<i>) without immediately seeing the lock is locked. (g<i>) The reason. Subsequently, G0 "wakes up" and unlocks the mutex. This allows the G1, G2, and G3 that are being blocked to have the opportunity to lock the mutex again. However, only one goroutine will succeed. A goroutine that successfully completes the lock operation continues to execute the statement after the operation. Other goroutine will continue to be blocked until a new opportunity arrives. This is the meaning of the last three lines in the printed content. Apparently, G1 grabbed the opportunity and managed to lock the mutex.

In fact, the reason why we are able to control the unique access to shared resources by using mutexes is precisely because of this feature. This is an effective way to eliminate the competing conditions.

The reverse operation of a mutex lock operation does not cause any goroutine blocking. However, it is possible to cause run-time panic. More specifically, when we unlock a mutex that is already in the unlocked state, a run-time panic is made. This situation is likely to occur in a relatively complex process-we may repeat the unlocking operation for the same mutex in one or more branches. The simplest and most effective way to avoid this is still to use the defer statement. This makes it easier to guarantee the uniqueness of the unlock operation.

While mutexes can be directly shared across multiple goroutine, it is strongly recommended that a pair of lock and unlock operations on the same mutex be placed in a block of code at the same level. For example, a mutex is locked and unlocked in the same function or method. For example, a mutex is used as a field in a struct type to use it in multiple methods of that type. In addition, we should make the variables that represent the mutex as low as possible. In this way, it is possible to avoid misuse in unrelated processes, resulting in incorrect behavior of the program.

Mutexes are the simplest of many of the synchronization tools we've seen. By following the few tips mentioned earlier, we can use the mutex in a correct and efficient way, using it to ensure the uniqueness of access to shared resources. Here we look at a slightly more complex lock implementation-read and write locks.

Second, read and write lock

Read-write locks are mutexes for read-write operations. The biggest difference between it and a common mutex is that it can lock and unlock operations for both read and write operations. Read-write locks follow an access control rule that differs from a mutex lock. It allows any read operation to be performed concurrently, within the scope of the read-write lock. At the same time, however, it allows only one write operation to be performed. Also, read operations are not allowed during the process of a write operation. In other words, read and write lock control under the multiple write operations are mutually exclusive, and write operations and read operations are mutually exclusive. However, there is no mutex between multiple read operations.

Such a rule is very appropriate for concurrent read and write to the same piece of data. Because, regardless of the amount of concurrency in the read operation, these operations do not cause changes to the data itself. The write operation will not only interfere with other write operations at the same time, but also may cause incorrect results of simultaneous read operations. For example, in a 32-bit operating system, read and write operations for Int64 type values cannot be completed by only one CPU instruction. During the process of a write operation, a read operation for the same one may read to a value that has not been modified. The value is neither equal to the old value nor equal to the new value. Such errors are often difficult to detect and are hard to fix. Therefore, in such a scenario, a read-write lock can greatly reduce the loss of program performance due to the use of locks, and the access control of shared resources is achieved.

In the go language, read and write locks are sync by struct type. Rwmutex representative. Similar to mutexes, sync. The 0 value of the Rwmutex type is already an immediately available read-write lock. A method collection of this type contains two pairs of methods, namely:

Copy Code code as follows:

Func (*rwmutex) Lock

Func (*rwmutex) Unlock

And

Copy Code code as follows:

Func (*rwmutex) Rlock

Func (*rwmutex) Runlock

The name and signature of the first pair of methods are exactly the same as the two methods of the mutex. They represent the lock and unlock of the write operation respectively. For short, they are write locks and write unlocks. The latter pair of methods represent the lock and unlock of the read operation respectively. Hereinafter referred to as read lock and read unlock.

Writing locks on read-write locks that have been written locked can cause the current goroutine to block until the read-write lock is written to unlock it. Of course, if more than one goroutine is blocked, the corresponding write unlock will only cause one of the goroutine's operations to be restored. Similarly, reading locks on a read-write lock that has been written locked can block the corresponding goroutine. But the difference is that once the read-write lock is written to unlock, all goroutine that are blocked by the read lock will be restored. If, on the other hand, the current read-write lock is found to be read-locked during the process, the write-lock operation waits until all read locks imposed on the read-write lock are cleared. Similarly, when multiple write-lock operations are waiting for this, the full purge of the corresponding read locks can only give one of the write-locking operations an opportunity to do so.

Now pay attention to write unlock and read unlock. If a write lock is written to a write-lock that is not read-locked, a run-time panic is raised. Similarly, a run-time panic can occur when reading and unlocking a read-write lock that is unread and locked. The write unlock attempts to wake up all goroutine that are blocked for read locking. While the read unlock is in progress, an attempt is made to awaken a goroutine that is blocked by a write lock.

Whether the lock is for write or read operation, we should try to unlock the corresponding lock as promptly as possible. We don't have to say much about writing unlocked. and read unlock the timely conduct is often more easily overlooked by us. Although read unlock does not have any effect on other ongoing read operations, it is closely related to the corresponding write lock. Note that for the same read-write lock, there can be more than one reading lock imposed on it. Therefore, only if we do the same amount of read unlock to the mutex, can we make a corresponding write lock get the opportunity to carry on. Otherwise, the latter continues to cause the goroutine to be blocked. Because of sync. Rwmutex and *sync. The Rwmutex type has no corresponding method for getting the number of read locks that have been made, so this is a very easy problem. Luckily we can use the defer statement to try to avoid this kind of problem. Keep in mind that write locks and read locks for the same read-write lock are mutually exclusive. Whether it is write-unlock or read-unlock, the operation will have a negative impact on the normal execution of the process using the read-write lock.

In addition to the two pairs of methods we explained in detail earlier, *sync. The Rwmutex type also has another method--rlocker. This rlocker method returns a value that implements the Sync.locker interface. Sync. The locker interface type contains two methods, namely: Lock and Unlock. Attentive readers may find that *sync. Mutex type and *sync. The Rwmutex type is the implementation type of the interface type. In fact, we're calling *sync. The resulting value of the Rlocker method of the Rwmutex type value is the value itself. However, the lock method and the Unlock method for this result value respectively correspond to the read lock operation and read unlock operation for the read-write lock. In other words, we are actually invoking the Rlock method or the Runlock method of the read-write lock when we call the lock method or the Unlock method of the result value of a read-write lock Rlocker method. Such an operation is not difficult to implement. We can also easily write the implementation of these methods ourselves. The practical significance of obtaining such a result value through the Rlocker method of reading and writing locks is that we can then manipulate the write lock and read lock in the read-write lock in the same way later. This provides convenience for flexible adaptation and replacement of related operations.

A complete example of a lock

Let's look at an example of the lock implementation above. There is a struct type named file in the standard library Code pack OS in the go language. Os. The value of a file type can be used to represent a file or directory in a file system. Its method collection contains a number of methods, some of which are used to write and read the corresponding files.

Suppose we need to create a file that holds the data. At the same time, there may be multiple goroutine to write and read the file separately. Each write operation should write several bytes of data to this file. This number of bytes of data should exist as a separate block of data. This means that the write operation can not interfere with each other, the written content can not be interspersed between the situation and confusion. On the other hand, each read operation should read a separate, complete block of data from this file. The blocks of data they read cannot be duplicated and need to be read sequentially. For example, the first read operation reads Block 1, so the second read should read Block 2, and the third read should read Block 3, and so on. There is no requirement for whether these read operations can be performed concurrently. Even if they are carried out at the same time, the program should distinguish their order.

To highlight the point, we stipulate that each block of data is of the same length. The length should be given at the time the initialization is initialized. If the write operation actually wants to write the data longer than the value, then the excess portion will be truncated.

When we get such a demand, we should first think about using the OS. The file type. It provides low-level support for us to manipulate files in the file system. However, this type of correlation method does not guarantee the security of concurrent operations. In other words, these methods are not concurrency-safe. I can only guarantee this by means of additional synchronization. Since there is a need for access control for both types of operations (i.e., write operations and read operations), read-write locks are more useful here than ordinary mutexes. However, we need to use other auxiliary means to solve the problem that multiple read operations are ordered and cannot be read repeatedly.

To achieve these requirements, we need to create a type. As a behavior definition for this type, we first write an interface like this:

Copy Code code as follows:

The interface type of the data file.

Type DataFile Interface {

Reads a block of data.

Read () (RSN int64, D Data, err error)

Writes a block of data.

Write (d Data) (WSN Int64, err Error)

Gets the serial number of the last data block that was read.

RSN () Int64

Gets the serial number of the last block of data that was written.

WSN () Int64

Get the length of a block of data

Datalen () UInt32

}

Where the type data is declared as a []byte alias type:

Copy Code code as follows:

Type of data

Type Data []byte

The name WSN and RSN are abbreviated forms of writing serial number and reading serial number respectively. They represent the serial number of the last data block to be written and the serial number of the last read block. The serial number described here is equivalent to a count value, which starts at 1. Therefore, we can get the number of data blocks that are currently being read and written by invoking the RSN method and the Wsn method.

Based on the simple analysis of the requirements and the DataFile interface type declaration, we can write a real implementation. We name this implementation type Mydatafile. Its basic structure is as follows:

Copy Code code as follows:

The implementation type of the data file.

Type mydatafile struct {

F *os. File//files.

Fmutex sync. Rwmutex//read/write lock for file.

Woffset the offset to be used for the int64//write operation.

Roffset the offset to be used for the Int64//read operation.

Wmutex sync. mutexes//write operations need to use the mutex.

Rmutex sync. Mutex/read operation requires mutexes.

Datalen UInt32//Data block length.

}

Type Mydatafile A total of seven fields. We have already explained the meaning of the first two fields in the previous one. Since write operations and read operations on data files are separate, we need two fields to store the progress of both types of operations. Here, this progress is represented by an offset. Thereafter, we refer to the Woffset field as the write offset, while the Roffset field is called the read offset. Note that when we write and read, we increase the values of these two fields separately. A race condition is generated when multiple writes are added to the value of the Woffset field. Therefore, we need mutual-exclusion lock wmutex to protect it. Similarly, Rmutex mutexes are used to eliminate the competing conditions that occur when multiple read operations increase the value of the Roffset field at the same time. Finally, it is clear from the above requirements that the length of the data block should be given when initializing the Mydatafile type value. This length is stored in the Datalen field of the value. It corresponds to the Datalen method declared in the DataFile interface. Let's take a look at the function newdatafile that was used to create and initialize datafile type values.

As for the writing of such functions, readers should have been very adept at it. The Newdatafile function returns a datafile type value, but in fact it creates and initializes a value of the *mydatafile type and takes it as its result value. The reason that this can be compiled is that the latter is one of the implementation types of the former. The full declaration of the Newdatafile function is as follows:

Copy Code code as follows:

Func newdatafile (Path string, Datalen UInt32) (datafile, error) {

F, err: = OS. Create (PATH)

If Err!= nil {

return nil, err

}

If Datalen = 0 {

return nil, errors. New ("Invalid data length!")

}

DF: = &mydatafile{f:f, Datalen:datalen}

Return DF, Nil

}

As you can see, we only need to initialize the fields F and datalen when we create *mydatafile type values. This is because the 0 values for both the Woffset and Roffset fields are 0, and their values should be the same in the case of no write operations or read operations. For fields Fmutex, Wmutex, and Rmutex, their 0 value is the available lock. So we don't have to initialize them explicitly.

The value of the variable DF as the first result of the Newdatafile function reflects our design intent. But to make the *mydatafile type truly an implementation type of the datafile type, we also need to write all the methods that have been declared in the DataFile interface type for the *mydatafile type. The most important of these is the Read method and the Write method.

Let's start by writing the *mydatafile type of Read method. The method should be implemented in the following steps.

(1) Gets and updates the read offset.

(2) reads a piece of data from a file based on the read offset.

(3) encapsulate the block into a data type value and return it as the result value.

wherein, the previous step should be protected by the mutex Rmutex when it is executed. Because we require multiple read operations to not read the same block of data, and they should read the data blocks in the file sequentially. And the second step, we will also use the read and write lock Fmutex to protect. Here is the first version of this read method:

Copy Code code as follows:

Func (DF *mydatafile) Read () (RSN int64, D Data, err error) {

Read and update read offsets

var offset int64

Df.rmutex.Lock ()

offset = Df.roffset

Df.roffset + + Int64 (Df.datalen)

Df.rmutex.Unlock ()

Read a block of data

RSN = Offset/int64 (Df.datalen)

Df.fmutex.RLock ()

Defer Df.fmutex.RUnlock ()

Bytes: = Make ([]byte, Df.datalen)

_, Err = Df.f.readat (bytes, offset)

If Err!= nil {

Return

}

d = bytes

Return

}

As you can see, we used the Rmutex field when reading and updating the read offset. This guarantees the two lines of code that might run in multiple goroutine at the same time:

Copy Code code as follows:

offset = Df.roffset

Df.roffset + + Int64 (Df.datalen)

The execution of the is mutually exclusive. This is what we need to do to get a repeat and correct read offset.

On the other hand, when we read a block of data, we do the reading lock and read unlock of the Fmutex field in a timely manner. The two operations of the Fmutex field guarantee that we are reading a complete block of data here. However, this complete block of data is not necessarily correct. Why would you say that?

Please imagine such a scene. In our program, there are 3 goroutine to execute the Read method of a *mydatafile type value concurrently, and there are 2 goroutine to execute the write method of that value concurrently. With the first 3 goroutine, the data blocks in the data file are read sequentially. However, because the goroutine of the write operation is less than the goroutine of the read operation, the value of the offset roffset will be equal to or even greater than the value of the write offset woffset. In other words, the read operation will soon have no data to read. This situation causes the second result value returned by the Df.f.readat method above to represent the nil of the error and will be equal to the io.eof value. In fact, we should not think of such a value as an erroneous representation, but rather as a boundary condition. Unfortunately, we did not deal with this boundary situation properly in this version of the Read method. In this case, the method returns the error value directly to its caller. The caller gets the serial number of the data block that read the error, but cannot attempt to read the block again. Because other read methods that are being or subsequently executed continue to increase the value of the reading offset Roffset, it is possible to read only the other blocks of data that follow this data block when the caller calls the Read method again. Note that the more times you run the Read method, the more data blocks you will be missing. To solve this problem, we wrote a second version of the Read method:

Copy Code code as follows:

Func (DF *mydatafile) Read () (RSN int64, D Data, err error) {

Read and update read offsets

Omit several statements

Read a block of data

RSN = Offset/int64 (Df.datalen)

Bytes: = Make ([]byte, Df.datalen)

for {

Df.fmutex.RLock ()

_, Err = Df.f.readat (bytes, offset)

If Err!= nil {

If err = = Io. EOF {

Df.fmutex.RUnlock ()

Continue

}

Df.fmutex.RUnlock ()

Return

}

d = bytes

Df.fmutex.RUnlock ()

Return

}

}

In the previous Read method display, we omitted several statements. The reason why the statements in this position do not change anything. In order to save space further, we will follow the principle of omission in the back.

The second version of the Read method uses a For statement to achieve the goal of continuing to try to get the same block of data when the Df.f.readat method returns a io.eof error, until the success is obtained. Note that if the read lock is Fmutex in the reading lock while the for code block is executed, the write lock for it will never succeed and the corresponding goroutine will be blocked. Because they are mutually exclusive. So, we had to add a read-unlock operation for the read-write lock in front of each return statement and continue statement in the For statement block and a read lock on the Fmutex at the beginning of each iteration. Obviously, this code looks ugly. Redundant code can greatly increase the maintenance cost and error probability of the code. And, when code in the for code block triggers runtime panic, it is difficult to read and write lock Fmutex in a timely manner. Even if you can do that, it will make the Read method more ugly to implement. We removed the defer df.fmutex.RUnlock () statement because we were dealing with a boundary condition. This approach has mixed advantages and disadvantages.

In fact, we can do better. But this involves other synchronization tools. Therefore, we will later on the Read method of further transformation. Incidentally, when the Df.f.readat method returns an error value that is not nil and not equal to io.eof, we should always discard the attempt to get the target block again and immediately return the error value to the caller of the Read method. Because such a mistake is likely to be serious (for example, the file represented by the F field is deleted), it needs to be handled by the upper program.

Now, let's consider the *mydatafile type of write method. The implementation of the Write method is simpler than the Read method. Because the latter does not involve boundary conditions. In this approach, we need to take two steps: Get and update the write offset and write a block of data to the file. We directly give the implementation of the Write method:

Copy Code code as follows:

Func (DF *mydatafile) Write (d Data) (WSN Int64, err error) {

Read and update write offsets

var offset int64

Df.wmutex.Lock ()

offset = Df.woffset

Df.woffset + + Int64 (Df.datalen)

Df.wmutex.Unlock ()

Write a block of data

WSN = Offset/int64 (Df.datalen)

var bytes []byte

If Len (d) > int (df.datalen) {

bytes = D[0:df.datalen]

} else {

bytes = d

}

Df.fmutex.Lock ()

Df.fmutex.Unlock ()

_, Err = Df.f.write (bytes)

Return

}

It should be noted here that when the value of the parameter d is greater than the maximum length of the block, we will first truncate and then write the data to the file. Without this truncated processing, the serial number of the read block and the serial number of the written block that we calculated later will be incorrect.

With the experience of writing the previous two methods, we can easily write the *mydatafile type of RSN method and the Wsn method:

Copy Code code as follows:

Func (DF *mydatafile) RSN () Int64 {

Df.rmutex.Lock ()

Defer Df.rmutex.Unlock ()

Return Df.roffset/int64 (Df.datalen)

}

Func (DF *mydatafile) WSN () Int64 {

Df.wmutex.Lock ()

Defer Df.wmutex.Unlock ()

Return Df.woffset/int64 (Df.datalen)

}

The implementation of these two methods involves the locking operation of mutex Rmutex and Wmutex respectively. At the same time, we also guarantee the timely unlocking of them by using the defer statement. Here, we compute the serial number of the read Block RSN and the serial number of the written block WSN the same way as in the previous example. They are the value of the corresponding serial number (or count) by the quotient of the associated offset divided by the length of the block.

As for the implementation of the *mydatafile type of Datalen method, we do not need to render. It simply returns the value of the Datalen field as its result value.

The main purpose of writing this complete example is to demonstrate the application of mutexes and read-write locks in the actual scenario. Since there are no other sync tools available in the Go language, all the places we need to sync in the relevant methods are locked. However, some of these problems are not sufficient or appropriate to be solved with a lock. We will improve them incrementally in the remainder of this section.

The

can be seen from the source of both locks, which are homologous. The internal of the read-write lock is a mutex to achieve mutual exclusion between write-lock operations. We can think of read and write locks as an extension of mutual exclusion locks. In addition, both of these lock implementations use the synchronization tool provided by the operating system--the semaphore. A two-value semaphore (a semaphore with only two possible values) is used inside the mutex to achieve a mutex between the locking operations. While the read-write lock uses a two-value semaphore and a multivalued semaphore (can have several possible values of the semaphore) to achieve a write-lock operation and read-lock operation of mutual exclusion. Of course, for precise coordination, they also use some of the other fields and variables. Because of the space reason, we are not here to repeat. If the reader is interested in this, you can read the Sync code package in the relevant source files.