Clojure STM notes-Part 1

A few years ago, I had time to pay attention to clojure, which focuses on lisp and its concurrent solutions. for the latter, the old hacker recommended a good article: "software transactional memory ". this article is a good starting point. On the one hand, you can learn clojure. On the other hand, you can take this opportunity to review chapter 12th of "programming language-the path to practice" and "concurrency ", it is not limited to the article itself. the article is long and the notes are separated and managed. It is completed in the first, second, and second sections. the pain of concurrency since it is about the concurrency solution, the beginning of the article is a review of the pain of concurrency and the summary of the existing solutions, memories of bitter sweet, water do not forget to dig into the trap. the concurrent scenario introduces many challenges that are not available for single-threaded programming. The root cause of the problem is that the execution sequence of the Code is not fixed, and the problem is difficult to reproduce and debug, software testing requires more effort. the author divides the solution into two categories: one is to distribute jobs in advance to avoid conflicts, and the other is to coordinate the execution of multiple jobs. obviously, the latter is a more common scenario. In this scenario, there are four typical solutions: locks, actors, trsactional memory, and future. This article only mentions the first three solutions, the future model clojure also provides support. the author makes a simple pros and cons analysis on the locks, actors, and trsactional memory mentioned later. Let's look at the Mind Map below: Future, let's first talk about future, which is not mentioned in this article, future outsourced the computation to other threads for completion. the corresponding primitive in clojure is future. let's take a look at the sample code"

user=> (def long-calculation (future (apply + (range 1e8))))#'user/long-calculationuser=> (type long-calculation)clojure.core$future_call$reify__6110user=> @long-calculation4999999950000000user=> (deref long-calculation)4999999950000000

 

The role of future macro is clearly described in Meta:

 

user=> (source future)(defmacro future  "Takes a body of expressions and yields a future object that will  invoke the body in another thread, and will cache the result and  return it on all subsequent calls to deref/@. If the computation has  not yet finished, calls to deref/@ will block, unless the variant of  deref with timeout is used. See also - realized?."  {:added "1.1"}  [& body] `(future-call (^{:once true} fn* [] ~@body)))nil

The @ symbol above is a brief character of the deref operator.

Promise promise is often mentioned together with future. Promise

Promise can parse referenced object values with the timeout feature. The parsing process is blocked until there is a value. promise can be assigned only once. however, promise does not create code or function that ultimately assigns values to variables. promise initializes an empty container and then fills in data through deliver.

user=> (def a (promise))#'user/auser=> (def b (promise))#'user/buser=> (def c (promise))#'user/cuser=> (future  (deliver c (+ @a @b))  (println "Delivery complete!"))#<core$future_call$reify__6110@74aa513b: :pending>user=> (deliver a 101)#<core$promise$reify__6153@23293541: 101>user=> (deliver b 102)Delivery complete!#<core$promise$reify__6153@582b0e7b: 102>user=>

Let's take a look at its implementation:

user=> (source promise)(defn promise  "Alpha - subject to change.  Returns a promise object that can be read with deref/@, and set,  once only, with deliver. Calls to deref/@ prior to delivery will  block, unless the variant of deref with timeout is used. All  subsequent derefs will return the same delivered value without  blocking. See also - realized?."  {:added "1.1"   :static true}  []  (let [d (java.util.concurrent.CountDownLatch. 1)        v (atom d)]    (reify     clojure.lang.IDeref       (deref [_] (.await d) @v)     clojure.lang.IBlockingDeref       (deref        [_ timeout-ms timeout-val]        (if (.await d timeout-ms java.util.concurrent.TimeUnit/MILLISECONDS)          @v          timeout-val))     clojure.lang.IPending      (isRealized [this]       (zero? (.getCount d)))     clojure.lang.IFn     (invoke      [this x]      (when (and (pos? (.getCount d))                 (compare-and-set! v d x))        (.countDown d)        this)))))

 

C # parallel FXThe example of C # parallel FX mentioned in "programming language-the path to practice" is relatively old. The following two examples are used:

Task. factory. startnew () => {console. writeline ("Hello parallel program. ") ;}); parallel. for (0,100, (INDEX) => {index. dump () ;}); // console. writeline ("Hello parallel program. ") is outsourced to other threads for computing, and the Implementation behind the task is the process pool. // The following example uses lazy <task <string> lazytask = new lazy <task <string> () =>{ return new task <string> (() ==>{ console. writeline ("task body working ...... "); Return" task result ";}); console. writeline ("calling lazy variable"); console. writeline ("result from task: {0}", lazydata. value. result );

 

Let's continue to look at the figure. The sTM in this section briefly introduces what is STM first? Definition of Wikipedia: in computer science, software transactional memory (STM) is a concurrency control mechanic analogous to database transactions for controlling access to shared memory in concurrent computing. it is an alternative to lock-based synchronization. A transaction in this context is a piece of code that executes a series of reads and writes to shared memory. these reads and writes logically OCCU R at a single instance in time; intermediate states are not visible to other (successful) transactions. the idea of providing hardware support for transactions originated in a 1986 paper by Tom Knight. the idea was popularized by Maurice Herlihy and J. eliot B. moss. in 1995 NIR Shavit and Dan touitou extended this idea to software-only transactional memory (STM ). STM has recently been the focus o F intense research and support for practical implementations is growing. in terms of performance, the memory modification in the transaction is like the modification committed by the transaction at a certain time point. All the changes before the transaction commit are invisible to other threads. the transaction is executed on a consistent memory snapshot. if the memory data is modified in transaction a but transaction B is committed first, the Code logic of transaction A will be re-executed. STM implements the processing of traditional database transactions in the memory, so it does not guarantee that durable, software crash or hardware failure will lead to data loss, persistence must be implemented through relational databases. TM uses optimistic locks. The execution of each transaction assumes that there is no concurrent write conflict. If this assumption is not true, the transaction will discard all the work that has been done from the beginning. since it is possible to "retry", it is necessary to ensure that the operations contained in the transaction can be undone. if an operation cannot be undone, the entire process is not undone and can be redone. For example, I/O is generated in the middle. clojure The solution packages are refs and agents, which will be mentioned later. transactional memory can distinguish the committed value from the in-transcation state value. In a transaction, the variable value is either the initial value or the value successfully committed by another transaction. the variable value is modified in the transaction. This modification is only visible to the current transaction. the changes made after the transaction is successfully committed are visible to the code outside the transaction. what are the advantages of STM?

• Parallelism to a greater extent, rather than serialization caused by pessimistic locks
• Low Development difficulty no longer consider the execution sequence and lock acquisition, release only needs to pay attention to which variables need to read and write consistent
• This mechanism ensures that the deadlock competition does not appear.

STM problems?

• A large amount of retry results in Waste
• Overhead caused by dedicated maintenance of storage status
• The support tool is missing. You need a tool to check the number of retries of a transaction and the reason for the retry (to facilitate troubleshooting and optimization)

Persistent data structures ?! Transactional memory performs best in mutable/immutable languages. variables can be modified only in transactions. without the support of language infrastructure, developers need to be blessed and ensure that they only modify the variable value in the transaction. clojure's immutable variable is implemented using persistent data structures. What is the relationship between these two concepts? One way to prevent data from being modified concurrently in a concurrent environment is to provide immutable variables, which do not need to be protected if the variables do not change. A common case for a program to process data is to create a new data based on existing data, such as adding an element to the list header. persistent data structures provides a structure basis: it maintains the data version, and the new data structure shares the memory with the existing data. this saves both memory and time. see Wikipedia's explanation of persistent data structures: in computing, a persistent data structure is a data structure that always preserves the previous version of itself when it is modified. such data structures are supported tively immutable, as their opera Tions do not (visibly) Update the structure in-place, but instead always yield a new updated structure. (a persistent data structure is not a Data Structure committed to persistent storage, such as a disk; this is a different and unrelated sense of the word "persistent. ") Take A Look At The Mind Map below: If you can update the latest data, query the historical version data, which is called partially persistent. if any version can be updated and queried, it is called full persistence. all data structures in the pure functional programming language (pure functional) Is fully persistent. STM can be implemented in many ways. clojure implementation is different from that of "data coordinated by transactions cannot be modified outside a transaction. "(the data status to be coordinated in the transaction cannot be modified outside the transaction ). this is a restriction on the language level rather than requiring developers to consciously abide by the rules. here, the next article will continue to study "software transactional memory" and focus on the basic primitives of clojure Language Processing concurrency. happy New Year! [1] race condition beyond the transactional memory/garbage collection analogy [3] software transactional memory [4] persistent data structures [5] persistent data structure and its applications. The last part is a thumbnail, happy New Year!