Golang Transaction Model

This is a creation in Article, where the information may have evolved or changed.

Preface

In the evolution process of software design, the author finally evolves the DDD hierarchical model of communication system software into a five-layer model, that is, the dispatch layer (Schedule), the transaction layer (Transaction DSL), the Environment layer (context), the domain layer (domain) and the infrastructure layer ( Infrastructure), let's take a brief look at:

Ddd-layer-with-dci-dsl.png

1. Scheduling layer: The state model of the UE is maintained, including only the essential state of the business, sending the received message to the transaction layer.
2. Transaction layer: Corresponds to a business process, such as UE Attach, that combines the processing of individual synchronous messages or asynchronous messages into a single transaction, which is rolled back when a transaction fails. When the transaction layer receives a message from the dispatch layer, the action of the environment layer is delegated for processing.
3. Environment layer: In action, process a synchronous message or asynchronous message, cast the domain object of the domain layer into the appropriate role, and let role interact to complete the business logic.
4. Domain layer: Includes not only the modeling of domain objects and their relationships, but also the explicit modeling of role roles of objects.
5. Basic implementation layer: provides common technical capabilities for other tiers, such as message communication mechanisms, object persistence mechanisms, and common algorithms

This article focuses on the transaction layer, which focuses on the transaction model, the code abstraction level and the business flowchart one by one counterpart.

Synchronization model

There is no doubt that the asynchronous model is complex. But in the components of the admin domain, there is no extreme requirement for real-time and performance, and the co-process (for example, Goroutine) is very lightweight, so using the synchronization model is a very smart and simple way to handle it, as shown in:

Synchronous-model.png

In a synchronous model, once a system sends a request message and waits for its answer, the current process goes into hibernation until the reply message arrives or times out. The process can be seen as a user-state lightweight thread that consumes very little resources and that the current system can run on hundreds of simultaneous processes.

Assuming that the action is an interaction of a synchronous message, the business flowchart corresponds to an action sequence.

Transaction

The term transaction (Transaction, trans) derives from the concept of data processing, and the following is the definition of a transaction by Wikipedia:

In computer science, transaction processing are information processing thatis divided into individual, indivisible operatio NS, called transactions. Each transaction must succeed or fail as a complete unit; It cannot remain in a intermediate state.

In general, a user flowchart for a single scenario corresponds to a transaction, and the transaction consists of an action sequence.

Transaction.png

A synchronous request processing from S1 to S2, standing in the S1 perspective is an action, while the S2 perspective is a transaction.

Transaction Process Control

Basic data structure

Transinfo

Transinfo is a very important data structure in the transaction model, which is used for data transfer during transaction execution, such as data injected into the environment layer by the transaction layer, and data in series between actions.

S1obj

The current system is S2, and when a synchronization request is received from S1, S2 initiates a process that processes the request. When the process calls to the dispatch layer, it is necessary to initialize the data variables transinfo and create the Realm object s1obj, then inject them into the transaction object, and finally execute the transaction. If the transaction fails to execute, it is rolled back.

func scheduleS1ReqTrans(req []byte) error {    transInfo := &context.TransInfo{Names: make([]string, 0)}    S1Obj := s1obj.CreateS1Obj(req)    s1ReqTrans := trans.NewS1ReqTrans()    err = s1ReqTrans.Exec(S1Obj, transInfo)    if err != nil {        s1ReqTrans.RollBack(S1Obj, transInfo)    }    return err}

Fragment

At the semantic level, a fragment is a process fragment.
At the level of code, a fragment is a interface.

type Fragment interface {    Exec(s1Obj *s1obj.S1Obj, transInfo *context.TransInfo) error    RollBack(s1Obj *s1obj.S1Obj, transInfo *context.TransInfo)}

Action

Action is the interaction of a synchronous message, defined in the environment layer, is provided to the transaction layer of atomic operations, is a fragment, implements the fragment interface, the implementation is closely related to the business.

Procedure

Procedure is the interaction of many closely related synchronous messages, defined in the transaction layer, is a more complex unit than action, and is a fragment that implements the fragment interface.
Procedure itself is a sequence of action or procedre, where action is a leaf node, procedure is an intermediate node, so Procedre is a multi-fork tree.

A generic procedure code is defined as follows:

type Procedure struct {    fragments  []Fragment}

Create a specific procedure code as follows:

func newA1Procedure() *Procedure {    a1Procedure := &Procedure{        fragments: []Fragment{            new(context.Action11),            newA2Procedure(),            new(context.Action12)}}    return a1Procedure}

The execution of the procedure is simple, and the primitive language for_each_fragments can be called directly from the transaction layer package.

Repeat

From a semantic level, repeat is used to decorate an action or procedure, stating that the action or procedure can be executed more than once and at least once.
From the code hierarchy, repeat is also a fragment because it implements the interface.
Repeat, in essence, is the packaging of action or procedure, as well as fragment behavior.

The code for repeat is defined as follows:

type repeat struct {    fragment Fragment}

The number of executions of repeat is dynamically determined by the previous action written to Transinfo.

With repeat, we can define a transaction as follows:

func NewS1Trans() *Transaction {    s1Trans := &Transaction{        fragments: []Fragment{            new(context.Action1),            new(context.Action2),            repeat{fragment:newA1Procedure()},            new(context.Action3)}}    return s1Trans}

Optional

Optional is similar to repeat ^-^.
From a semantic level, optional is used to decorate an action or procedure, stating that the action or procedure executes at most once and can not be performed.
From the code hierarchy, optional is also a fragment because it implements the interface.
Optional, in essence, is the packaging of action or procedure, as well as fragment behavior.

The number of executions of optional is determined by the predicate specification, specification is a interface, which is defined as follows:

type Specification interface {    Ok(s1Obj *s1obj.S1Obj, transInfo *context.TransInfo) bool}

Instances of predicates come from two aspects of acknowledgment:

1. Whether a switch on the system is turned on, that is, when the switch is on, the predicate is true, the action or procedure is executed once, or 0 times.
2. Whether the current state of the system satisfies a condition, that is, when the condition is satisfied, the predicate is true, the action is executed once or procedure, or it is executed 0 times.

With optional, we can define a transaction as follows:

func NewS1Trans() *Transaction {    s1Trans := &Transaction{        fragments: []Fragment{            new(context.Action1),            optional{spec:new(context.ShouldExecAction2),                fragment:new(context.Action2)},            new(context.Action3),            repeat{fragment:newA1Procedure()},            new(context.Action4)}}    return s1Trans}

Default

From the semantic level, an action or procedure that does not have a repeat or optional modifier is the default, stating that action or procedure is executed only once.

Transaction rollback

For a transaction, execution succeeds or fails. When a transaction fails, a rollback must be triggered to cause the system to leak or remain without resources.
When the execution of a transaction fails, it is certain that a fragment execution fails, as we remember Fragments[i], and that the transaction rollback process is:

1. Fragments[i] To complete the reclamation of its assigned resources and to clean up the data it has written;
2. From fragments[i-1] to Fragments[0], call its rollback method in turn.

Action

The action is the atomic performer of the transaction, and from the leaf node, the transaction is the action sequence.
When an action execution fails, the action-related resource reclamation or data cleanup is done within the Exec method, and the action's rollback function is not called.
The rollback method implementation of the action is simple, with only all the resource reclamation and data cleanup related to that action.

As an example:

Action5 before execution fails, the file File1 is opened and a record is written in the table table1, then it deletes the record in table table1 before returning the error, and closes the file file1, that is, in reverse order for resource recycling and data cleansing.
As for Action1 to Action4 to open what resources or write what data, Action5 not care at all.
ACTION5 returns an error, the transaction rollback framework automatically calls the rollback function of [Action4,action3, Action2, Action1] in turn to complete the rollback of the transaction.

Procedure

If the procedure execution fails, "error handling" is performed in the Exec method:

func (this *Procedure) Exec(knitterObj *knitterobj.KnitterObj, transInfo *context.TransInfo) error {    index, err := for_each_fragments(this.fragments, knitterObj, transInfo)    if err != nil {        if index <= 0 {            return err        }        back_each_fragments(this.fragments, knitterObj, transInfo, index)    }    return err}

The Exec method uses the primitive language for_each_fragments and back_each_fragments of the transaction layer in the implementation:

1. For the for_each_fragments primitive, it has been mentioned in the transaction Process Control section that a forward traversal of fragments is invoked, calling its Exec method.
2. For the back_each_fragments primitive, first subtract one (index--) from the entry index (the last argument), and then, starting with index, the reverse traversal of fragments, calling its rollback method.

If procedure executes successfully, the rollback method is called directly when rolling back:

func (this *Procedure) RollBack(knitterObj *knitterobj.KnitterObj, transInfo *context.TransInfo) {    back_each_fragments(this.fragments, knitterObj, transInfo, len(this.fragments))}

Repeat

If the repeat execution fails, "error handling" is performed:

func (this repeat) Exec(knitterObj *knitterobj.KnitterObj, transInfo *context.TransInfo) error {    for i := 0; i < transInfo.Times; i++ {        transInfo.RepeatIdx = i        err := this.fragment.Exec(knitterObj, transInfo)        if err != nil {            if i == 0 {                return err            }            i--            for j := i; j >= 0; j-- {                transInfo.RepeatIdx = j                this.fragment.RollBack(knitterObj, transInfo)            }            return err        }    }    return nil}

The transinfo.repeatidx here need to explain:

1. A value of I is assigned before this.fragment.Exec to find the corresponding domain object when repeat executes action or procedure.
2. This.fragment.RollBack was assigned a value of J before it was repeat to find the corresponding realm object when "error handling" was rolled back when the action or procedure was completed. For example, for example, the maximum number of repeat is 5, when an error occurs to the 4th time, you need to roll back the first three action or procedure.

If repeat executes successfully, the rollback method is called directly when rolling back:

func (this repeat) RollBack(knitterObj *knitterobj.KnitterObj, transInfo *context.TransInfo) {    for i := transInfo.Times; i >= 0; i-- {        transInfo.RepeatIdx = i        this.fragment.RollBack(knitterObj, transInfo)    }}

Optional

Optional is simpler, and if an error occurs during execution, nothing is done because the action or procedure has completed error handling as follows:

func (this optional) Exec(s1Obj *s1obj.S1Obj, transInfo *context.TransInfo) error {    if this.spec.Ok(s1Obj, transInfo) {        this.isExec = true        return this.fragment.Exec(s1Obj, transInfo)    }    return nil}

If optional executes successfully, the rollback requires an action or procedure rollback based on whether an action or procedure has been performed, as follows:

func (this optional) RollBack(s1Obj *s1obj.S1Obj, transInfo *context.TransInfo) {    if this.isExec {        this.fragment.RollBack(s1Obj, transInfo)    }}

transactional concurrency

The execution of a transaction is a synchronous model, and the transactions are asynchronous. Resources may be shared among multiple transactions, so concurrency control is performed on the transaction.
In Golang, the concurrency control between the threads is typically used with channel, which is very simple and efficient.
If a set of processes uses a shared resource, then a channel control is used, so that multiple sets of processes need to be controlled by more than one channel. We can use Map,key as the channel for Shareid,value.

Read Channel

Depending on the orchestration, the channel is read in a specification (predicate, the first parameter of optional). Assuming the predicate is issomethingnotexist, the sample code is as follows:

func (this *IsSomethingNotExist) Ok(s1Obj *s1obj.S1Obj, transInfo *TransInfo) bool {    ...    <- transInfo.Chan    transInfo.ChanFlag = true    ...}

To read the channel, you must first inject it. Based on the localization principle, we inject in the predicate issomethingnotexist, not in the previous action or specification, and the sample code becomes:

func (this *IsSomethingNotExist) Ok(s1Obj *s1obj.S1Obj, transInfo *TransInfo) bool {    ...    concurrencyctrl.ChanMapLock.Lock()    value, ok := concurrencyctrl.ChanMap[shareId]    if ok {        transInfo.Chan = value    } else {        transInfo.Chan = make(chan int, 1)        transInfo.Chan <- 1        concurrencyctrl.ChanMap[shareId] = transInfo.Chan    }    concurrencyctrl.ChanMapLock.Unlock()    <- transInfo.Chan    transInfo.ChanFlag = true    ...}

Write Channel

Depending on the business process, the channel is written in an action after the specification of the channel is read. Assuming the action is discussaction, the sample code is as follows:

func (this *DiscussAction) Exec(s1Obj *s1obj.S1Obj, transInfo *TransInfo) error {    ...    transInfo.Chan <- 1    transInfo.ChanFlag = false    ...}

The attentive reader may have discovered that the description above "to write channel in an action after reading the channel's specification" exists in two cases:

1. The specification is the first argument of optional, and the action or procedure containing the action is the corresponding second argument
2. The action is followed by the specification corresponding to the optional operation

Regardless of whether the specification OK method returns True, the second case will always be the write channel operation, and the first case is not necessarily, that is, when the specification OK method returns false, the write channel operation is not performed. So there are flaws. The remedy for this flaw is to be judged within the OK method of the specification, and if the return value is False, the write channel operation is performed. Assuming that the predicate is Issomethingneeddel, the sample code is:

func (this *IsSomethingNeedDel) Ok(s1Obj *s1obj.S1Obj, transInfo *TransInfo) bool {    ...    concurrencyctrl.ChanMapLock.Lock()    value, ok := concurrencyctrl.ChanMap[shareId]    if ok {        transInfo.Chan = value    } else {        transInfo.Chan = make(chan int, 1)        transInfo.Chan <- 1        concurrencyctrl.ChanMap[shareId] = transInfo.Chan    }    concurrencyctrl.ChanMapLock.Unlock()    <- transInfo.Chan    transInfo.ChanFlag = true    ...    if flag {        log.Infof("***IsSomethingNeedDel: true***")    } else {        transInfo.Chan <- 1        transInfo.ChanFlag = false        log.Infof("***IsSomethingNeedDel: false***")    }    return flag}

Error and exception handling

During the execution of a transaction, whether it encounters an error or an exception (panic), there may be a case where the channel read is not written, that is, the closed operation of the channel is not implemented during the transaction, which causes other threads (Goroutine) of the group to be blocked.

The solution to this problem is to capture the exception using the defer modified closure in the entry method of the transaction dispatch, while attempting to close the channel for errors or exceptions, the sample code is as follows:

func scheduleS1ReqTrans(req []byte) （err error) {    transInfo := &context.TransInfo{Names: make([]string, 0)}    defer func() {        if p := recover(); p != nil {            str, ok := p.(string)            if ok {                err = errors.New(str)            } else {                err = errors.New("panic")            }            log.Info("S1ReqTrans panic recover start!")            log.Error("Stack:", string(debug.Stack()))            log.Info("S1ReqTrans panic recover end!")        }        if transInfo.ChanFlag {            transInfo.Chan <- 1        }    }()    S1Obj := s1obj.CreateS1Obj(req)    s1ReqTrans := trans.NewS1ReqTrans()    err = s1ReqTrans.Exec(S1Obj, transInfo)    if err != nil {        s1ReqTrans.RollBack(S1Obj, transInfo)    }    return err}

Summary

In the components of the admin domain, there is no extreme requirement for real-time and performance, and the goroutine is very lightweight, so using a synchronization model is a very smart and simple way to handle it. The transaction model discussed in this paper is based on the synchronization process, the process control of the transaction is elaborated, then the general design framework is given for the rollback of the transaction, and finally the simple and efficient solution is given for the concurrency control of the transaction. The transaction model is located on the fourth level in the hierarchical architecture of DDD, with a high level of code abstraction and strong expression, corresponding to business flowchart one by one, and the code can be reused with action or procedure for granularity.