Step by Step-build your own ORM series-data access layer

Document directory

• I. Opening
• Ii. Summary
• Iii. Outline
• Iv. Analysis of data access layer in ORM
• V. Implementation of ORM-related code
• Vi. Summary in this Chapter
• VII. Series progress
• 8. next announcement

I. Opening

It's been too long before the previous Step by Step-build your own ORM series-the beginning of the article. I can't help you because you have to write several series in advance, to continue this series.

In the process of writing these series, I had a new understanding and improvement on the original Implementation ideas IN THE ORM series. Of course, these are not very advanced ideas or understandings, it may also be the idea you have seen. I hope I can

During the writing of the Design Pattern series, I interspersed with the ORM series. Of course, the series I built can only be said to be simple. I usually develop a small application tool or something, he may be used because it was developed by himself. After all

It's easy to use! In line with your own operating habits.

Of course, when I write this series, there will also be some areas of extreme understanding or incorrect ideas. Please point out and criticize them more. I also learned a lot of valuable experience in my current project. In fact

We should be able to see the convenience and inconvenience provided by the orm. We take its essence and eliminate the dregs, but this is really difficult. In fact, I have not studied much about the underlying implementation of some popular ORM. For example, nhib.pdf, I

I only learned about Hibernate, but I learned it from JAVA at the time. If I don't know it deeply, the Castle framework has been used for a while. EntityFreamWork has never been used. It's just a symbolic download of the latest version, experience the AOP

Method, I think there are many times when we use the AOP method to improve the flexibility of our program. This may require more guidance from the experts.

My ideal ORM is to achieve persistence and transparency. How can this be understood? That is to say, during program development, I do not want to write the corresponding persistent operations in the Business Code, nor do I care how to call your

ORM to complete CRUD operations. I only care about my business logic. This is a bit like the Domain Layer in DDD (domain-driven development). I only care about the business logic inside the domain, but not about other things, this makes it easy for us to quickly

And do not affect the implementation of business logic.

Ii. Summary

This article describes the data access layer in Step by Step-build your own ORM series-data access layer, in fact, this article is to teach you how to complete

Operations on the data access layer in ORM, which provides a unified data access method. Of course, operations here are mainly concentrated on database operations, including how to return the object list based on the object, including generating SQL statements

The implementation of several types of statements also includes the maintenance of subsequent ORM configuration management. Here we provide the configuration of visual XML files. How can we do this? Because we usually use ORM directly

XML file, we can provide a visual interface for the user to configure these settings. Through these configurations, we can achieve smooth database migration and cache application configuration, including some other related settings

. In general, these operations can all be performed through the data access layer.

Let's take a look at the important role and position of the data access layer in ORM:

We know that the basis of all our related functions is based on the data access layer, so we have processed the relevant logic of this layer.

Later, the problem will be easier to carry out. Next we will start to address these questions

Solve the related functions that our data access layer should provide one by one! The general functions should have the following functions:

1. Persistent operation method CUD. It can be expanded to provide related automatic script tools, such as creating tables and modifying other tables. Provides a persistent and transparent method.

2. the cache service is provided to cache the corresponding ing information of the object. This will greatly improve the efficiency of subsequent operations such as generating statements.

3. When processing objects, we should be able to process information that can only be changed. If there is no change, do we not need to perform the Update operation? Reduce the number of database operations.

4. provides basic query methods. All future ORM-based queries will be extended based on this query. Provides a persistent and transparent query method.

5. Concurrency and transaction control. Here, we may provide an internal version number. Once this object has been modified or changed, we will perform operations at any time on this version number record.

Use internal methods.

Iii. Outline

1. Start.

2. Summary.

3. Outline of this article.

4. Data access layer analysis of ORM.

5. ORM-related code implementation.

6. Summary in this chapter.

7. Series progress.

8. next announcement.

Iv. Analysis of data access layer in ORM

First, let's look at the above issues. We will give Implementation ideas to analyze the feasibility of the ideas and how to resolve them. The specific code section describes the core implementation scheme.

4.1. provides general persistent operations

This specific analysis has already given the corresponding ideas in the previous article. We will do it by providing corresponding methods at the underlying layer. In general, for the four operations corresponding to the database, we also provide the corresponding statement at the data access layer.

The automatic construction process and specific construction are implemented through features. The specific database field, type, length, and other column parameters are defined in the features. I will not repeat it here. I will analyze it here.

Benefits of this implementation. We know that the inheritance method is quite good. Why do I say this? By providing a base class, the base class defines the general CUD operation method, as long as it inherits the subclass of this class, there will be CUD operations

But in order to provide a persistent and transparent solution, there is no doubt that for persistent operations, we do not want the business logic objects in the business logic layer to complete. How can we do this? Through the data access layer

Allows the service layer to perform persistent operations on business objects. In this way, we can achieve a persistent and transparent solution.

Therefore, we can do this. In the data access layer, we provide an interface that defines several solutions for persistent operations. Through different implementation configurations, we can specify it in the XML configuration file.

The heavyweight ORM is also a lightweight ORM, so we can ideally implement the features of low coupling.

At the same time, for different file operations, we can support Writing of multiple file types. Of course, for different database storage, if we use relational databases, we need ORM and object database operations, or

The operations on XML files have changed our ORM at this time.

4.2 provide cache service

We know that in the first article we mainly deal with it through custom features + reflection: we have not provided complete feature operations, but there are still many situations, such, we can also provide mappings to views

For the features of a view. There are many other features, and the complete code structure will be provided in the future. We know that custom features + reflection, if every time you map an object to a database table, how low is efficiency?

Ah, in this case, we can consider using the cache method to map database columns in the database table to attribute columns in the object. We can put these mappings in the cache, if we encounter

When database-related operations require object ing, we first go to the cache to find whether the ing database table column with the specified key value exists, and take out the generated SQL statement, otherwise, the corresponding database table is retrieved through reflection.

Column in the cache. Below. Describe the process:

Of course, what I will provide here is definitely a reverse SQL statement that generates operation data based on objects. It is not considered here, when the number

When the database table changes, I should automatically synchronize the ing set information in the cache. Of course, we can implement this bidirectional synchronization problem through certain policies. For example, the following method may be feasible.

, Through the above method, I

Through the synchronization component, we can apply better policies before each data operation, such as record or traverse the modification status of the file, and compare the last modification date of the file to see if any modification has occurred, or when a file is repaired

After the modification, we record it in a configuration file, so that we can improve the synchronization efficiency, because in this way, we do not need to check whether the object has changed every time, in this way, if we find that the object has not changed,

So we should not let the synchronization component check whether the object has changed, so as to improve efficiency, and support when the ing object changes, we do not need to modify our relational database. As we all know, object-oriented design and building

The biggest challenge of the model and relational database service is that it is difficult to determine the synchronous solution of changes between the two sides. Here I just give a simple idea and method, and there may be better solutions, please tell me more.

4.3 Update statement operations

I don't know. During the interview, if you often develop the underlying object-oriented ORM, you may encounter this problem. When we query a ing object, we may only need to retrieve

How do we specify some columns instead of all data columns? When filling the object, we only need to fill in the specified column? Or we need to keep the unchanged data while saving and editing.

Library, in fact, is mainly required. When we generate SQL statements, we hope that the update statements do not appear and there is no change in the data column settings. How can this be done? I think we can use the following two methods:

.

1. Copy an object through object serialization, and cache the object in the system. Before editing, cache the object and release it after submission. This is provided by the system by default. The premise is that the object is edited.

In this way, copy the object before it is modified. Otherwise, it makes no sense to copy the object after modification.

2. Use a dictionary to store data columns with updated values. We store the ing data columns in the dictionary and mark the changed data columns and the values of the Data columns. We are generating a new language.

. Directly traverse this set and generate corresponding operation statements for columns whose column status changes. These are all feasible methods, which we mentioned earlier in the architecture design. I have provided some ideas, and I will not repeat them here.

.

3 ,.... There may be other better methods. Please come up with some good ideas. Here is my remarks!

4.4. Basic query methods are provided.

The basic query method is based on database operations. We provide several common query methods, including the method for Automatically Generating version numbers, our version number can be in the form of date + serial number

Generate or otherwise. GUID is also a feasible method. However, maintenance may not be very convenient. Therefore, it is easier to provide corresponding operation methods at the underlying layer.

Here we will provide the following basic query methods and complex query methods. We can provide an entry for doing this. For example, we provide the following methods:

1. The method for generating the version number at the underlying layer, the ID serial number, and the ID generation rule customized according to different generation rules to organize the general method for generating the ID.

2. Provides conversions between entities and database row sets. We need to convert database records to entity sets. Returns an object set through the query method. Returns the object set of the specified Primary Key.

3. Return all records in a data table or view.

4. Return the number of incoming pages, the paging sorting fields, and the paging set of paging conditions.

5. Return the query method of the specified column. (I didn't think of a good solution here. How can we flexibly specify the returned columns, for example, 1 column, 10 columns, and 5 columns?) I hope you can provide good comments and suggestions!

6. provides a unified portal for writing SQL statements to be passed in to the data access layer for query and retrieval. Generic objects are returned Based on the specified return type.

4.5 concurrency and transaction control

I think what must be considered in a system is transaction processing. When we perform batch operations, if the data is not synchronized, it is too painful and unusable, we hope our ORM can be automatically integrated.

Transactions and concurrency, of course, the concurrency mentioned here is that when the number of users reaches a certain level, this problem will occur. Theoretically, as long as there are more than two users, you must consider concurrent operations! Concurrency we have several control ideas,

In general, it should be said that an automatically generated version number in the previous design is the best choice. What does it mean? Let's explain:

For concurrency control, we know that the concurrency control problem is: The write is lost, and the read data is dirty data, which is undoubtedly the two common problems, so how can we limit the write loss? General Solutions

Both are handled through optimistic locks. Two people can read a certain information at the same time, but only one person can edit it. However, the last modified content will overwrite the previously modified information, this is an optimistic lock processing method.

The pessimistic lock is that as long as someone is modifying it, you may not be able to modify or read it. In this way, of course, the synchronization and consistency of information modification can be ensured, however, the user's ease of use and friendliness are not human-friendly.

In contrast, if someone modifies it, it cannot be modified by others. However, the reading experience is poor, but there are different use cases. In general, pessimistic locks are a supplement to optimistic locks.

We are neither optimistic locks nor pessimistic locks here. We use a version to record the version of a record. Once a change occurs, the recorded version will change, here we will update the version of this row set.

You can generate a version number using the version generation rules provided by ORM, or use a trigger. Of course, performance is also part of our consideration.

For transactions, I think it is not a distributed operation application. We use the transaction service provided by the database to basically meet our daily needs, there is nothing that is particularly difficult. I think I will

To put it bluntly, let's simply talk about the consistency of distributed transactions. For such distributed transaction operations, we can adopt the offline concurrency mode for processing. How can this be understood? It is implemented through work units. We set

Each operation looks at a unit of work. If we return 0 or other results that are not the expected results during the execution of a transaction operation, we will not commit any operations, if all operations pass,

We cyclically commit all the units of work, otherwise we will roll back all the system transactions. We regard such a distributed transaction as a business transaction, which is composed of the worker units of some columns, which are regarded as system transactions.

V. ORM-related code implementation 5.1. Basic implementation code of CUD:

1. Implementation of the Create statement:

Private Dictionary <string, Column >_autoincrementcolumns = new Dictionary <string, Column> ();
Private Dictionary <string, Column> _ updateColumns = new Dictionary <string, Column> ();

Public string TableName
{
Get
{
Return string. Empty;
}
}

Public Dictionary <string, Column> UpdateColumns
{
Get
{
Return this. _ updateColumns;
}
Set
{
This. _ updateColumns = value;
}
}

Public Dictionary <string, Column> AutoIncrementColumns
{
Get
{
Return this. _ autoIncrementColumns;
}
Set
{
This. _ autoIncrementColumns = value;
}
}

Public virtual IDbCommand GetDbCommand ()
{
// If the column value is not updated, null is returned.
If (this. UpdateColumns. Count = 0)
{
Return null;
}

ArrayList fieldList = new ArrayList ();
ArrayList valueList = new ArrayList ();
SqlCommand cmd = new SqlCommand ();

Foreach (Column column in this. UpdateColumns. Values)
{
FieldList. Add ("[" + column. Key + "]");
ValueList. Add ("@" + column. Value );
}

String fieldString = string. Join (",", (string []) fieldList. ToArray (typeof (string )));
String valueString = string. Join (",", (string []) valueList. ToArray (typeof (string )));
String plain text = string. Format ("insert into [{0}] ({1}) VALUES ({2 })",
This. TableName,
FieldString,
ValueString );

String sqlGetIndentityID = null;

If (this. AutoIncrementColumns. Count = 1)
{
SqlGetIndentityID = string. Format ("SELECT [{0}] = SCOPE_IDENTITY ()");
}

If (sqlGetIndentityID! = Null)
{
Plain text = plain text + ";" + sqlGetIndentityID;
}

Cmd. CommandText = plain text;

Return cmd;

}
}

 

The following is an Update Statement, which is edited from the above Update set and generated by adding the columns whose status has changed to the statement. The sample code is as follows:

Public virtual IDbCommand GetDbCommand ()
{
// If the column value is not updated, null is returned.
If (this. UpdateColumns. Count = 0)
{
Return null;
}

ArrayList fieldList = new ArrayList ();
ArrayList valueList = new ArrayList ();
SqlCommand cmd = new SqlCommand ();

String updateSQL = string. Empty;
Foreach (Column column in this. UpdateColumns. Values)
{
If (column. State)
UpdateSQL + = "[" + column. Key + "] =" + "@" + column. Value;
}

String plain text = string. Format ("UPDATE {0} SET {1 }={ 2}", updateSQL );

Cmd. CommandText = plain text;

Return cmd;
}

The deletion code is relatively simple. I will not provide the deletion code here, and the form is the same in general.

5.2 cache service code

I have two articles about caching. The most difficult problem in caching is the cache expiration problem. The corresponding reflection performance problem also has an expiration problem. For example, our database table has changed, or the attribute in the object is sent

After the changes occur, the content in our cache must be updated. Otherwise, the database operation statements we generate will be incorrect. Our policy here is to put the mapped objects in the cache on the server. Of course,

The Caching methods and policies that B/S and C/S systems may adopt are different. B/S our cache can be cached to the server or through the cache server, usually through Remoting to cache the server and

The server completes communication. Let's take a look at the simple sample code:

Public class Cache
{
Private static System. Web. Caching. Cache cache = HttpRuntime. Cache; // The service Cache of the current application is used by default.

Public static object Get (string key)
{
Return cache [key];
}

Public static bool Remove (string key)
{
Return! (Null = cache. Remove (key ));
}

Public static void Set (string key, object value)
{
Cache. Insert (key, value, null, System. Web. Caching. Cache. NoAbsoluteExpiration, TimeSpan. FromMinutes (3 ));
}
}

The sample code of the cache class shown above. For specific operations, after reflection is used, the object metadata information after reflection is cached and cached by the Object Name:

The Code is as follows:

PropertyInfo [] property = null;
If (Cache. Get ("")! = Null)
{
Property = (PropertyInfo []) Cache. Get ("");
}
Else
{
Type t = Type. GetType ("");

Property = t. GetProperties ();
}

Through the above few lines of simple code, we can express our above-mentioned ideas. We have also provided some ideas on how to expire. Maybe everyone has a better idea, I will not make an axe here.

5.3. provides basic query services

I think everyone is familiar with the operations on query statements. How can we better complete the unified query service may be a concern of us, I won't provide implementation of multiple databases here, but I can give it to everyone.

One idea is that when we define the returned query command, if we support multiple data items, we can define a unified interface. Different databases provide different implementation interfaces, then, you can call different

Component to generate SQL statements and complete the call operation.

The related query service code is as follows:

/// <Summary>
/// The version number automatically generated by the system
/// </Summary>
/// <Returns> </returns>
Public string GetVersion ()
{
Return DateTime. now. year. toString () + DateTime. now. month. toString () + DateTime. now. day. toString () + DateTime. now. minute. toString () + DateTime. now. second. toString ();
}

Public int GetMax <T> ()
{
// Obtain the maximum column of T based on the T type to complete the query.
String sqlText = "Select MAX (ISNULL (column name, 0) + 1 FROM TableName ";

Return 0;
}

Public List <T> GetAll <T> ()
{
// Obtain the maximum column of T based on the T type to complete the query.
String sqlText = "Select * FROM TableName ";

Return new List <T> ();
}

Public List <T> GetList <T> (string condition, int pagesize, string orderField)
{
// Obtain the maximum column of T based on the T type to complete the query.
String sqlText = "Select * FROM TableName where" + condition + "order by" + orderField;

Return new List <T> ();
}

Not all of the Code provided above, but some of the Code should be completed by yourself. Here I want to see some of the complicated Custom Code passed in through an interface, to call the basic query service.

Here we provide a general interface definition:

CommandType
{
Get;
Set;
}

String whereCondition
{
Get;
Set;
}

String orderCondition
{
Get;
Set;
}

String TableName
{
Get;
Set;
}

Column [] ColumnList
{
Get;
Set;
}

String SQL
{
Get;
Set;
}

The following examples are provided by default:

Public class BaseSQL: ISelect
{
Public System. Data. CommandType
{
Get
{
Throw new NotImplementedException ();
}
Set
{
Throw new NotImplementedException ();
}
}

Public string whereCondition
{
Get
{
Throw new NotImplementedException ();
}
Set
{
Throw new NotImplementedException ();
}
}

Public string orderCondition
{
Get
{
Throw new NotImplementedException ();
}
Set
{
Throw new NotImplementedException ();
}
}

Public string TableName
{
Get
{
Throw new NotImplementedException ();
}
Set
{
Throw new NotImplementedException ();
}
}

Public Column [] ColumnList
{
Get
{
Throw new NotImplementedException ();
}
Set
{
Throw new NotImplementedException ();
}
}

Public string SQL
{
Get
{
Throw new NotImplementedException ();
}
Set
{
Throw new NotImplementedException ();
}
}

The calling code is as follows:

Public class SpecialSQL: BaseSQL
{
Public void Test ()
{
This. TableName = "";
This. SQL = "SELECT * FROM TEST ";
This. whereCondition = "ID = 4 ";
This. orderCondition = "order by id desc ";
}
}

Of course, the inheritance method here is not very recommended. We can use the abstract factory mode to create this query object, and then we are calling this query object, we can customize this SQL query object.

ORM automatically parses and completes the unified query service entry for custom SQL statements.

Of course, if you have a better solution, you can come up with it. Thank you very much!

Vi. Summary in this Chapter

This article mainly describes the data access layer in the orm. For some special reasons, the Code is not particularly detailed. On the one hand, it is because the previous code is lost, now it is difficult to restore, so there are

Some codes do not provide very complete information. Please forgive me. Some of the content in this article has encountered many problems during implementation, please help me solve my specific problems.

I have encountered the following problems during the architecture design process:

1. When implementing the persistent and transparent service at the service layer, I also want to make the query service transparent, which means that the Business Object and service layer are only completed through DTO, all data of the Business Object is accessed by the service layer.

2. If there are complex business logic operation statements, what should I do if I put this SQL statement on the data access layer? Or where? Specific responsibilities should be clearly defined.

3. I just want the business logic layer to process the business data. How does the specific business data come from? I want the business logic to focus only on DTO.

4. For the unified query method provided by the service layer, how can I transmit parameters when calling the performance layer, the traditional approach is no longer under consideration.

Thank you for your comments and suggestions! I believe this ORM series will get better and better!

VII. Series progress

1. Step by Step-build your own ORM series-start

2. Step by Step-build your own ORM series-data access layer

3. Step by Step-build your own ORM series-configure the management layer

4. Step by Step-build your own ORM series-object ing layer [on]

5. Step by Step-build your own ORM series-object ing layer [medium]

6. Step by Step-build your own ORM series-object ing layer [bottom]

7. Step by Step-build your own ORM series-test the ORM framework

8. Step by Step-build your own ORM series-bottleneck and Optimization

9. Step by Step-build your own ORM series-instance

8. next announcement

In the next article, we will explain how to implement the general configuration management layer. We will introduce in detail how to design the general configuration management layer. Through the configuration management layer, according to our actual project needs, dynamically complete the settings and packages of the ORM framework

This includes cache expiration policy settings, database access layer configuration, and other information settings. You are welcome to make suggestions and suggestions.