Analysis of data access layer technology under. net

Introduction

Since. Net was truly a developer, the word "efficiency" has become a huge numberProgramA hot topic. Whether it is from the Development Mode (Cross Language), the system framework (. NET Framework), or a variety of convenient tools (vs. Net), all reflect its superior.

At the same time, on the other hand, whether. Net can be truly competent for enterprise application development is still controversial.

Generally, there are many considerations for an enterprise-level application, such as security, performance, scalability, and ease of use. In this article, the author is more willing to discuss with you. NET data access layer related technologies, which may be a sensitive topic that has been widely concerned since the birth of multi-tier architecture (n-tier architecture). For most developers, this may also be the most frustrating or controversial part of the project.

In the following discussion, for the sake of unification, the author temporarily calls the data access layer (DAL ).

L analyze problems

After a simple statistical analysis, it is not difficult to find that Dal is daunting, not due to technical problems, or even the opposite, many developers think this is one of the least technical aspects (from the perspective of the large and small projects experienced by the author, the development time of this layer is generally short, it is also a "bitter" that many developers do not want to bear, but it is only the architecture needs or some ideas (such as for Dal and Dal) this so-called layer-4 architecture was added (the traditional three-layer architecture did not propose the Dal idea ).

The proposal of Dal indeed posed a huge challenge to the traditional architecture model, and the goal of joining the program is certainly to use it to further improve production efficiency. In this mode, the ideal situation is: since then, most developers have been able to get rid of the difficulties of DBAs, or even completely cut off the direct relationship with the database. The pain of SQL will leave us, and the whole oo world will be quiet.

However, the ideal is the ideal, and whether it can become a reality must pass the project test.

Next, the author tries to analyze several popular and representative solutions to see if they can draw some valuable conclusions and provide some reference for us in designing and implementing the Dal.

U ado. net

First of all, when we talk about the Dal under. net, we immediately see ADO. net.

That's right. Almost all Dal solutions (allow the author to use solution instead

Framework) must develop from it and have no choice, which is also characteristic of. net.

Implementation Method (compared with J2EE ).

Excluding commercial factors and CLR needs, what ADO. Net really brings to us

Not much, it is worth mentioning that dataset (for the projects experienced by the author, more is used

Datatable and dataview ). From Microsoft's early Memory Database (memory database)

Some people are familiar with the popularity of dataset today, and the twists and turns are not just words.

In short, it is certain that the dataset option is available under. net.

The Dal can be as colorful as today, and everyone's ideas can be more open.

Duwamish

There are a lot of good examples in this regard. The most classic is the enterprise-level development package that Microsoft strongly recommends: duwamish.

For those who want to learn about the Dal design under. net, this is a good start.

For a complete analysis, you can refer to "csdn development expert, 2003.11". This article does not

Let's go into details.

The duwamish solution was used in a project that the author was involved in.

Phase, I feel that this is a good reference, and I started to design it without in-depth analysis. Recall now

And found that there are still many shortcomings.

For example, the duwamish solution does not consider Cache Management,

For enterprise-level applications, this is sometimes an issue that has to be considered.

Although duwamish said goodbye to SQL statements (all implemented using Stored Procedures ),

However, the database trace is still very obvious, such as the definition of some field names and associated table names.

.

There is another headache in the development process. At the beginning

Some relatively simple data tables are easy to implement, and some data tables that contain relationships

Our Dal engineers felt the pressure. Later, we almost made DBA again.

I have already done some work during database modeling, but this time I changed the database script

C # implementation (or: Changing the database structure to a dataset with OO characteristics on the surface)

.

Possible, duwamish is a classic implementation, but sometimes it does not mean

Taste best practice. Taking our project as an example, although it is successfully delivered

From the perspective of Type reuse or development efficiency, it is not very successful. Use a buzzword:

In fact, we can do better!

Petshop

Another noteworthy Dal implementation on ADO. NET is the famous petshop.

Of course, similar to duwamish, fame may not be practical. Although petshop

It makes up for duwamish's shortcomings in some aspects, for example, supporting multiple numbers through factory

Data library storage introduces the cache mechanism to provide more convenient SQL helper, but also

There are some other problems. Among them, the most troublesome thing is the introduction of SQL statements, and

Different SQL statements stored in different databases (mainly SQL Server and Oracle

).

On the other hand, although petshop does not use dataset, it is more concise and common

Object (model), but it still converts the result of datareader to include object

In this sense, the image list is used offline. Even,

In some scenarios, such as data filtering or navigation between master and slave data

More inconvenient (in this case, a simple collection or list cannot meet the requirements, DBA

And Dal developers can only provide other methods to achieve the goal ).

From the two examples above, we can see that even in Microsoft's Development Team

No agreement was reached on the Dal issue. I am interested in more details about this.

For more information, seeArticle:

Http://www.microsoft.com/china/community/column/67.mspx.

Practice

The two solutions analyzed above show us their respective advantages and disadvantages, and the complex environment of enterprise-level applications is unlikely to require a universal framework to solve all the problems. Therefore, it can only be analyzed based on specific situations.

The author once participated in the development of a large (. NET) outsourcing project. He was lucky enough to have a glimpse of his Dal design idea and was deeply shocked. I would like to discuss it with you.

Take the SQL server's northwind database as an example. The following is a call based on the Dal.Code(The author made some name adjustments ):

// Return its title based on the employee ID

Boemp = new employeedal ();

Boemp. Keys ["emp_id"] = 1; // note: the actual field name is: employeeid

Boemp. Select ();

String strtitle = boemp ["emp_title"]; // note: the actual field name is: Title

......

// Return all qualified employees based on the City

Boemp = new employeedal ();

Boemp. Keys ["emp_city"] = "Seattle ";

Boemp. Select (); // Note: This method is exactly the same as the preceding call.

Datatable dtemp = boemp. Table;

If you do not consider object creation (Object pooling or cached object can be used) and post-call processing, the actual code is only two lines!

Even more surprising, the above employeedal class does not have any real implementation code. It just declares the class name and inherits from a general base class !!

The most elegant part is not here. In fact, even in the base class, there is no gang competition like sqlconnection or oracleadapter.

I believe you have also guessed it. That's right. It uses the petshop implementation and uses the factory mode to ensure that the Dal can be applied to different database storage. However, this implementation is quite different from petshop: at least, it does not produce different SQL statements, there are no different parameter calling methods ("@" is generally used in SQL Server, and ":" is generally used in Oracle!

Of course, this is because of the factory implementation skills, but the more important factor is the exquisite design method. In fact, in the. NET Framework, the cornerstone of this design method has been provided. To put it bluntly, it is the interfaces in system. data (such as idbconnection and idataadapter ).

Based on this design, we no longer need to provide different data access implementations for different database storages for each Dal class. For example, in petshop, order classes are required for order data. Naturally, the system implements Order Classes for SQL Server and Oracle respectively, and uses different providers (sqlclient and oracleclient) provides methods to perform operations. In actual calls, petshop dynamically creates the real order class in the factory mode and activates the corresponding method. A solution for different database storage is on the paper.

In fact, the petshop solution is already more flexible. If it saves the trouble of "writing different order classes", it will actually deliver Buddha to Tian J. All of these functions have been fully implemented in the project that the author participates in!

As for the above "employeedal (of course, including all other Dal classes), there is no real implementation code", just a small configuration technique: store different Dal classes and related Stored Procedure (Note: it is not a table or view) in the XML file according to the namespace.

As you can see, theoretically, you can complete all the above work by simply using a dal class! However, in practice, different Dal classes may have some nuances in data processing (such as data verification and format conversion ).

In general, it is impossible for all developers (except dBA and Dal framework Developer) to understand ADO in such a large project. net, although the author has studied this, but in this project, only two classes are used from the beginning to the end: able, dataview (or even transaction )!

Others

Before the ADO. Net profiling, I had to mention the brother between datareader and dataset.

Competition.

Based on the materials read by the author, we recommend that you analyze the actual situation.

Very few are determined by personal habits.

When learning ADO. net, the author holds this idea and keeps in mind the information.

The terms summarized above (just like when I learned about gof 23 in the year, I can almost back up

Streaming J), think of a day can also be under ADO. Net amazing.

Unfortunately, the reality is not just what people want. I have made several projects in a row, regardless of the size

Dataset solution is adopted!

Now, let's look back at the most frequently opened petshop project when learning ADO. net,

Only by comparing the two sides can we see some clues.

Simply put, petshop uses the following "curve saving the nation" method to implement data

Exchange:

Datareader obtains data => creates a data entity class => Number of filled Fields

Data entity class => Add a data entity to the List class (only

Occasion)

(Supplement: using data entity or collection classes can easily implement cache manament,

The normal datareader cannot meet this requirement due to its Data Reading restrictions)

This process is similar to that of dataadapter. Fill (), except that,

In fill (), dataadpater automatically creates a datareader to obtain data, and then creates

Datatable (equivalent to the data entity class), and fill the datatable according to the field type, of course

If multiple records may be returned, datatable can process them completely, and there is no need to implement columns.

Table operation.

Readers may have a question immediately: Why does the petshop still need data

What about the body class?

There are some differences.

First, the data entity class is a lightweight structure, which generally only contains data fields and does not

Which operation method has some performance advantages over datatable or datarow?

(When the data volume is small, it can be ignored); on the other hand, operations on the data entity class are relatively

Simple, no need for developers to possess any ado. Net knowledge (in fact, datatable

That's not a problem.) Click the attribute.

However, according to the author's practice, these two aspects do not seem to be enough for humans to use them.

The datareader solution has the following reasons:

(1) When the data volume is large, batch reading can be adopted, which is similar to the data paging effect of the DataGrid;

(2) For simple data, Entity classes can cope with it. Once associated data is involved, only other methods can be written. All of these are very easy to process in dataset (for enterprise-level applications, complicated data needs to be processed in most cases );

(3) datatable supports data set operations by nature, which is easier to control and more natural than the "set + entity" hybrid mode (petshop;

(4) When declaring an object class, you need to determine all data types. When filling in data, you need datareader to pay attention to the data types corresponding to the object again, so there is no error! In this regard, datatable is very convenient, and you only need to pay attention to the type once during the operation;

(5) The dataset solution can easily support serialization operations (such as remoting and WebServices). At the same time, the relationship with XML is extremely close, this is also crucial for interaction with other systems.

I have analyzed some technologies and solutions and believe that readers have some experiences. At the end of this day, if you have to provide a "summary" here, the author's suggestion is very clear:

In enterprise-level application development, use dataset (datatable/dataview) + Cache Management solution as much as possible!

In other development scenarios, datareader is considered only in the following four cases (in the author's experience, most of the cases where datareader is used are 2nd ):

(1) When resource requirements are demanding, the resources here mainly refer to memory and database connections;

(2) You want to perform custom processing when reading the returned result set of the database. For example, you want to terminate the processing immediately after reading a record or perform computation operations during reading.

(Note: This is similar to the simple API for XML technology in XML. You do not need to read all XML data at a time to perform operations. On the contrary, Dom (Document Object Model) you must load all the XML data before you can start the operation. (msxml4.0 has started to allow you to read only part of the XML document data to start the operation. This is not the case )!)

(3) you only want to obtain the number of returned records or some fields of the returned records, for example:

String getnamebyid (int nid) // return the employee name based on the employee ID.

// Read the name field;

(Note: In this case, you can execute a specific query or stored procedure to solve the problem directly)

(4) for some reasons (for example, the N-tier system strictly distinguishes responsibilities between layers), it is impossible (or prohibited) to filter queries by the database itself, in this case, only datareader can be used for filtering during read!

(Note: Although dataview can achieve this goal, it must read all returned data before filtering, so it is not as good as datareader !)

UO/R Mapping

The full name of O/R Mapping is: object relational mapping, which aims to build a ing between the traditional RDBMS and OO language, so that developers can completely break away fromData PersistenceThis piece of scissors is constantly suffering from chaos.

For more information about o/R Mapping or recently popular o/X mapping, see"Programmers,2004.01, P86 "), You may need a special article to discuss in detail. The purpose of this article is to briefly analyze the advantages and disadvantages of the existing solution and provide some reference information in practice.

Compared with the J2EE platform, the O/R Mapping in. Net has no history, and no mature and available solutions have yet to be tested. However, with the attention of major vendors and the increasing popularity of open-source projects ,. net o/R Mapping has also begun to keep up with the pace, making this part of the J2EE domain a new competitor (will it make more developers invest. net camp? J). It also allows many Dal developers who are struggling to get back and forth in SQL clause or ADO. Net to see the bright path ".

Next, let's take a look at the solution worth exploring on the broader land than ADO. net.

ØOpen Source

Open-source. net, O/R Mapping is very few, but from the author's download trial and source code analysis, I think the following two solutions are of some reference value:OPF,OJB.

For more information about the two open-source projects, see"Programmers,2004.01, P13".

The full name of OPF is:OBjectPErsistentFRamework.

The full name of OJB is:OBJECT relationalBRidge.

In terms of implementation methods, the two solutions have completely different ideas and have their own representativeness.

The OPF route is similar to typed dataset or Borland ECO (refer to the introduction below,Provides more source code-Level ControlThe implementation of OJB is similar to Microsoft objectspaces (refer to the introduction below ),The configuration file is used.Is relatively complicated.

The basic framework of these two schemes is as follows:

OPF:

It is not difficult to see:

(1) The persistent class assumes the role of dataset. In addition to conventional object data operations, you can also set relationships between different objects (such as master-slave relationships and set relationships, this can also be omitted in the code generated by Borland ECO).Provides more source code-Level Control;

(2) persistentsqldatamanager assumes the role of dataadapter and performs real database operations through pre-configured commands. In the actually written employee data manager, developers really need to provide basic SQL statements, as they have set in sqlcommond (Borland eco goes further, replacing SQL with OCl );

(3) The role of objectbroker is very important. It serves as a bridge between objects and data. The registerpersistent method establishes thisVirtual(Object) andReality(RDBMS)Link;

(4) in the declaration of employee business object, the correspondence between object attributes and database fields is through. net attribute mechanism, so it is more convenient to modify it, although it is not flexible enough than the configuration file method (refer to the introduction of OJB), for example: need to re-compile, developers have to pay attention to database fields.

OJB:

 

It is not difficult to see:

(1) The implementation of this solution is complicated, but the code that the user needs to actually write is reduced (only the employee business object needs to be written). The key lies in the introductionConfiguration FileAt the same time, due to the introduction of the configuration file, we do not need to call the registerobject method similar to the OPF solution in Hello world application (refer to the OPF class diagram above, all of this (even including database connection information) is well known!

(2) In this solution, SQL commands are completely replaced by the criteria class, while queryfacade acts as the adapter function and communicates with the database through the real command persistencebroker;

(3) Whether it is repository. the xml configuration file, or the criteria and queryfacade classes, can be found in objectspaces (see the Introduction below) Similar implementations (is it a coincidence ?), At the same time, I personally think that this method is alsoMore consistentO/R MappingThis reduces the burden on developers!

(4) ojbalso has a cool tool named repositorygen.exe "which can be used to generate repository. xml configuration file (similarly, the source code is offered to J for free), which is not even implemented by objectspaces (think about so many fields, attributes, associations, mappings, it's crazy. J )!

? Microsoft objectspaces

This is a technology that excited many. Net guy's neck a few years ago. For the author, at that time, as long as we mentioned this topic, it was usually done in the laughter of J2EE guy, and we hate to make an ENB (relative to EJB) ourselves) or NCMP (relative to CMP) or something.

Finally, we can see it in. NET Framework 1.2 (which can be found in vs. Net 2004whidbey or Yukon, which is currently a beta version.

First, let's take a look at what the code written with objectspaces looks like (still using the above employee example ):

// Initialize objectspace

Sqlconnection conn = new sqlconnection ("Data Source = localhost;

Integrated Security = sspi; database = northwind ");

Objectspace OS = new objectspace ("map. xml", Conn );

// Return its title based on the employee ID

Employee oemp = (employee) OS. GetObject (

New objectquery (typeof (employee), "id = 1 "));

// Note: the actual field name is: employeeid

String strtitle = oemp. title;

......

// Return all qualified employees based on the City

Objectset oset = OS. getobjectset (

New objectquery (typeof (employee), "city = '" Seattle '"));

// Note: The returned data is not a able, but an object set.

Foreach (employee oemp in oset)

{

...... // Note: You can perform any operations on oemp.

}

There is also a solution for the second code above, that is, replacing objectset with objectreader, which contains differences similar to ADO. NET 1.0 (ADO. net is also called ADO. NET 2.0) the difference between dataset/able and datareader (I have to admire Microsoft's foresight in terms of consistency ).

By carefully analyzing the above code, we can find that it has a striking similarity with the OJB discussed earlier (the author in OJB only draws a basic class diagram, however, we can see that this idea is close )!

For example, the objectspace class basically provides the queryfacade function in OJB; The objectquery class basically provides the criteria function in OJB; at the same time, the two solutions use configuration files to store o/R Mapping Information, which is convenient for applications. A slight difference may be in the data return format (in this case, objectspaces takes into account more carefully and you can refer to the above Code), but this has little impact on the actual code implementation.

If you compare the call code in objectspaces with that in ADO. net, it is not difficult to see that the Code provided by objectspaces is easier to read and understand, even if you are not familiar with ADO. net overall architecture developers can also easily get started (the only code involved in RDBMS is only required to establish a database connection ). For those who are familiar with ADO. Net or have been familiar with O/R Mapping (such as hibernate under J2EE), it is a piece of cake!

As you can see from the. NET Framework 1.2 documentation, objectspaces provides a total of three namespaces with a clear structure:

System. Data. objectspaces

System. Data. objectspaces. Query

System. Data. objectspaces. Schema

Objectspaces and query have been seen in the code above. They can be guessed from their names. They are mainly responsible for providing basic access interfaces (such as query, addition, deletion, and modification) to external entities) and parse various query conditions (such as object filtering), the schema namespace is mainly used to operate the O/R Mapping configuration file and provide services for the classes in the other two namespaces.

In objectspaces, the O/R Mapping configuration file mainly refers to map. xml. The name of this file can be changed at will, similar to repository. XML in OJB. The other two configuration files describing the database structure and object structure are also very important: RSD. XML (relational schema definition) and OSD. XML (Object Schema Definition ). They can be understood as XSD files in the typed dataset. Without them, all data/object mapping and validation will be "invalid" J!

In this article, I am not going to explore objectspaces in depth, nor will I provide any samples to illustrate its advantages. In this regard, the. NET Framework SDK has already provided a wide range of packages.

The author just hopes that, from the Dal perspective, what does objectspaces technology bring to us? Does it mean to say goodbye to datareader/dataset or bring new troubles to developers?

The following is a few examples of advantages:

(1) All objectspaces use Object-based access to data, which greatly relieves SQL (or RDBMS) phobias of many developers;

(2) For relatively simple database structure changes, you only need to modify the configuration file, without re-compiling the code (compared with the OPF ing relationship.. Net attribute is encapsulated in code, which is more flexible and convenient );

(3) For complex database structure changes, modification is much easier because only object operations are involved;

(4) After the O/R Mapping configuration file is adopted, the database design and Dal development can be carried out separately, and the mutual influence is also reduced to the lowest point;

Below are more important topics:

(1) Chinese (permanent topic J) queries are not supported in the current version!

(2) The current version only supports database systems of SQL Server 2000 or later versions. It is weak (this is an intriguing restriction. If you are interested, you may wish to find out why )!

(1 and 2 refer to the. NET Framework SDK document. These two points have already exceeded many eager friends. Although the. Net project I participated in was not affected by 1, due to frequent use of Oracle, I had to temporarily reluctantly cut J)

(3) performance issues. Although objectspaces also provides functions similar to datareader (objectreader), after all, a strong data filling is required. In any case, there will be loss. If the returned data volume increases, it will be a problem that must be considered;

(4) performance problems. Map. XML is a good stuff, but how to optimize access to it and make the correct validation (based on RSD. XML, OSD. XML) after all, it takes time, even in some cases (the database structure is complicated), which will cause more serious consequences;

After talking about some shortcomings, you don't have to worry too much. After all, if there is no perfect solution, it depends on your own decisions.

At the end of this chapter, the author provides a summary for your reference. After all the analyses are completed, the author tries to provide "my solution (in writing)" based on his own practices, hoping to help readers.

Author profile:

"Zhang Xuefeng, author of this article, is a senior development engineer at BIBO Global Development Center. He is currently working in the core/EAI Department of BIBO global development center in Shanghai, China, and is engaged in research on. NET Technology and Development of related projects. You can contact him through the xuefeng.zhang@bearingpoint.com ."