Dynamically updatable multi-data source Persistence Layer Based on spring and ibatis

Preface

We often encounter some web projects that need to capture data from different data sources for analysis. These data sources may change and need to be dynamically maintained and added by users. However, most web applications use application servers or container middleware to manage the lifecycle of data sources. Therefore, data source changes cannot be independent of programs, but must be maintained by professionals, if necessary, you also need to re-release the program to adapt to the changes of the data source, and the number of data sources and database types are also limited.

So how can we break through these limitations to completely achieve the user's need to remotely maintain and manage the data source? This article proposes an effective solution. The general idea of this solution is to save the configuration information of the data source in a database or local file system with a fixed address, the system will automatically generate a data source based on the configuration information (the system will regenerate the new data source whenever the data information changes ), then, these data sources are automatically allocated to corresponding class objects through the persistence layer of the factory mode. In this way, you only need a user interface to manage the configuration information of the data source, so that you can implement the purpose of maintaining the data source system. The following describes the implementation of the solution in detail using a spring + ibatis framework instance.

Back to Top

Related Technologies

Ibatis

Apache ibatis is developed by Clinton begin and is now a generalized framework supported by the Apache Foundation to accelerate JDBC programming. It is a semi-automatic ORM framework widely used in current IT projects, different from the fully-automated frameworks such as Hibernate, ibatis provides more flexible control over database operations. For those that often need to call local database functions to customize SQL statements, for developers who prefer to optimize SQL Execution efficiency, ibatis is a very good choice. The widely used open-source enterprise architecture Spring framework also integrates ibatis, making it easier and faster to use ibatis in spring. Below are some key components of ibatis:

• Sqlmapclient: It is an important interface of ibatis and thread-safe. This interface involves SQL ing execution and batch processing.
• Sqlmap-Config. xml: Is the starting point for using ibatis. It is responsible for combining all SQL ing files (sqlmap. XML. This configuration file tells ibatis how to connect to the database and obtain the SQL ing files (sqlmap. XML ).
• Sqlmap. xmlContains the SQL statement we will run and is referenced by the Sqlmap-config.xml file.

Figure 1. ibatis Architecture

The reason why we chose ibatis instead of hibernate as the development framework of this solution is mainly because ibatis is more flexible than Hibernate and can easily rewrite its structure. Ibatis is more suitable for different database structures that implement the same logic, especially in terms of encapsulation of different database structures. In contrast, Hibernate needs to encapsulate the database structure, which means that different Po classes should be generated for different database structures, which makes development effort cumbersome. Of course, users can also choose to use hibernate as the framework. The concept is the same. The difference is only the means of implementation.

Spring support for ibatis

Spring provides good support for ibatis through the DAO mode.

• Sqlmapclient: Is the main interface in ibatis. The xml configuration file allows spring containers to manage sqlmapclient object creation. Spring provides sqlmapclientfactorybean to generate this object.
• Sqlmapclientfactorybean: Sqlmapclientfactorybean is a factory class provided by spring to generate sqlmapclient objects. When the spring configuration file is used to inject sqlmapclientfactorybean as an implementation class of sqlmapclient, the spring container calls its GetObject method according to the definition in the interface, and finally returns an implementation class of the sqlmapclient interface. The object generated by sqlmapclientfactorybean has two important attributes: The configlocation attribute is used to determine the sqlmap-config.xml, And the datasource attribute is used to determine the data source.
• Sqlmapclientdaosupport: The database operation class provided by spring. The DAO of the persistence layer of the application can inherit this class. Sqlmapclientdaosupport requires spring to inject the implementation object of the sqlmapclient interface to determine which data source to use and what sqlmap-config.xml to use.

Back to Top

Architecture and design of the persistent layer

As described above, we can see that to enable the traditional spring + ibatis framework to support a dynamic multi-data source persistence layer, we need to improve it. The data source is defined by the properties of the sqlmapclient object. Therefore, you must change the implementation object of the sqlmapclient interface to achieve the goal. Spring uses an xml configuration file to store sqlmapclient object information. Therefore, you only need to be able to dynamically generate the xml configuration file based on the data source configuration information to implement dynamic injection to the sqlmapclient interface.

Figure 2. Persistence Layer architecture flowchart (View the larger image)

The specific process of the persistence layer architecture is shown in:

1. create a configuration file generation class sqlmapclientfactory. When the application server is started, the Spring framework starts the init method of the sqlmapclientfactory class (this method is also restarted whenever the data source configuration information changes ), this method reads the data source configuration information stored in the database or local file system, and then dynamically generates the spring xml configuration file. In this xml configuration file, many different sqlmapclient objects are defined based on different data sources, and their corresponding data sources and sqlmap-config files are defined.

2. create the implementation class routingsqlmapclient of the sqlmapclient interface, and inject all generated sqlmapclient objects to routingsqlmapclient in the form of map through spring, when it is called, the corresponding sqlmapclient implementation object will be used as required to override the routingsqlmapclient method.

3. Inject routingsqlmapclient into all DAO implementation classes that inherit sqlmapclientdaosupport. Dao will decide which data source to use based on actual needs.

Back to Top

Implementation of the Persistence Layer

Use the sqlmapclientfactory class to generate an xml configuration file

As described above, the first step is to create the sqlmapclientfactory class and create a method to generate an xml configuration file for sqlmapclient. Then configure spring so that it can automatically call this method of sqlmapclient when the program starts. This method reads the data source configuration information from the local file system or the database system with a relatively fixed address. The main fields of the data source configuration information read by sqlmapclientfactory are as follows:

Note:

Table 1 data source configuration

ID	Name	Connection	User	Password	Dbtype
00001	PROJECTA	jdbc:db2://:<portNum>/<databaseName>	<userName>	<password>	DB2
00002	PROJECTB	jdbc:microsoft:sqlserver://: <portNum>;DatabaseName=<databaseName>	<userName>	<password>	sqlserver
00003	PROJECTC	jdbc:db2://:<portNum>/<databaseName>	<userName>	<password>	DB2
00004	PROJECTD	jdbc:db2://:<portNum>/<databaseName>	<userName>	<password>	DB2

Sqlmapclientfactory generates the xml configuration file of the corresponding sqlmapclient object based on the configuration information. The following describes in detail the main components of the XML configuration file.

1. Generate a map list of all sqlmapclient Interfaces Based on the ID and name of the data source configuration information.

Listing 1. Configure routingsqlmapclient

<bean id="routingSqlMapClient" class="com.ibm.mbps.tsd.dao.RoutingSqlMapClient"> <property name="targetSqlMapClients"> <map key-type="java.lang.String"> <entry key="PROJECTA" value-ref="sqlmapClient_00001"/> <entry key="PROJECTB" value-ref="sqlmapClient_00002"/> <entry key="PROJECTC" value-ref="sqlmapClient_00003"/> <entry key="PROJECTD" value-ref="sqlmapClient_00004"/> …… </map> </property> </bean>

2. Create a data source for each sqlmapclient interface implementation object. The data source is generated based on the preceding configuration information.

Listing 2. Data source configuration example

<bean id="datasource_00001" class="org.apache.commons.dbcp.BasicDataSource"> <property name="driverClassName"> <value>com.ibm.db2.jcc.DB2Driver</value> </property> <property name="url"> <value> jdbc:db2://hostname:portNum/databaseName</value> </property> <property name="username"> <Value>userName</value> </property> <property name="password"> <value>password</value> </property> </bean>

3. Inject data sources and sqlmap-config configuration files into each sqlmapclient object. Note that the sqlmap-config configuration file is for a type of data source, for example, if the database type and content of multiple data sources are the same, the same configuration file should be used.

Listing 3. sqlmapclient object configuration example

<bean id=" sqlmapClient_00001" class="org.springframework.orm.ibatis.SqlMapClient FactoryBean"> <property name="configLocation" value="classpath:/sqlmap/db2/sql-map-config.xml"/> <property name="dataSource"> <ref local=" datasource_00001"/> </property> </bean>

Override the sqlmapclient interface implementation class routingsqlmapclient

After the sqlmapclient object is generated, we also need to create a routingsqlmapclient implementation class to override the corresponding sqlmapclient interface method. Routingsqlmapclient creates a map variable to undertake the map list of the previously generated sqlmapclient implementation object, and then determines which implementation object to use to dynamically override the routingsqlmapclient Class Based on the keyword. Routingsqlmapclient uses the targetsqlmapclients variable to receive the sqlmapclient Object List.

Listing 4. code snippet of the routingsqlmapclient class

public class RoutingSqlMapClient implements SqlMapClient { private Map<String, SqlMapClient> targetSqlMapClients; public void flushDataCache() { targetSqlMapClients.get(getDSType()).flushDataCache(); } public SqlMapSession getSession() { return targetSqlMapClients.get(getDSType()).getSession(); } public int delete(String id, Object parameterObject) throws SQLException { return targetSqlMapClients.get(getDSType()).delete(id,parameterObject); } public Object insert(String id, Object parameterObject) throws SQLException { return targetSqlMapClients.get(getDSType()).insert(id,parameterObject); } public List queryForList(String id, Object parameterObject) throws SQLException { return targetSqlMapClients.get(getDSType()).queryForList(id,parameterObject); } …… }

Create a DAO class that inherits sqlmapclientdaosupport

We chose to use routingsqlmapclient to override the implementation method of sqlmapclient, instead of injecting the sqlmapclient implementation object directly into the corresponding Dao because a DAO class may correspond to multiple data sources, if the sqlmapclient that contains only one data source is directly injected into Dao, the reusability of Dao is severely restricted. Therefore, we load the entire sqlmapclient implementation Object List into the routingsqlmapclient class. At the logic layer, we define which sqlmapclient object is used to override the routingsqlmapclient. The Dao architecture of the persistence layer is shown in:

Figure 3. Persistence Layer Dao architecture diagram (View the larger image)

The Dao object class inherits sqlmapclientdaosupport and implements different interfaces ., Different interfaces correspond to different upper-layer logic, while DAO implementation class that implements its logic references different data sources and sqlmap-config.xml, these data sources and XML are defined in sqlmapclient object, when we call the persistence layer DAO class to operate the database, we need to first call the setdstype () method in routingsqlmapclient to determine which data source to use, and use the corresponding sqlmapclient to implement object Rewriting for routingsqlmapclient, in this way, the sqlmapclient corresponding to this data source can be passed into Dao as a routingsqlmapclient object to implement a structure where multiple data sources coexist.

After the multi-data source persistence layer that supports dynamic updates is developed, a set of UI components should be developed for you to update and maintain the data source information. After each update, the user must call the init method of the sqlmapclientfactory class to regenerate the xml configuration file of sqlmapclient. In this way, the data source can be dynamically updated and added without restarting the server.

Back to Top

Summary

This article describes how to implement a multi-data source persistence layer system that can be dynamically updated, which can be used as a reference for developers of similar projects. However, because the main purpose of this article is to describe a conceptual method rather than a specific implementation, some related technologies and specific implementations have not been written, however, developers can choose their preferred methods based on the ideas in this article. In addition, some terms and functions of ibatis and spring are not described in detail, it is recommended that readers who are not familiar with the ibatis architecture can refer to other tutorials on their own. In this article, the spring + ibatis framework is used as an example only, but it is only recommended and does not guarantee the use results and effects. (This article only represents the author's personal opinion)