Implementation of Spring Dynamic Registration of multiple data sources, spring Dynamic Data sources

Implementation of Spring Dynamic Registration of multiple data sources, spring Dynamic Data sources

Recently, I am working on SaaS applications. The database adopts a single-instance multi-schema architecture (see reference 1). Each tenant has an independent schema and the entire data source has a shared schema, therefore, it is necessary to solve the problem of dynamic addition, deletion, and data source switching.

After searching many articles on the Internet, many of them talk about the configuration of the Master/Slave Data source, or the data source configuration has been determined before the application is started. Little talk about how to dynamically load the data source without downtime, so write this article for your reference.

Technologies used

• Java 8
• Spring + SpringMVC + MyBatis
• Druid connection pool
• Lombok
• (The above technology does not affect the implementation of ideas, just to facilitate browsing the following code snippets)

Ideas

When a request comes in, determine the tenant to which the current user belongs, switch to the corresponding data source based on the tenant information, and then perform subsequent business operations.

Code Implementation

TenantConfigEntity (Tenant information) @ EqualsAndHashCode (callSuper = false) @ Data @ FieldDefaults (level = AccessLevel. PRIVATE) public class TenantConfigEntity {/*** tenant id **/Integer tenantId;/*** tenant name **/String tenantName; /*** tenant name key **/String tenantKey;/*** Database url **/String dbUrl;/*** database username **/String dbUser; /* Database Password */String dbPassword;/* database public_key **/String dbPublicKey;} DataSourceUti L (auxiliary tool class, not required) public class performanceutil {private static final String DATA_SOURCE_BEAN_KEY_SUFFIX = "_ data_source"; private static final String JDBC_URL_ARGS = "? UseUnicode = true & characterEncoding = UTF-8 & useOldAliasMetadataBehavior = true & zeroDateTimeBehavior = convertToNull "; private static final String CONNECTION_PROPERTIES =" config. decrypt = true; config. decrypt. key = ";/*** concatenate the spring bean key of the data source */public static String getDataSourceBeanKey (String tenantKey) {if (! StringUtils. hasText (tenantKey) {return null;} return tenantKey + DATA_SOURCE_BEAN_KEY_SUFFIX;}/*** concatenate the complete jdbc url */public static String getJDBCUrl (String baseUrl) {if (! StringUtils. hasText (baseUrl) {return null;} return baseUrl + JDBC_URL_ARGS;}/*** concatenate the complete Druid connection attribute */public static String getConnectionProperties (String publicKey) {if (! StringUtils. hasText (publicKey) {return null;} return CONNECTION_PROPERTIES + publicKey ;}}

DataSourceContextHolder

Use ThreadLocal to save the key name of the data source of the current thread and implement the set, get, and clear methods;

public class DataSourceContextHolder { private static final ThreadLocal<String> dataSourceKey = new InheritableThreadLocal<>(); public static void setDataSourceKey(String tenantKey) {  dataSourceKey.set(tenantKey); } public static String getDataSourceKey() {  return dataSourceKey.get(); } public static void clearDataSourceKey() {  dataSourceKey.remove(); }}

DynamicDataSource (important)

Inherit AbstractRoutingDataSource (we recommend that you read the source code to learn about the process of dynamically Switching Data Sources) to achieve dynamic data source selection;

public class DynamicDataSource extends AbstractRoutingDataSource { @Autowired private ApplicationContext applicationContext; @Lazy @Autowired private DynamicDataSourceSummoner summoner; @Lazy @Autowired private TenantConfigDAO tenantConfigDAO; @Override protected String determineCurrentLookupKey() {  String tenantKey = DataSourceContextHolder.getDataSourceKey();  return DataSourceUtil.getDataSourceBeanKey(tenantKey); } @Override protected DataSource determineTargetDataSource() {  String tenantKey = DataSourceContextHolder.getDataSourceKey();  String beanKey = DataSourceUtil.getDataSourceBeanKey(tenantKey);  if (!StringUtils.hasText(tenantKey) || applicationContext.containsBean(beanKey)) {   return super.determineTargetDataSource();  }  if (tenantConfigDAO.exist(tenantKey)) {   summoner.registerDynamicDataSources();  }  return super.determineTargetDataSource(); }}

DynamicDataSourceSummoner (key)

Load data source information from the database, dynamically assemble and register spring bean,

@ Slf4j @ Componentpublic class DynamicDataSourceSummoner implements ApplicationListener <ContextRefreshedEvent> {// The default data source id of the spring-data-source.xml must be consistent with the private static final String identifier = "defaultDataSource"; @ Autowired private ConfigurableApplicationContext; @ Autowired private DynamicDataSource dynamicDataSource; @ Autowired private TenantConfigDAO tenantConf IgDAO; private static boolean loaded = false;/*** run the command */@ Override public void onApplicationEvent (ContextRefreshedEvent event) after Spring is loaded. {// prevent repeated if (! Loaded) {loaded = true; try {registerDynamicDataSources ();} catch (Exception e) {log. error ("Data Source initialization failed, Exception:", e) ;}}/*** read the tenant's DB configuration from the database, and dynamically inject the Spring container */public void registerDynamicDataSources () {// obtain the DB configuration List of all tenants <TenantConfigEntity> tenantConfigEntities = tenantConfigDAO. listAll (); if (CollectionUtils. isEmpty (tenantConfigEntities) {throw new IllegalStateException ("application initialization failed. Please configure data first Source ");} // register the data source bean to the container addperformancebeans (tenantConfigEntities );} /*** create a bean Based on DataSource and register it to the container */private void addDataSourceBeans (List <TenantConfigEntity> tenantConfigEntities) {Map <Object, Object> targetDataSources = Maps. newLinkedHashMap (); defalistlistablebeanfactory beanFactory = (DefaultListableBeanFactory) applicationContext. getAutowireCapableBeanFactory (); for (TenantConfigEntity enti Ty: tenantConfigEntities) {String beanKey = performanceutil. getDataSourceBeanKey (entity. getTenantKey (); // if the data source has already been registered in spring, if (applicationContext. containsBean (beanKey) {DruidDataSource existsDataSource = applicationContext. getBean (beanKey, DruidDataSource. class); if (isSameDataSource (existsDataSource, entity) {continue ;}// assemble bean AbstractBeanDefinition beanDefinition = getBea NDefinition (entity, beanKey); // register bean beanFactory. registerBeanDefinition (beanKey, beanDefinition); // put it in map. Note that the targetDataSources bean object was just created. put (beanKey, applicationContext. getBean (beanKey);} // set the created map object to targetDataSources; dynamicDataSource. setTargetDataSources (targetDataSources); // The resolvedDataSources in AbstractRoutingDataSource will be reinitialized only when this operation is performed. afterPr OpertiesSet ();}/*** assemble spring bean */private AbstractBeanDefinition getBeanDefinition (TenantConfigEntity entity, String beanKey) {BeanDefinitionBuilder = BeanDefinitionBuilder. genericBeanDefinition (DruidDataSource. class); builder. getBeanDefinition (). setAttribute ("id", beanKey); // other configurations inherit defadatasource datasource builder. setParentName (DEFAULT_DATA_SOURCE_BEAN_KEY); builder. setInitMethodName ("Init"); builder. setDestroyMethodName ("close"); builder. addPropertyValue ("name", beanKey); builder. addPropertyValue ("url", performanceutil. getJDBCUrl (entity. getDbUrl (); builder. addPropertyValue ("username", entity. getDbUser (); builder. addPropertyValue ("password", entity. getDbPassword (); builder. addPropertyValue ("connectionProperties", DataSourceUtil. getConnectionProperties (entity. getDbPublicKey (); Return builder. getBeanDefinition ();}/*** determine whether the DataSource information in the Spring container is consistent with that in the database * Note: public_key is not determined here, because the other three pieces of information are determined to be unique */private boolean isSameDataSource (DruidDataSource existsDataSource, TenantConfigEntity entity) {boolean sameUrl = Objects. equals (existsDataSource. getUrl (), performanceutil. getJDBCUrl (entity. getDbUrl (); if (! SameUrl) {return false;} boolean sameUser = Objects. equals (existsDataSource. getUsername (), entity. getDbUser (); if (! SameUser) {return false;} try {String decryptPassword = ConfigTools. decrypt (entity. getDbPublicKey (), entity. getDbPassword (); return Objects. equals (existsDataSource. getPassword (), decryptPassword);} catch (Exception e) {log. error ("Data Source password verification failed, Exception: {}", e); return false ;}}}

Spring-data-source.xml

<! -- Introduce the jdbc configuration file --> <context: property-placeholder location = "classpath: data. properties" ignore-unresolvable = "true"/> <! -- Public (default) data source --> <bean id = "defaultDataSource" class = "com. alibaba. druid. pool. druidDataSource "init-method =" init "destroy-method =" close "> <! -- Basic attribute url, user, password --> <property name = "url" value = "$ {ds. jdbcUrl} "/> <property name =" username "value =" $ {ds. user} "/> <property name =" password "value =" $ {ds. password} "/> <! -- Configure the initialization size, minimum, and maximum --> <property name = "initialSize" value = "5"/> <property name = "minIdle" value = "2"/> <property name = "maxActive" value = "10"/> <! -- Configure the timeout time for obtaining connections, in milliseconds --> <property name = "maxWait" value = "1000"/> <! -- Configure the interval for a test to detect idle connections that need to be closed, in milliseconds --> <property name = "timeBetweenEvictionRunsMillis" value = "5000"/> <! -- Configure the minimum time for a connection to survive in the pool, unit: millisecond --> <property name = "minEvictableIdleTimeMillis" value = "240000"/> <property name = "validationQuery" value = "SELECT 1"/> <! -- Unit: seconds to check whether the connection is valid. --> <property name = "validationQueryTimeout" value = "60"/> <! -- We recommend that you set this parameter to true without affecting performance and ensuring security. Check the connection request. If the idle time is greater than timeBetweenEvictionRunsMillis, run validationQuery to check whether the connection is valid. --> <property name = "testWhileIdle" value = "true"/> <! -- When applying for a connection, run validationQuery to check whether the connection is valid. This configuration will reduce the performance. --> <Property name = "testOnBorrow" value = "true"/> <! -- Execute validationQuery to check whether the connection is valid when the connection is returned. This configuration will reduce the performance. --> <Property name = "testOnReturn" value = "false"/> <! -- Config Filter --> <property name = "filters" value = "config"/> <property name = "connectionProperties" value = "config. decrypt = true; config. decrypt. key =$ {ds. publickey} "/> </bean> <! -- Transaction Manager --> <bean id = "txManager" class = "org. springframework. jdbc. datasource. dataSourceTransactionManager "> <property name =" dataSource "ref =" multipleDataSource "/> </bean> <! -- Multiple data sources --> <bean id = "multipleDataSource" class = ". b. c. dynamicDataSource "> <property name =" defaultTargetDataSource "ref =" defaultDataSource "/> <property name =" targetDataSources "> <map> <entry key =" defaultDataSource "value-ref =" defaultDataSource "/> </map> </property> </bean> <! -- Annotation Transaction Manager --> <! -- The order value must be greater than the order value of DynamicDataSourceAspectAdvice --> <tx: annotation-driven transaction-manager = "txManager" order = "2"/> <! -- Create SqlSessionFactory and specify the data source --> <bean id = "mainSqlSessionFactory" class = "org. mybatis. spring. sqlSessionFactoryBean "> <property name =" dataSource "ref =" multipleDataSource "/> </bean> <! -- DAO interface package name, Spring will automatically find the DAO under --> <bean id = "mainSqlMapper" class = "org. mybatis. spring. mapper. mapperScannerConfigurer "> <property name =" sqlSessionFactoryBeanName "value =" mainSqlSessionFactory "/> <property name =" basePackage "value =". b. c. *. dao "/> </bean> <bean id =" defaultSqlSessionFactory "class =" org. mybatis. spring. sqlSessionFactoryBean "> <property name =" dataSource "ref =" defaultDataSource "/> </bean> <Bean id = "defaultSqlMapper" class = "org. mybatis. spring. mapper. mapperScannerConfigurer "> <property name =" sqlSessionFactoryBeanName "value =" defaultSqlSessionFactory "/> <property name =" basePackage "value =". b. c. base. dal. dao "/> </bean> <! -- Other configurations are omitted -->

DynamicDataSourceAspectAdvice

Use AOP to automatically switch data sources for reference only;

@ Slf4j @ Aspect @ Component @ Order (1) // note: the order must be smaller than the order of tx: annotation-driven, that is, execute the DynamicDataSourceAspectAdvice plane and then execute the transaction plane, to obtain the final data source @ EnableAspectJAutoProxy (proxyTargetClass = true) public class DynamicDataSourceAspectAdvice {@ Around ("execution (*. b. c. *. controller. *. *(..)) ") public Object doAround (ProceedingJoinPoint jp) throws Throwable {ServletRequestAttributes sra = (ServletRequestAttributes) RequestContextHolder. getRequestAttributes (); HttpServletRequest request = sra. getRequest (); HttpServletResponse response = sra. getResponse (); String tenantKey = request. getHeader ("tenant"); // the front-end must input the tenant header; otherwise, the system returns the 400 if (! StringUtils. hasText (tenantKey) {WebUtils. toHttp (response ). sendError (HttpServletResponse. SC _BAD_REQUEST); return null;} log.info ("current tenant key :{}", tenantKey); performancecontextholder. setperformancekey (tenantKey); Object result = jp. proceed (); performancecontextholder. clearperformancekey (); return result ;}}

Summary

The above section describes how to implement Spring Dynamic Registration of multiple data sources. I hope it will help you. If you have any questions, please leave a message and I will reply to you in a timely manner. Thank you very much for your support for the help House website!