The configuration of Spring master-slave database and the principle of dynamic Data source switching

Original: https://www.liaoxuefeng.com/article/00151054582348974482c20f7d8431ead5bc32b30354705000

In large applications, configuring a master-slave database and using read-write separation is a common design pattern. In spring applications, to implement read-write separation, it is best not to change the existing code, but to support it transparently at the bottom.

Spring has a built-in AbstractRoutingDataSource that can configure multiple data sources as a map, and then return different data sources based on different keys. Because it AbstractRoutingDataSource is also a DataSource interface, the application can set the key first, access the database code can be AbstractRoutingDataSource obtained from a corresponding real data source, so as to access the specified database. Its structure looks like this:

┌───────────────────────────┐
   │        controller         │
   │  set routing-key = "xxx"  │
   └───────────────────────────┘
                 │
                 ▼
   ┌───────────────────────────┐
   │        logic code         │
   └───────────────────────────┘
                 │
                 ▼
   ┌───────────────────────────┐
   │    routing datasource     │
   └───────────────────────────┘
                 │
       ┌─────────┴─────────┐
       │                   │
       ▼                   ▼
┌─────────────┐     ┌─────────────┐
│ read-write  │     │  read-only  │
│ datasource  │     │ datasource  │
└─────────────┘     └─────────────┘
       │                   │
       ▼                   ▼
┌─────────────┐     ┌─────────────┐
│             │     │             │
│  Master DB  │     │  Slave DB   │
│             │     │             │
└─────────────┘     └─────────────┘

Step one: Configure multiple data sources

First, we configure two data sources in Springboot, where the second data source is ro-datasource:

 
spring: datasource:
    jdbc-url: jdbc:mysql://localhost/test username: rw password: rw_password
    driver-class-name: com.mysql.jdbc.Driver hikari:
      pool-name: HikariCP
      auto-commit: false
      ...
  ro-datasource:
    jdbc-url: jdbc:mysql://localhost/test username: ro password: ro_password
    driver-class-name: com.mysql.jdbc.Driver hikari:
      pool-name: HikariCP
      auto-commit: false
      ...

In the development environment, it is not necessary to configure the master-slave database. Only need to set two users to the database, one rw with read and write permissions, one ro only select permission, this simulates the production environment of the master-slave database read and write separation.

In Springboot's configuration code, we initialize two data sources:

 
@SpringBootApplication public class MySpringBootApplication { /**
     * Master data source.
     */ @Bean("masterDataSource") @ConfigurationProperties(prefix = "spring.datasource")
    DataSource masterDataSource() {
       logger.info("create master datasource..."); return DataSourceBuilder.create().build();
    } /**
     * Slave (read only) data source.
     */ @Bean("slaveDataSource") @ConfigurationProperties(prefix = "spring.ro-datasource")
    DataSource slaveDataSource() {
        logger.info("create slave datasource..."); return DataSourceBuilder.create().build();
    }

    ...
}

Step Two: Write Routingdatasource

We then use the spring built-in Routingdata source to proxy two real data sources into a dynamic Data source:

 
public class RoutingDataSource extends AbstractRoutingDataSource { @Override protected Object determineCurrentLookupKey() { return "masterDataSource";
    }
}

For this RoutingData Source, you need to configure and set the primary data source in Springboot:

 
@SpringBootApplication public class MySpringBootApplication { @Bean @Primary
    DataSource primaryDataSource( @Autowired @Qualifier("masterDataSource") DataSource masterDataSource, @Autowired @Qualifier("slaveDataSource") DataSource slaveDataSource
    ) {
        logger.info("create routing datasource...");
        Map<Object, Object> map = new HashMap<>();
        map.put("masterDataSource", masterDataSource);
        map.put("slaveDataSource", slaveDataSource);
        RoutingDataSource routing = new RoutingDataSource();
        routing.setTargetDataSources(map);
        routing.setDefaultTargetDataSource(masterDataSource); return routing;
    }
    ...
}

Now, Routingdatasource is configured, but the choice of routing is to write dead, that is, to return forever"masterDataSource",

Now the question is: how to store the dynamically selected key and where to set the key?

In the servlet threading model, using Threadlocal to store keys is most appropriate, so we write a routingdatasource context to set up and dynamically store the key:

 
public class RoutingDataSourceContext implements AutoCloseable { // holds data source key in thread local: static final ThreadLocal<String> threadLocalDataSourceKey = new ThreadLocal<>(); public static String getDataSourceRoutingKey() {
        String key = threadLocalDataSourceKey.get(); return key == null ? "masterDataSource" : key;
    } public RoutingDataSourceContext(String key) {
        threadLocalDataSourceKey.set(key);
    } public void close() {
        threadLocalDataSourceKey.remove();
    }
}

Then, to modify the Routingdatasource, get the key code as follows:

 
 
public class RoutingDataSource extends AbstractRoutingDataSource {
    protected Object determineCurrentLookupKey() {
        return RoutingDataSourceContext.getDataSourceRoutingKey();
    }
}

In this way, a DataSource key can be dynamically set up somewhere, such as within a controller's method:

 
@Controller
public class MyController {
    @Get("/")
    public String index() {
        String key = "slaveDataSource";
        try (RoutingDataSourceContext ctx = new RoutingDataSourceContext(key)) {
            // TODO:
            return "html... www.liaoxuefeng.com";
        }
    }
}

To this end, we have successfully implemented dynamic routing access to the database.

This method is feasible, but the need to read from the database, you need to add a large piece of try (RoutingDataSourceContext ctx = ...) {} code, it is very inconvenient to use. Is there a way to simplify it?

Yes!

We think about the declarative transaction management that spring provides, which requires only one@Transactional()annotation, placed on a Java method, that automatically has a transaction.

We can also write a similar@RoutingWith("slaveDataSource")note to the method of a controller, which automatically selects the corresponding data source within the method. The code should look like this:

 
@Controller
public class MyController {
    @Get("/")
    @RoutingWith("slaveDataSource")
    public String index() {
        return "html... www.liaoxuefeng.com";
    }
}

In this way, the application's logic is not modified at all, and it is easiest to add annotations only where necessary and automatically implement dynamic Data source switching.

To write less code in the application, we have to do a little bit more of the bottom line: We have to use a mechanism like spring to implement declarative transactions, that is, to implement dynamic Data source switching with AOP.

Implementing this feature is also very simple, write one RoutingAspect, and use ASPECTJ to implement an Around interception:

 
@Aspect
@Component
public class RoutingAspect {
    @Around("@annotation(routingWith)")
    public Object routingWithDataSource(ProceedingJoinPoint joinPoint, RoutingWith routingWith) throws Throwable {
        String key = routingWith.value();
        try (RoutingDataSourceContext ctx = new RoutingDataSourceContext(key)) {
            return joinPoint.proceed();
        }
    }
}

Note the second parameter of the methodRoutingWithis the spring incoming annotation instance, which we get configured according to the annotationvalue()key. You need to add a maven dependency before compiling:

 
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

So far, we've implemented the ability to dynamically select data sources with annotations. The last step refactoring is to replace the and scattered around with string constants"masterDataSource" "slaveDataSource".

Usage restrictions

Constrained by the servlet threading model, a Dynamic Data source cannot be set in a request and then modified, that is,@RoutingWith not nested. In addition, when @RoutingWith @Transactional mixing, you set the priority of AOP.

This code requires Springboot support, and JDK 1.8 compiles and opens -parameter scompilation parameters.

The configuration of Spring master-slave database and the principle of dynamic Data source switching