Debugging of dynamic configuration and loading usage of multiple data sources and interaction between frameworks

This is the problem I encountered. The project uses spring + hibernate + proxool for database connection management and access. The requirement is to dynamically configure and load multiple data sources. The idea is:

1. Implement the applicationlistener interface using a kind of advanceddatasourceinitizer. When the contextrefreshevent event is published, the database configuration is automatically read from the database, converted to a proxooldatasource object, and added to a map <datasourcename,
Proxooldatasource>;

package opstools.moonmm.support.listener;import java.util.List;import java.util.Map;import javax.sql.DataSource;import opstools.framework.datasource.MultiDataSource;import opstools.moonmm.clusterconfig.entity.ClusterConfig;import opstools.moonmm.clusterconfig.service.ClusterConfigService;import opstools.moonmm.support.utils.DBUtil;import org.springframework.beans.BeansException;import org.springframework.context.ApplicationContext;import org.springframework.context.ApplicationContextAware;import org.springframework.context.ApplicationEvent;import org.springframework.context.ApplicationListener;import org.springframework.jdbc.datasource.lookup.MapDataSourceLookup;public class AdvancedDataSourceInitializer implements ApplicationListener, ApplicationContextAware {    private   String             desiredEventClassName;    protected ApplicationContext applicationContext;    public void onApplicationEvent(ApplicationEvent event) {        if (shouldStart(event)) {                        Map<String, DataSource> cachedMap = (Map<String, DataSource>)applicationContext.getBean("dataSources");            ClusterConfigService clusterConfigService = (ClusterConfigService)applicationContext.getBean("clusterConfigService");            List<ClusterConfig> cclist = clusterConfigService.getAllClusterConfigInstances();                        DBUtil.addCachedDatasources(cachedMap, cclist);            MapDataSourceLookup dsLookup =  (MapDataSourceLookup) applicationContext.getBean("dataSourceLookup");            dsLookup.setDataSources(cachedMap);            MultiDataSource mds = (MultiDataSource) applicationContext.getBean("dataSource");            mds.setTargetDataSources(cachedMap);              mds.afterPropertiesSet();        }    }    protected Class<?> getDesiredType() {        try {            return Class.forName(desiredEventClassName);        } catch (ClassNotFoundException e) {            throw new RuntimeException(e);        }    }    public String getDesiredEventClassName() {        return desiredEventClassName;    }    public void setDesiredEventClassName(String desiredEventClassName) {        this.desiredEventClassName = desiredEventClassName;    }    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {        this.applicationContext = applicationContext;    }    protected boolean shouldStart(ApplicationEvent event){        Class<?> clazz = getDesiredType();        return clazz.isInstance(event);    }}

Dbutil. Java: used to convert the database configuration to the proxooldatasource object and classify it into the connection pool management.

package opstools.moonmm.support.utils;import java.util.List;import java.util.Map;import javax.sql.DataSource;import opstools.moonmm.clusterconfig.entity.ClusterConfig;import opstools.moonmm.monitorconfig.entity.MonitorConfig;import org.logicalcobwebs.proxool.ProxoolDataSource;public class DBUtil {        private DBUtil() {}        private static final String MYSQL_DRIVER = "com.mysql.jdbc.Driver";        public static DataSource cluconfig2DataSource(ClusterConfig cc)    {        ProxoolDataSource ds = new ProxoolDataSource();        String url = "jdbc:mysql://"+cc.getDbIp()+":"+cc.getDbPort()+"/"+cc.getDbName();        ds.setDriver(MYSQL_DRIVER);        ds.setAlias(cc.getDataSource());        ds.setDriverUrl(url);        ds.setUser(cc.getDbUser());        ds.setPassword(cc.getDbPassword());        ds.setPrototypeCount(5);        ds.setMinimumConnectionCount(10);        ds.setMaximumConnectionCount(50);        return ds;    }    public static DataSource moniconfig2DataSource(MonitorConfig mc)    {        ProxoolDataSource ds = new ProxoolDataSource();        String url = "jdbc:mysql://"+ mc.getIp() +":"+ mc.getPort() + "/" + mc.getMonitordbName();        ds.setDriver(MYSQL_DRIVER);        ds.setAlias(mc.getNickname());        ds.setDriverUrl(url);        ds.setUser(mc.getUser());        ds.setPassword(mc.getPassword());        ds.setPrototypeCount(5);        ds.setMinimumConnectionCount(10);        ds.setMaximumConnectionCount(50);        return ds;    }            public static void addCachedDatasources(Map<String, DataSource> cachedMap, List<ClusterConfig> cclist)    {        for (ClusterConfig cc: cclist) {            cachedMap.put(cc.getDataSource(), cluconfig2DataSource(cc));        }    }    }

2. Use springeventpublisher to implement applicationcontextaware to obtain the applicationcontext instance. When the application is started and the database configuration is added or deleted, the contextrefreshevent event is published to trigger dynamic loading of the data source;

package opstools.moonmm.support.listener;import org.springframework.beans.BeansException;import org.springframework.context.ApplicationContext;import org.springframework.context.ApplicationContextAware;import org.springframework.context.event.ContextRefreshedEvent;public class SpringEventPublisher implements ApplicationContextAware {    private ApplicationContext appContext;        @Override    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {        this.appContext = applicationContext;    }        public void publishContextRefreshEvent()    {        appContext.publishEvent(new ContextRefreshedEvent(appContext));     }}

3. Use a multidatasource class to inherit abstractroutingdatasource to locate and switch data sources.

package opstools.framework.datasource;import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;public class MultiDataSource extends AbstractRoutingDataSource {@Overrideprotected Object determineCurrentLookupKey() {return DataSourceHolder.getCurrentDataSource();}}

package opstools.framework.datasource;public class DataSourceHolder {private static final ThreadLocal<String> contextHolder = new ThreadLocal<String>();public static String getCurrentDataSource() {return (String) contextHolder.get();}   public static void setDataSource(String dataSource){contextHolder.set(dataSource);}public static void setDefaultDataSource(){contextHolder.set(null);}public static void clearCustomerType() {   contextHolder.remove();   }  }

The bean instances of the above three classes can be directly configured in the spring file.

        <util:map id="dataSources"><entry key="master" value-ref="masterDataSource" /></util:map><bean id="dataSourceLookup"class="org.springframework.jdbc.datasource.lookup.MapDataSourceLookup"></bean><bean id="dataSource" class="opstools.framework.datasource.MultiDataSource"><property name="targetDataSources" ref="dataSources"/><property name="defaultTargetDataSource" ref="masterDataSource" /><property name="dataSourceLookup" ref="dataSourceLookup" /></bean><bean id="sessionFactory"class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean"><property name="dataSource" ref="dataSource" /><property name="configLocation" value="classpath:hibernate.cfg.xml" /><property name="packagesToScan" value="opstools.*.*.entity" /><property name="configurationClass" value="org.hibernate.cfg.AnnotationConfiguration" /><property name="namingStrategy"><bean class="org.hibernate.cfg.ImprovedNamingStrategy"></bean></property></bean><bean id="transactionManager"class="org.springframework.orm.hibernate3.HibernateTransactionManager"><property name="sessionFactory" ref="sessionFactory" /></bean><bean id="dataSourceInitializer" class="opstools.moonmm.support.listener.AdvancedDataSourceInitializer">        <property name="desiredEventClassName" value="org.springframework.context.event.ContextRefreshedEvent"/>    </bean>        <bean id="eventPublisher" class="opstools.moonmm.support.listener.SpringEventPublisher">    </bean>

However, in actual use, you cannot switch the data source correctly. You can only switch to the first data source to use. After verification, it is found that the proxool alias and the number of connections must be set.

Public static proxooldatasource cluconfig2datasource (clusterconfig CC ){

Proxooldatasource PPS = new proxooldatasource ();

PPS. setdriverurl (...);

...

PPS. setalias (CC. getdatasource (); // This row and the following rows must exist; otherwise, it is difficult to work.

PPS. setminimumconnectioncount (5 );

PPS. setmaximumconnectioncount (50 );

PPS. setprototypecount (10 );

}

The entire debugging process is as follows:

First,The premise is to prepare the source code, which can be downloaded using the maven plug-in of Eclipse.. Select the specified jar package, right-click Maven ---> download sources, and place it in the specified \. m2 \ repository directory. In Windows, it is generally stored in the Documents and Settings \ User directory \. m2 \ repository \
In Linux ~ /. M2/Repository /. If the source code package of the corresponding class is missing during single-step debugging, the source look up interface and button will appear. After you click Add source code package, the interface will become the source code interface of the corresponding class. We recommend that you use the maven project building tool instead of searching and downloading it from the official website.

Because the code for framework interaction may have problems in many places, you can only perform one-step debugging. However, the execution of one line is too slow. Therefore,Analyze the error features and set some key breakpoints. For example, the key points here are: Set the region where the CENAME is located (verify that the key of the correct data source is passed in) and obtain the datasource (verify that the corresponding data source object is located ), obtain
Connection (verify that the correct database connection is obtained. Note that running in debug mode is the icon with bugs, not the right arrow icon. Through one-step debugging, you can know the specific process of getting the proxool database connection is as follows (it is better to draw a UML sequence diagram ):

Performanceholder. setdatasource (datasourcename) ---> abstractroutingdatasource. determinetargetdatasource (datasourcename) ---> proxooldatasource. getconnection () --->
Connectionpool. getconnection () ---> proxyconnections. getconnection (nextavailableconnection)

Indexoutofboundsexception is thrown here. Proxyconnections does not contain the connection to the data source that has just been switched. I assume that proxool should automatically create several connections in advance and put them in the corresponding connection pool. After the number of connections is set in the Code, the connection succeeds. A similar error occurs later, which is solved by setting the alias.

It is assumed that proxool will automatically create the default number of connections in advance (the number of connections is not set in the static configuration file, and many articles on the Internet also refer to the existence of the default number of connections ), I thought the alias was irrelevant. I didn't expect an error. Therefore, it is surprising that the proxool data source depends on the alias.

Why is the alias of proxooldatasource so important? Because proxool uses alias to identify connection pools of different databases. Code:

Proxooldatasource. getconnection:

 /**     * @see javax.sql.DataSource#getConnection()     */    public Connection getConnection() throws SQLException {        ConnectionPool cp = null;        try {            if (!ConnectionPoolManager.getInstance().isPoolExists(alias)) {                registerPool();            }            cp = ConnectionPoolManager.getInstance().getConnectionPool(alias);            return cp.getConnection();        } catch (ProxoolException e) {            LOG.error("Problem getting connection", e);            throw new SQLException(e.toString());        }    }

The connection pool manager is used to obtain the connection pool code connectionpoolmanager. getconnectionpool. A map is used to store the connection pool. The key is the alias of the connection pool, and the value is the connection pool instance.

class ConnectionPoolManager {    private static final Object LOCK = new Object();    private Map connectionPoolMap = new HashMap();    private Set connectionPools = new HashSet();    private static ConnectionPoolManager connectionPoolManager = null;    private static final Log LOG = LogFactory.getLog(ProxoolFacade.class);    public static ConnectionPoolManager getInstance() {        if (connectionPoolManager == null) {            synchronized (LOCK) {                if (connectionPoolManager == null) {                    connectionPoolManager = new ConnectionPoolManager();                }            }        }        return connectionPoolManager;    }    private ConnectionPoolManager() {    }    /**     * Get the pool by the alias     * @param alias identifies the pool     * @return the pool     * @throws ProxoolException if it couldn't be found     */    protected ConnectionPool getConnectionPool(String alias) throws ProxoolException {        ConnectionPool cp = (ConnectionPool) connectionPoolMap.get(alias);        if (cp == null) {            throw new ProxoolException(getKnownPools(alias));        }        return cp;    }    /**     * Convenient method for outputing a message explaining that a pool couldn't     * be found and listing the ones that could be found.     * @param alias identifies the pool     * @return a description of the wht the pool couldn't be found     */    protected String getKnownPools(String alias) {        StringBuffer message = new StringBuffer("Couldn't find a pool called '" + alias + "'. Known pools are: ");        Iterator i = connectionPoolMap.keySet().iterator();        while (i.hasNext()) {            message.append((String) i.next());            message.append(i.hasNext() ? ", " : ".");        }        return message.toString();    }    /**     * Whether the pool is already registered     * @param alias how we identify the pool     * @return true if it already exists, else false     */    protected boolean isPoolExists(String alias) {        return connectionPoolMap.containsKey(alias);    }    /** @return an array of the connection pools */    protected ConnectionPool[] getConnectionPools() {        return (ConnectionPool[]) connectionPools.toArray(new ConnectionPool[connectionPools.size()]);    }    protected ConnectionPool createConnectionPool(ConnectionPoolDefinition connectionPoolDefinition) throws ProxoolException {        ConnectionPool connectionPool = new ConnectionPool(connectionPoolDefinition);        connectionPools.add(connectionPool);        connectionPoolMap.put(connectionPoolDefinition.getAlias(), connectionPool);        return connectionPool;    }    protected void removeConnectionPool(String name) {        ConnectionPool cp = (ConnectionPool) connectionPoolMap.get(name);        if (cp != null) {            connectionPoolMap.remove(cp.getDefinition().getAlias());            connectionPools.remove(cp);        } else {            LOG.info("Ignored attempt to remove either non-existent or already removed connection pool " + name);        }    }    public String[] getConnectionPoolNames() {        return (String[]) connectionPoolMap.keySet().toArray(new String[connectionPoolMap.size()]);    }}

This explains why the relationship between proxool and the alias is so tight.

You need patience to debug the interaction between frameworks. Because the specific error location may be distributed in any unexpected location,It may be skipped directly if it is considered unrelated. You need to return the result and locate the previous position. The previous position is repeated until the error location is approaching step by step.. For example, when you start to locate the problem, you do not have to perform a detailed analysis. Instead, you can skip the execution step at will and jump from the spring source code to proxool.
To jump to the source code of Hibernate and then back to spring,Later, I found a small clue, gradually narrowing down the scope, and finally located the problem.. Today, we spent a whole day debugging the two problems that occurred when switching data sources. The usage of the development framework increases the difficulty of debugging and increases the maintenance cost.

The main achievement is: I finally successfully debugged a question about the framework Interaction :-)