JBoss DataGrid cluster deployment and access

Cluster deployment

The cache mode of JDG includes Local mode and cluster mode. This project is deployed in multi-node Clustered mode. Data is replicated between subsets of multiple nodes instead of all nodes. Using subset replication can improve the fault tolerance efficiency, but it does not significantly affect scalability. JGroup should be configured before deployment in Clustered mode.

1. broadcast using UDP.

L applicable to large clusters (over 100 nodes );

L applicable to Invalidation and Replication modes;

L improve the efficiency of socket communication.

2. Use TCP broadcast.

It is more suitable for small-scale (less than 100 nodes) clusters in distribution mode, because TCP is more efficient in point-to-point communication.

The Clustered mode is divided into Invalidation, Replication, and Distribution modes.

Distribution Mode

The Distribution mode of JDG stores cached data on a subset node of the cluster, rather than storing data to each node. Data redundancy and fault tolerance are usually stored on more than one node.

In Distribution mode, consistent Hash algorithms are used to select data storage nodes from the cluster. Consistent Hash algorithms are configured to store cached data to multiple copies. The number of replicas must be set to balance performance and fault tolerance. Too many replicas will affect performance, and too few replicas will cause data loss when the node fails.

In Distribution mode, a put operation performs num_copies remote calls, And the get operation on any node performs at least one remote call. In fact, get operations may even execute num_copies remote calls, but they are parallel. If one is returned, the results will be returned to the caller.

Read consistency

Because the get operation obtains the number from multiple nodes in parallel and uses the first returned result, data inconsistency may occur when asynchronous mode is used. Note that this situation only occurs in the asynchronous call method. If it is synchronous, there will be no read inconsistency.

Configure cache containers and caches

When using JDG Cache, you must configure the Cache to be used in the standalone/configuration/clustered. xml configuration file. If the client accesses an unspecified Cache, the operation is considered invalid.

In <subsystemxmlns = "urn: infinispan: server: core: 5.2" default-cache-container = "clustered"> ...... </Subsystem> Configure cache-container and distributed-cache. This project uses the distributed cache mode.

Add the cache configuration node to the node <cache-containername = "clustered" default-Cache = "default">, as shown in the following figure ):

<Cache-container name = "clustered" default-cache = "default"> <Transport executor = "infinispan-transport" lock-timeout = "60000"/> <Distributed-cache name = "default" mode = "SYNC" segments = "20" owners = "2" remote-timeout = "30000" start = "EAGER"> <Locking isolation = "READ_COMMITTED" acquire-timeout = "30000" concurrency-level = "1000" striping = "false"/> <Transaction mode = "NONE"/> </Distributed-cache> </Cache-container>

 

Not all of the above configuration items are required. For details about each node, see:

# Define the default cache expiration Policy

# Lifespan: Maximum lifetime of a cache entry, in milliseconds.-1 indicates that the cache entry never expires.

# Max-idle: Maximum idle time of the cache entry, in milliseconds.-1 indicates that the cache entry never expires. If the idle time exceeds this value, the entry is deemed to have expired.

# Interval: The execution interval (MS) for clearing expired entries from the cache ). If you want to completely disable the periodic recovery task, set it to-1.

<Expirationlifespan = "2000" max-idle = "1000"/>

# Define the default cache reclaim Policy

# Eviction: Specify the recycle policy. Available recycling policies include UNORDERED, FIFO, LIRS, and NONE ).

# Max-entries: specifies the maximum number of entries in the Cache instance.-1 indicates no restriction. The actual value is> = the minimum value of the power of 2 of the selected value

<Evictionstrategy = "LRU" max-entries = "10000"/>

# Define a cache lock

# Isolation: defines the isolation level. Available isolation levels include NONE, READ_UNCOMMITED, READ_COMMITED, REPEATABLE_READ, and SERIALIZABLE. The default value is REPEATABLE_READ.

# Striping: locks are striped. If it is true, the shared lock pool is used to maintain all the entries to be locked. Otherwise, create a lock for each entry. Locking strip helps to control memory usage, but may reduce the system's concurrency capability.

# Acquire-timeout: Maximum timeout time for trying to obtain a specific lock.

# Concurrency-level: the concurrency level of the lock container. This value is adjusted based on the number of threads that interact with infinispan concurrently.

# Concurrent-updates: used only for non-transaction cache. If it is set to true (default value), the cache maintains data consistency when updating concurrency. The cluster mode brings additional RPC costs. If your application does not write data concurrently, disable this flag to improve performance.

<Lockingisolation = "READ_COMMITTED" acquire-timeout = "30000" concurrency-level = "1000" striping = "false"/>

# Define cache transactions

# Although the server cache supports transactions, there is currently no protocol available to support transactions.

<Transactionmode = "NONE"/>

Start JDG Cluster

JDG can be deployed on multiple nodes to form a cluster. Currently, our project uses the data cache service to use a distributed cluster with three default nodes. The nodeName of each node must be differentiated at startup, for example, named nodeA, nodeB, and nodeC respectively.

JDG is running in the JBoss container. JBoss is started at startup. If JBOSS is already running on the server, set Port offset at startup to avoid conflict.

To avoid conflicts with the JBoss server's JBOSS_HOME, define the environment variable JDG_HOME (for example, exportJDG_HOME =/usr/local/jboss-datagrid-server-6.1.0) and replace all JBOSS_HOME in bin/clustered. sh with JDG_HOME.

Take the nodeA node as an example (the JDG port for external access is 11222, and the following offset is 11322), the server ip address is 192.168.1.100, And the startup command is executed:

./Clustered. sh-Djboss.socket.binding.port-offset = 100-Djboss. bind. address = 192.168.1.100-Djboss. node. name = nodeA

Client Access

Remote clients can use the REST, memcached, or HotRod protocols. Here we use the HotRod protocol, which is a binary protocol with good performance. It also provides automatic load balancing and failover.

Configuration File

The configuration file for client access to the cache server is named hotrod-client.properties by default. For definitions, see.

########### JDG server configuration ############

# Request balancing policy, default = org. infinispan. client. hotrod. impl. transport. tcp. RoundRobinBalancingStrategy

# Infinispan. client. hotrod. request_balancing_strategy =

# Server LIST, default = 127.0.0.1: 11222

Infinispan. client. hotrod. server_list = 192.168.1.100: 11322; 192.168.1.101: 11322; 192.168.1.102: 11322

# Whether to force the return value, default = false

# Infinispan. client. hotrod. force_return_values =

# TCP_NO_DELAY, default = true

# Infinispan. client. hotrod. tcp_no_delay =

# Whether to send a PING request at startup to obtain the CLUSTER topology. default = true

# Infinispan. client. hotrod. ping_on_startup =

# Control the transmission mechanism used. Currently, only TCP is supported. default = org. infinispan. client. hotrod. impl. transport. tcp. TcpTransportFactory

# Infinispan. client. hotrod. transport_factory =

# Serialize aller used for serialization, default = org. infinispan. marshall. jboss. GenericJBossMarshaller

# To reduce the transmission load, set it to ApacheAvroMarshaller

# Infinispan. client. hotrod. effecaller =

# Specify the custom AsyncExecutorFactory, default = org. infinispan. client. hotrod. impl. async. DefaultAsyncExecutorFactory

# Infinispan. client. hotrod. async_executor_factory =

# Specify the concurrent thread pool size, default = 10

# Infinispan. client. hotrod. default_executor_factory.pool_size =

# Specify the concurrent queue size, default = 100000

# Infinispan. client. hotrod. default_executor_factory.queue_size =

# Version and consistent Hash algorithm implemented by the Hash function, which are related to the server version of HotRod

# Infinispan. client. hotrod. hash_function_impl.1 =

# The serialization and deserialization key cache allows the number of bytes to avoid array size adjustment, default = 64

# Infinispan. client. hotrod. key_size_estimate =

# The number of bytes allowed in the cache of serialization and deserialization values to avoid array size adjustment. default = 512

# Infinispan. client. hotrod. value_size_estimate =

# Socket read timeout, default = 60000 (60 seconds)

Infinispan. client. hotrod. socket_timeout = 50000

# Socket connection timeout, default = 60000 (60 seconds)

Infinispan. client. hotrod. connect_timeout = 10000

# Specify the Protocol version used by the client. default = 1.1. Other values include 1.0.

# Infinispan. client. hotrod. protocol_version =

# Number of Retries for an error, default = 10

# Infinispan. client. hotrod. max_retries =

 

########### Connection pool configuration ############

# Specify the maximum number of connections per server. A negative value indicates no limit. The default value is-1.

MaxActive = 100

# Specify the number of global persistent connections allowed in the server group. A negative value indicates no limit. The default value is-1.

MaxTotal = 100

# Specify the maximum number of idle persistent connections for each server. A negative value indicates no limit. The default value is-1.

MaxIdle = 20

# Specify how the server responds when the connection pool is exhausted:

#0-throw an exception to the caller

#1-blocking callers until idle connections exist

#2-create a new connection (not restricted by maxActive)

# The default value is 1.

# WhenExhaustedAction = 1

# Check the time interval between idle connection Eviction threads. The default value is 2 minutes.

# TimeBetweenEvictionRunsMillis = 120000

# How long does a connection in the idle pool need to be destroyed? A negative value indicates that no idle connection is destroyed. The default value is 30 minutes.

# MinEvictableIdleTimeMillis = 1800000

# Specify whether idle connections are verified by sending a TCP packet to the server during Eviction thread execution,

# Connections that cannot be verified will be cleared from the pool

# The default value is true.

# TestWhileIdle = true

# Specify the minimum number of Idle threads available for each server. The default value is 1.

# MinIdle = 1 client access code

Import java.net. URL; Import java. util. Map;   Import org. infinispan. client. hotrod. RemoteCache; Import org. infinispan. client. hotrod. RemoteCacheManager; Import org. infinispan. client. hotrod. ServerStatistics;     Public class Quickstart {   Public static void main (String [] args ){   URL resource = Thread. currentThread (). getContextClassLoader () . GetResource ("hotrod-client.properties "); RemoteCacheManager cacheContainer = new RemoteCacheManager (resource, true );   // Obtain a remote Cache RemoteCache cache = cacheContainer. getCache ("myCache ");   // Put the data to the cache and confirm whether the data exists. Cache. put ("name", "paul "); If (cache. get ("name"). equals ("paul ")){ System. out. println ("Cache Hit! "); } Else { System. out. println ("Cache Miss! "); }   // Delete cache data Cache. remove ("name ");   CacheContainer. stop (); } }