Java cache technology introduction (JCs open-source project) -- index disk backup Cache

The backup cache of the JCS index disk is an optional plug-in. It mainly aims to provide a second-level cache to reduce the cache pressure on the memory. When the memory cache exceeds the maximum value, the cache processing checks whether "disk" backup cache is configured for the storage area. If the index disk backup cache is used, the cache center saves the items deleted from the memory to the disk.

Disk Index

The
Indexed disk auxiliary cache) is the fastest disk cache mode. The project is saved at the end of the cache file. The first byte of each disk project specifies the object length. The key of the Project is referenced at the beginning of the file stored in the memory ). Although this still requires memory overhead, the overhead is almost insignificant compared with the performance improvement. Based on the size of the key, 0.5 million disk entities only need about 3 MB of memory. Locating a project is as fast as searching by map, and only two disk accesses are required.

 

When a project is deleted from the disk cache, the location of the valid storage area on the storage file is recorded in an optimized array of sorting, the size of this array cannot exceed the maximum number of keys allowed in the memory. This allows the disk cache to reuse the blank area, so the file size is minimized.

 

Purgatory)

Disk cache write processing is asynchronous and improves efficiency by using a memory area called purgatory. When obtaining a project, check the purgatory and then the disk. When a project is sent to purgatory, it is also stored in the disk queue. If a project is obtained from purgatory, the project is no longer written to the disk because the cache center will move it to the memory. Purgatory is used to ensure that you do not need to wait for disk write processing, but also avoid the necessary write processing for the project boundary, and the project is always valid.

 

Persistence)

When the disk cache is properly disabled, the memory index is written to the disk and the value files are sorted out. When the cache starts, the disk cache reads or deletes the index file according to the configuration, which provides an unreliable persistence mechanism.

 

Configuration)

The configuration method is very simple. It can be done in the backup cache segment of the cache. CFF configuration file. In the following example, a disk backup cache referenced by DC is created, and the files used are stored in the "diskpath" directory.

The disk index adopts the LRU storage limitation. The maximum number of keys is configured through the maxkeysize parameter. If the maximum key size is smaller than 0, the maximum number of keys is not limited. The default maximum number of keys is 5000.

JCs. Auxiliary. Dc =

Org. Apache. JCs. Auxiliary. disk. Indexed. indexeddiskcachefactory

JCs. Auxiliary. DC. Attributes =

Org. Apache. JCs. Auxiliary. disk. Indexed. indexeddiskcacheattributes

JCs. Auxiliary. DC. Attributes. diskpath = G:/dev/Jakarta-turbine-stratum/Raf

JCs. Auxiliary. DC. Attributes. maxkeysize = 100000

 

Other configuration options

Index disk cache provides some additional configuration options

The size of the disk cache Purgatory is limited by LRU rules. The maxpurgatorysize parameter is used to set the maximum number of elements allowed in purgatory. The default maximum purgatory size is 5000.

 

The test shows that the disk cache performs better when the key and purgatory size are limited.

JCs. Auxiliary. DC. Attributes. maxpurgatorysize = 10000

When a project is deleted from the disk cache, the location of the corresponding project in the data file becomes vacant, and the index disk cache maintains a path with a null position in the data file, this allows them to be reused. The empty space is saved in an optimized sorting array --- recycle bin. When the index cache reaches the specified size limit, the minimum used items will be deleted from the recycle bin. The recycle bin setting value (maxrecyclebinsize) cannot be greater than the setting value of the cache key (maxkeysize). If the value of maxkeysize is smaller than 0, the default value of the recycle bin is 5000.

 

If the size of all items in the disk cache is the same, the recycle bin will always return the exact match with the disk cache. However, if the project size is different, the disk cache will be used at no less than and is closest to the self-free location of the project to be written to the disk, therefore, the size of the collected items is larger than that of the items actually written to the disk. In this way, blank positions cannot be used. The purpose of optimization is to delete this defect.

JCs. Auxiliary. DC. Attributes. maxrecyclebinsize = 10000

 

Through configuration, the disk cache can sort data files at run time, because the sorting process is only required when the project is deleted, so the sorting interval is determined by the number of deletion processes. Currently, there is no way to perform sorting at the specified time. If you set the optimizeatremovecount parameter to-1, the data file is optimized until the cache is disabled. The default value is-1.

 

In JCs 1.2.7 In. 0, the optimization processing is significantly improved, and temporary files are not needed when processing occurs.

JCs. Auxiliary. DC. Attributes. optimizeatremovecount = 30000

 

A complete configuration instance

In the cache. CFF file in this example, configure the cache to use the default disk cache called DC, and explicitly set the cache zone named myregion1 to DC. Custom settings are specified for all index cache configuration parameters.

######################################## ######################

##### Default region Configuration

JCs. Default = Dc

JCs. Default. cacheattributes = org. Apache. JCs. Engine. compositecacheattributes

JCs. Default. cacheattributes. maxobjects = 100

JCs. Default. cacheattributes. memorycachename = org. Apache. JCs. Engine. Memory. LRU. lrumemorycache

 

######################################## ######################

##### Cache regions

JCs. region. myregion1 = Dc

JCs. region. myregion1.cacheattributes = org. Apache. JCs. Engine. compositecacheattributes

JCs. region. myregion1.cacheattributes. maxobjects = 1000

JCs. region. myregion1.cacheattributes. memorycachename = org. Apache. JCs. Engine. Memory. LRU. lrumemorycache

 

######################################## ######################

##### Auxiliary caches

# Indexed disk cache

JCs. Auxiliary. Dc = org. Apache. JCs. Auxiliary. disk. Indexed. indexeddiskcachefactory

JCs. Auxiliary. DC. Attributes = org. Apache. JCs. Auxiliary. disk. Indexed. indexeddiskcacheattributes

JCs. Auxiliary. DC. Attributes. diskpath = target/test-sandbox/indexed-disk-Cache

JCs. Auxiliary. DC. Attributes. maxpurgatorysize = 10000

JCs. Auxiliary. DC. Attributes. maxkeysize = 10000

JCs. Auxiliary. DC. Attributes. optimizeatremovecount = 300000

JCs. Auxiliary. DC. Attributes. optimizeonshutdown = true

JCs. Auxiliary. DC. Attributes. maxrecyclebinsize = 7500

 

Use thread pool to reduce thread count

The index disk cache allows less threads than the active storage area. By default, the disk cache uses a standard cache event queue dedicated to threads. Although standard queues will shut down after a worker thread stops working for one minute, the total number of threads can be limited, which can be achieved by using the event queue pool.

The following configuration file defines a disk cache called DC2, which uses a pooled-type time queue named disk_cache_event_queue. Disk_cache_event_queue is defined at the bottom of this file.

######################################## ######################

################## Default Cache
Region #####################

# Sets the default aux value for any non
Configured caches

JCs. Default = DC2

JCs. Default. cacheattributes = org. Apache. JCs. Engine. compositecacheattributes

JCs. Default. cacheattributes. maxobjects = 200001

JCs. Default. cacheattributes. memorycachename = org. Apache. JCs. Engine. Memory. LRU. lrumemorycache

JCs. Default. cacheattributes. usememoryshrinker = false

JCs. Default. cacheattributes. Max memoryidletimeseconds = 3600

JCs. Default. cacheattributes. shrinkerintervalseconds = 60

JCs. Default. elementattributes = org. Apache. JCs. Engine. elementattributes

JCs. Default. elementattributes. iseternal = false

JCs. Default. elementattributes. maxlifeseconds = 700

JCs. Default. elementattributes. idletime = 1800

JCs. Default. elementattributes. isspool = true

JCs. Default. elementattributes. isremote = true

JCs. Default. elementattributes. islateral = true

 

######################################## ######################

################# Auxiliary caches
Available ################

 

# Disk cache using a pooled event queue --
This allows you

# To control the maximum number of threads
It will use.

# Each region uses 1 thread by default in
The single model.

# Adding more threads than regions does not
Help performance.

# If you want to use a separate pool
Each disk cache, either use

# The single model or define a different
Auxiliary for each region and use the pooled type.

# Single is generally best unless you ahve
A huge # of regions.

JCs. Auxiliary. DC2 = org. Apache. JCs. Auxiliary. disk. Indexed. indexeddiskcachefactory

JCs. Auxiliary. dc2.attributes = org. Apache. JCs. Auxiliary. disk. Indexed. indexeddiskcacheattributes

JCs. Auxiliary. dc2.attributes. diskpath = target/test-sandbox/Raf

JCs. Auxiliary. dc2.attributes. maxpurgatorysize = 10000

JCs. Auxiliary. dc2.attributes. maxkeysize = 10000

JCs. Auxiliary. dc2.attributes. maxrecyclebinsize = 5000

JCs. Auxiliary. dc2.attributes. optimizeatremovecount = 300000

JCs. Auxiliary. DC. Attributes. optimizeonshutdown = true

JCs. Auxiliary. dc2.attributes. eventqueuetype = pooled

JCs. Auxiliary. dc2.attributes. eventqueuepoolname = disk_cache_event_queue

 

######################################## ######################

################# Optional Thread Pool
Configuration ########

 

# Disk cache event queue pool

Thread_pool.disk_cache_event_queue.useboundary = false

Thread_pool.remote_cache_client.maximumpoolsize = 15

Thread_pool.disk_cache_event_queue.minimumpoolsize = 1

Thread_pool.disk_cache_event_queue.keepalivetime = 3500

Thread_pool.disk_cache_event_queue.startupsize = 1