CachingFlowManager for Azkaban source code reading

CachingFlowManager is divided into the cache Management Part of Azkaban in the management job.

[Java],
Public CachingFlowManager (FlowManager baseManager, final int cacheSize)
{
This. baseManager = baseManager;
 
This. flowCache = Collections. synchronizedMap (
New LinkedHashMap <String, FlowExecutionHolder> (int) (cacheSize * 1.5), 0.75f, true ){
@ Override
Protected boolean removeEldestEntry (Map. Entry <String, FlowExecutionHolder> eldest)
{
Final boolean tooManyElements = super. size ()> cacheSize;
 
If (tooManyElements ){
Final Status status Status = eldest. getValue (). getFlow (). getStatus ();
 
If (status! = Status. RUNNING ){
Return true;
}
Else {
Log. warn (String. format (
"Cache is at size [% s] and shoshould have evicted an entry, but the oldest entry wasn' t completed [% s]. Perhaps the cache size is too small ",
Super. size (),
Status
));
}
}
 
Return false;
}
}
);
}

Public CachingFlowManager (FlowManager baseManager, final int cacheSize)
{
This. baseManager = baseManager;

This. flowCache = Collections. synchronizedMap (
New LinkedHashMap <String, FlowExecutionHolder> (int) (cacheSize * 1.5), 0.75f, true ){
@ Override
Protected boolean removeEldestEntry (Map. Entry <String, FlowExecutionHolder> eldest)
{
Final boolean tooManyElements = super. size ()> cacheSize;

If (tooManyElements ){
Final Status status Status = eldest. getValue (). getFlow (). getStatus ();

If (status! = Status. RUNNING ){
Return true;
}
Else {
Log. warn (String. format (
"Cache is at size [% s] and shoshould have evicted an entry, but the oldest entry wasn' t completed [% s]. Perhaps the cache size is too small ",
Super. size (),
Status
));
}
}

Return false;
}
}
);
}

In this method, I have not learned or encountered any other issues.

Next we will study it one by one.

 

Zookeeper -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

1. Collections. synchronizedMap

Let's take a look at the explanation of Collections in JDK:

 

Next, synchronizedMap

As the name suggests, synchronize is a synchronous map.

Public static <K, V> Map <K, V> synchronizedMap (Map <K, V> m) is a synchronous response that is supported by the specified ing (thread-safe) ing method. That is, a thread-safe map is returned.

 

2. new LinkedHashMap <String, FlowExecutionHolder> (int) (cacheSize * 1.5), 0.75f, true)

LinkedHashMap is a new collection class introduced by JDK1.4. I believe many people have never used LinkedHashMap.

LinkedHashMap is equivalent to an extension of HashMap. It supports Linked in the implementation of Hash.

In most cases, we use the constructor without parameters of these collection classes to create objects. What are the meanings of the three parameters here?

[Java]
Public writable hashmap (int initialCapacity,
Float loadFactor,
Boolean accessOrder ){
Super (initialCapacity, loadFactor );
This. accessOrder = accessOrder;
}

Public writable hashmap (int initialCapacity,
Float loadFactor,
Boolean accessOrder ){
Super (initialCapacity, loadFactor );
This. accessOrder = accessOrder;
} A. initialCapacity: indicates the initial capacity of the LinkedHashMap.

B. loadFactor: the load factor of LinkedHashMap. The load factor is related to LinkedHashMap resize. The default value is 0.75f.

C. accessOrder: sorting method. True is the sequence from the longest time to the most recently used, and false is the sort of plug-ins.

3,

The input parameter here is the oldest entry. In the method, you need to determine whether the entry needs to be removed from the cache.

The method identified in Azkaban is as follows: (if the cache is full) if the oldest job is no longer running, the oldest job is removed from the cache; otherwise, no operation is performed.