Reproduced Vertica "Error:too Many ROS containers exist"

Recently encountered a problem when using Vertica, Vertica after running for a period of time always appear similar to the following error

  
  For
  
  new files = 10)

If you encounter a problem, you have to talk about the Vertica storage mechanism. Vertica writes newly written data to WoS (write optimization) by default, and then writes the data in the WoS to Ros (read-optimized) according to certain conditions (for example, a certain period of time), when it is possible that many of the Ros are fragments of very small chunks, This is the Vertica will merge these Ros data blocks into large ROS files after a certain period of time.

The process of writing data from WoS to Ros Vertica called Moveout Operations, and the process of merging scattered Ros into large Ros Vertica called Mergeout operations.

Well, let's take a look at our problem now. The mistake is that there are too many Ros, and that's probably because
1. WoS write too many Ros, the reason for this is that the data set is too small for each insert/update, causing too many fragments to be generated.

2. There are too many Ros, and the time interval between the configured Moveout and Mergeout is too long, resulting in moveout and mergeout being too late.

Well, let's take a look at my app.
1. For the first possible reason, it is really a question of the needs of our application, which we cannot change at this time.

2. For the second possible reason, we checked the Vertica data, in Vertica the default mergeoutinterval is 600,moveoutinterval is 300. These two parameters can be viewed by the following command

  SELECT Get_config_parameter (' moveoutinterval ');
  SELECT Get_config_parameter (' mergeoutinterval ');

Because our application itself produces a lot of ROS fragments, we think that we can make Vertica moveout and mergeout as quickly as possible by reducing the interval of moveout and mergeout. So we've modified the parameters of the Vertica

  SELECT Set_config_parameter (' moveoutinterval ',);
  SELECT Set_config_parameter (' mergeoutinterval ', +);

After modifying these two parameters, our application did not appear again after a long time of operation.

In fact about this problem, there are several parameters can be adjusted, the specific information can be referenced
Vertica Tuple Mover Parameters

Reproduced Vertica "Error:too Many ROS containers exist"