Vertica "Error:too Many ROS containers exist"

Original link: Vertica "error:too many ROS containers exist"

Recently in the Vertica with a problem, vertica after running for some time, always appear similar to the following error

Java.sql.SQLException: [Vertica][vjdbc] (5065) error:too Many ROS containers exist for the following projections: 
;p rojection> (limit = 18078, ROS files = 12088, DV files = 5992, new files = 10)

Encounter to find a problem to have to talk about Vertica storage mechanism. Vertica writes newly written data to WoS (write optimization) by default. Then according to certain conditions (such as a certain period of time) and then put the data in the WoS into the Ros (read optimization), then Ros may be a lot of small pieces of data fragments, This is the Vertica will merge these Ros data blocks into large ROS files after a certain period of time.

The process of writing data from WoS to Ros is Vertica called Moveout operation, and the process of merging a fragmented Ros into a large Ros Vertica called Mergeout operation.
OK, now let's look at our problem. There are too many Ros in the error, the probable reason is
1. WoS write too many Ros, the reason for this is very likely that each insert/update data set is too small, resulting in too much debris generated.

2. Ros too much, and the configuration of moveout and mergeout time interval is too long, resulting in too late to do moveout and mergeout.
All right, let's see my application.
1. For the first possible reason, it is indeed the demand for our application, which we cannot change at this time.

2. For the second possible reason, we checked the Vertica data, in the Vertica the default mergeoutinterval is 600,moveoutinterval is 300. These two parameters can be viewed by using the following command

SELECT get_config_parameter (' Moveoutinterval ');
SELECT get_config_parameter (' Mergeoutinterval ');

Because our application produces a lot of ROS fragments, we think about whether we can get Vertica to do moveout and mergeout as soon as possible by reducing the interval of moveout and mergeout. So we modified the Vertica parameters.

SELECT set_config_parameter (' Moveoutinterval ',);
SELECT set_config_parameter (' Mergeoutinterval ', 30);

After modifying both of these parameters, our application did not appear again after a long time running.
In fact, on this issue, there are several parameters can be adjusted, specific information can be referred to
Vertica Tuple Mover Parameters