Vertica "ERROR: Too many ROS containers exist", verticacontainers

Source: Internet
Author: User

Vertica "ERROR: Too many ROS containers exist", verticacontainers

Original article: Vertica "ERROR: Too many ROS containers exist"

Recently I encountered a problem when using Vertica. After running Vertica for a while, it always encountered the following errors.

java.sql.SQLException: [Vertica][VJDBC](5065) ERROR: Too many ROS containers exist for the following projections: <projection> (limit = 18078, ROS files = 12088, DV files = 5992, new files = 10)

If you have a problem, you have to talk about the storage mechanism of Vertica. By default, Vertica writes newly written data to WOS (write optimization), and then according to certain conditions (such as a certain period of time) write the data in WOS to ROS (read optimization). At this time, many ROS may be fragments of small data blocks, this means that Vertica will merge these ROS data blocks into large ROS files after a certain period of time.


Vertica calls the MoveOut operation to write data from WOS to ROS, while Vertica calls the MergeOut operation to merge scattered ROS into large ROS.


Now let's take a look at our problem. Too many ROS is reported in the error. The possible cause is
1. Too many ROS writes on WOS. The reason is that the data set of each insert/update operation is too small, resulting in too many fragments.

2. There are too many ROS, and the interval between the configured MoveOut and MergeOut is too long, resulting in no time for MoveOut and MergeOut.


Let's take a look at my application.
1. For the first possible reason, it is indeed a problem with our application requirements, which cannot be changed at present.

2. For the second possible reason, we checked the Vertica information. In Vertica, the default MergeOutInterval is 600, and MoveOutInterval is 300. The two parameters can be viewed using the following command:

SELECT GET_CONFIG_PARAMETER('MoveOutInterval');SELECT GET_CONFIG_PARAMETER('MergeOutInterval');

Because our application will generate a lot of ROS fragments, we thought about whether Vertica can implement MoveOut and MergeOut as soon as possible by reducing the Interval of MoveOut and MergeOut. Therefore, we modified the Vertica parameters.

SELECT SET_CONFIG_PARAMETER('MoveOutInterval', 60);SELECT SET_CONFIG_PARAMETER('MergeOutInterval', 30);

After these two parameters are modified, the preceding problem does not occur after the application runs for a long time.


In fact, there are several parameters that can be adjusted for this question. For details, refer
Vertica Tuple Mover Parameters






Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.