What is the secret of Impala to improve 3~90 times query efficiency?

&http://www.aliyun.com/zixun/aggregation/37954.html ">NBSP;

Cloudera Impala is a real-time search engine based on Hadoop Open source project, its efficiency is very high, compared to Hive promoted 3~90 times.

Why code generation?

The basis of all this is the optimal query, the engine must be a native application, because they are developed for your data format and only support your query. For example, here is an ideal code:

1 2 3 Select COUNT (*) from tbl where Col like%xyz%

This is as efficient as the grep-c "XYZ" tbl.

Another example, select sum (col) from TBL. If the table has only one int64 column, use the Little endianness encoding, which can be run by a dedicated application:

1 2 3 4 5 int64_t sum = 0; int64_t* values = (int64_t*) buffer;   for (int i = 0; i < num_rows ++i) {sum = Values[i]; }

These two queries are reasonable (because the second code is for column data), but are slow to run an existing query engine. (This is a hypothetical case of enforcement; a database can, of course, be run with an index or a calculated value, which is more efficient than simple applications.) Of course, the technology here should also be implemented using a non enforcing policy. This is because most of today's applications follow the interpretation of adding multiple execution costs.