[Big Data Search Database Ydb] Ydb new Feature Introduction-View chart

Ydb View Chart Full Library Resource Aggregation search efficient solution

There are hundreds of tables in the business, each of which is a few hundred columns.
What if you want to search for a record with that keyword based on the keyword? Do you write a SQL for each table? Do you want each column to be spelled into the Where condition to search? No in fact, for the resource aggregation type of search can be done by a SQL, and compatible with the original table structure, the rest of the business can still be a single table query.
Is there a scenario where the amount of data in each table is not much, but your table is very many, dozens of or even hundreds of tables. If each table creates a physical table, it is a waste of resources

1. Each table to maintain its own buffer, not good on the Oom, the import performance is too poor.

2. Have to open a thread alone to maintain the watch.
3. The resulting indexes are small, but the number of files is too large, frequent merging of indexes causes Namenode to be very busy, etc.
   Now Ydb's view chart can help you solve this problem. The view representation is based on the physical table. A physical table above can put a lot of view charts, these tables unified management, unified heartbeat, unified an index, for external users of the query on the chart and the physical table query, there is no difference in use. With the view chart, the original need to create separate tables for those small businesses, the problem of building clusters alone can be resolved. A small business that can share the same cluster, share the same physical table, and between business and business is diagram isolation.
   Depending on the chart, the data is stored in its corresponding physical table by mapping, as shown in the graph, which does not actually store the data.
   

Create a View chart

create ydbview  ydbview_007(phonenum dblikebig [email protected],usernick string [email protected],ydb_sex string [email protected],ydb_province string [email protected],ydb_grade string [email protected],ydb_age string [email protected],ydb_blood string [email protected],ydb_zhiye string  [email protected],ydb_earn string [email protected],ydb_prefer string [email protected],ydb_consume string [email protected],ydb_day string [email protected],amtdouble tdouble [email protected],czylong tlong [email protected]_y_tlong_id [email protected],content dblikebig [email protected]);

1. If you do not configure, the default view chart corresponds to the physical table name ydb_physical_table, all columns are aggregated
2. There are three ways to show the fields in the view chart Combineonly,fieldonly,all
1) Specify Combineonly fields to store the data in the Aggregate column Ydbview_like field, which is not visible in the view chart, visible in the physical table
2) The fields specified in the fieldonly are available in the view chart, are not available in the physical table, and are not stored in the Ydbview_like
3) Specify all field data in the view chart and physical table can be found, and stored in the Ydbview_like a copy of the
4) When creating a table, you can

amtlong tlong  [email protected]_y_tlong_id,

This way to specify the correspondence between the view chart and the field in the physical table
5) as shown in the ① section, specify the physical table name that corresponds to the chart, and modify the corresponding physical table name by modifying the Physical_example section. The ② section specifies that, without specifying a storage policy, the default storage policy is the ' all ' policy and can be modified. ①② parts are optional, not added, default parameters physical table named Ydb_physical_table Storage Policy is all

Use the tools provided by Ydb to forge some data

cd /opt/ydbsoftware/ya100hadoop fs -rm -r /data/example/quickstart/hadoop fs -mkdir -p /data/example/quickstart/hadoop jar ./lib/ydb-1.1.9-pg.jar cn.net.ycloud.ydb.server.reader.kafka.MrMakeShuData 30000000  /data/example/quickstart/1.txt

Import data into a view chart

cd /opt/ydbsoftware/ya100sh  ./bin/ydbimport.sh -ov -t ydbview_007 -p 3000w -tp txt  -f /data/example/quickstart -sp , -fl phonenum,usernick,ydb_sex,ydb_province,ydb_grade,ydb_age,ydb_blood,ydb_zhiye,ydb_earn,ydb_prefer,ydb_consume,ydb_day,amtdouble,amtlong,content

Next we verify that the set parameters are in effect
View Chart Data preview

select * from ydbview_007 where ydbpartition=‘3000w‘ limit 10

Results

Here we can see in the view chart phonenum usernick ydb_sex ydb_province ydb_grade Ydb_age,czylong in the view chart is not visible
Next we look at the physical table

In the physical table we can see the fields Phonenum usernick ydb_sex ydb_province ydb_grade ydb_age ydb_consume ydb_day amtdouble amtlong Content
These few columns are visible,

Through the above test we can verify the previous conclusion:
Fields that use the all method are visible in both the view chart and the physical table, and can be used for fuzzy retrieval using the Ydbview_like field.
Use the Combineonly field to be visible only in a physical table and not visible in the view chart, which can be retrieved by Ydbview_like field blur.
Using fieldonly is not visible in the physical table, only visible in the view chart, and cannot be retrieved using the Ydbview_like field.

The query method for the Ydb view chart is the same as the normal Ydb physical table query, and the physical table query corresponding to the chart is different from the normal physical table.

We can see that if the physical table data that is queried using SELECT * is two columns, the Ydbview_name column stores the name of the view chart, and the Ydbview_like field stores the aggregated columns that use the All,combineonly storage method.
, if you want to return the result of the specified column, you need to specify the field you want to check after the Select, for example

select phonenum,usernick from ydb_physical_table where ydbpartition=‘3000w‘ and ydbview_name=‘ydbview_007‘ limit 10

Although it is possible to return data, this data is not used for retrieval, but in a specific way

Next we look at the ydbview_007 corresponding Ydb physical table field name through the Desc ydbview_007 method

We can see that the red box on the left is the field of view, and the red box on the right is the specified corresponding physical table field, and if the table is not specified, a field will be automatically created to associate with it, and here I see that the corresponding physical tables fields of Ydb_day are ' v_1_y_string_ ID '
Next we query this field and filter it in a specific way

 select ydbview_name,v_1_y_string_id,ydbview_like from ydb_physical_table  where ydbpartion = ‘3000w‘ and ydbview_like like ‘@@[email protected]‘ order by v_1_y_string_id desc  limit 10;

In the We see, we query the Ydb_day data with a phonenum value of 13882555844, and sort by that field, where

ydbview_like like ‘@@[email protected]‘

is the fixed notation ' @@ 字段 name @ corresponding value '

For Ydb View chart use method is introduced here, if you have questions about Ydb's functions and products, you can get trial and technical support by the following ways
(If you can't get clear QR code information, click to enlarge ↓)

Introduction to Big Data Search database Ydb]ydb new Features-View chart