The principle of SQL parsing based on simple SQL statement and its application in big data

based on Simple SQL Statement of SQL analytic principle and its application in big Data

Li Wanhong

SQL parsing is usually based on the analysis of Lexandyacc . , is the character by the analysis, performance is not high, if the analysis based on the SQL statement without subqueries, the speed will improve a lot, here the principle of the explanation.

     General sql select  from  where sql color:red Style= "color:red" >sql statement resolves to multiple simple child sql statement. These child sql sql syntax parsing is a revolutionary new ideas.

Big data is usually used HBASE wait NOSQL , for SQL development, it is extremely inconvenient to use a distributed relational database to preserve big data, such as ORACLE , MYSQL Save a table sub-library to multiple databases, so that you can handle both massive data and traditional programming, support MYBATIS , SPRING , HIBERNATE , the development is extremely convenient. The corresponding difficulty is the uniqueness of the ID of each library table and the processing of the SQL statement.

forIDthe uniqueness of the solution is not difficult, the key lies in the distributedSQLthe processing. WithSPARKsuch as big Data software comes into being, for the processing of distributed data is justified. can be usedSPARKto deal with the distributed data, how do you get the data? For this reason, we can use theSQLThe Analytic principleSQLstatement to parse a complexSQLthe statement resolves to a much simplerSQLstatements, such asSELECT a.*form USER A WHERE a.id>222, and then find it through a data dictionaryUSERtables, execute queries on each sub-library, and then use each query resultSPARKprocessing, for other sub-SQLstatement, and then according to the masterSQLthe conditions of the statement are as followsJOIN,GROUP by,ORDER byand so on all the query data to the Operation summary processing, and finally return the results.

In short, by getting each simple sub- SQL sentence parsing or sentence decomposition is a good and effective way to solve a common problem, especially for the use of high-efficient database processing Big data has special significance, which lays the cornerstone. This is an extremely effective solution and deserves to be applied.

2014-10-17

The principle of SQL parsing based on simple SQL statement and its application in big data