Newsql database manufacturer Nuggets industry Big Data

When it comes to big data, many people first think of internet companies, such as Google, Baidu and Alibaba. Internet companies in the large data analysis has indeed walked in the forefront of the industry, its aura also obscured the industry's big data shine. IDC defines big data with "4 V": volume represents greater capacity, produced represents a variety of data, velocity represents faster processing speed, and value means that large data can create more value. Tianjin Nanda General Data Technology Co., Ltd. (hereinafter referred to as the General) chief technical Officer Vounie said that if the "4V" standards, the industry large data and the Internet large data in the amount of data, data types and data processing speed of the same level, The only difference is that the industry's big data is more dense than the data on the Internet. As a result, the industry's big data brings more business opportunities to database vendors than big Internet data.

Database three points World

If backtracking is the source of the change in the architecture of data processing, it is necessary to start with a paper from the famous American database scientist Michael Blake (Michael Stonebraker). Michael Blake points out in his paper that the trend of industry technology is to transform all applications from one architecture to support multiple applications with multiple architectures. In the context of large data and cloud computing, this theory has led to a large fission of the database market: The database market is divided into three camps, including oldsql (traditional database), Newsql (New database) and NoSQL (non relational database).

From a technical standpoint, the typical feature of Oldsql is row storage, relational and SMP (symmetric multi-processing architecture). Oldsql's representative products include TimesTen, Altibase, solidDB and Exadata. The traditional relational database represented by Oldsql can not meet the requirement of large data for large capacity, high performance and multiple data types. To better meet the needs of cloud computing and large data, Newsql and NoSQL stand out, and have a great deal of later-living.

NoSQL's technology is mainly from internet companies, such as Google, Yahoo, Amazon, Facebook and so on. NoSQL products are widely used in Key-value, MapReduce, MPP (large-scale parallel processing) and other core technologies. In the Internet Big Data application, NoSQL occupies the dominant position.

Vounie that the Newsql database has a very high commercial value and will become a mainstream database product category. "The database industry is at a turning point in technological change, driven by large data needs." Globally, there are at least 30 emerging database vendors and about 50 new products pouring into the market, and the traditional monopoly of the market by several database manufacturers will eventually be broken. "Newsql, based on a relational model, has innovated core technologies such as storage structure, computing architecture, and memory usage," Vounie said. In the future, Newsql and NoSQL will change the oldsql of a framework that serves all applications, and the three categories of products will each have the applicable application types and customer base. ”

The user's strong demand for high processing performance has driven innovation in the database industry. To further improve the performance of the products, the Newsql, NoSQL and Oldsql of the three camps have adopted a number of new technologies, such as distributed computing, distributed file systems, memory technology, and actively adopt some new hardware, including large memory, flash and high-speed network connectivity technology ( Million Gigabit Ethernet and InfiniBand). In contrast, NoSQL and newsql improvements in technology to meet the needs of large data, such as newsql products commonly used in the column storage technology, and NoSQL products widely used key-value technology. Vounie said: "NoSQL and Newsql in the processing of large amounts of data have shown a strong ability to expand." The main advantages of NoSQL are in the processing of unstructured data, while newsql support for full data formats is becoming more sophisticated. In addition, Newsql is more advantageous than nosql in real-time, complex analysis, instant query and scalability. ”

Traditional relational database is not easy to expand and parallel processing, so it is difficult to deal with massive data. In large industry data applications, analytical data management systems such as the NTU General Gbase 8a will replace traditional databases. At present, a large number of public cloud databases are based on NoSQL technology, such as HBase, BigTable and so on. These products are non-linear, distributed, lateral expansion and other technical characteristics of the Internet industry is very suitable for cloud computing and large data processing, but the application type is relatively simple. Large-scale industry data application requires that the database has a multiple table association analysis capability of complex data, which can guarantee the consistency of data and be easy to use. This demand directly promotes the development of new database technology based on cloud architecture. Based on the traditional database, this new type of database uses the Shared-nothing cluster to improve the scalability of the system, including the EMC Greenplum, HP Vertica and the Gbase 8a MPP cluster.

Vounie the future development trend of the database: to provide better support for all data types, using a larger MPP and data management cluster technology to achieve cross-platform integration, large data integration machine will be popular.

Opportunities for China's big data makers

The database market was formed in the 80 's of the last century. In the past more than 30 years, the global database market basically by the United States manufacturers (its database products mainly to deal with the main) monopoly, not only Chinese database manufacturers difficult to find a breakthrough, even Germany and Japan's manufacturers are struggling. With the rise of cloud computing and large data, a new type of database which mainly deals with the application of analytical classes has received increasing attention. Big data has given Chinese database makers a chance to challenge traditional database vendors.

2013 is the year of large data applications. According to the reporter understand, China's three major telecom operators, CCB headquarters, postal Reserve Bank, Huaxia Bank, PetroChina and other units have completed or will be completed in the first half of this year, the technical selection of large data, product testing and application planning. "Our large database product Gbase 8a has entered the test list for these projects," Vounie told reporters. ”

In the Chinese market, the Internet large data and industry data two markets coexist, and there is a huge space for development. The Internet market and the enterprise-level market, which is represented by enterprises such as finance and telecommunication, are actually two distinct markets. "There is a very different demand for it from internet companies and businesses. "The head of a server manufacturer told reporters. In general, internet companies have a large number of their own research and development staff, whether hardware or software such as big data are inclined to develop their own, and open source software. The Chinese database manufacturer, represented by NTU, has become accustomed to dealing with commercial enterprises and focusing on relational databases, making it difficult to find a breakthrough in the big Internet data market in a short time. On the other hand, the industry's big data market is large enough to give a lot of opportunities to manufacturers like NTU.

Vounie the industry's big data market into four categories: business class, management category, regulatory category and Professional category. In the case of business class, telecommunication bills, financial bills, power dispatching and smart grid are all belong to the large data application based on structured data. China Mobile, a provincial province, will add 300TB of data a year. This shows that the industry big data market promising.

The most critical of enterprise users is the performance of the database. Different from the traditional data processing, one of the main features of the large analysis is to deal with the data in real time. The South General Gbase 8a Large data platform is the location of analytical class application and full data processing, its biggest bright spot is has the high performance. Gbase 8a is able to achieve high performance, relying mainly on two technologies: one is the column storage database, the other is the new shared NOTHING+MPP architecture technology. Unlike row storage databases, each column of a table in the Gbase 8a column store database is physically stored separately, each column is organized in packets, and only the columns that are accessed and queried generate I/O. Therefore, the greater the number of columns in the table, the more I/O efficiency of the Gbase 8a column storage database, the more obvious the performance advantage. In addition, the Gbase 8a MPP cluster architecture is the most suitable architecture for handling large data. Compared with the traditional shared disk architecture, it has more lateral scalability and higher performance, and can be dynamically scaled.

More than 90% of the data in internet large data belong to unstructured data, while the industry large data is mainly based on structural data processing. Compared with those internet companies that have to face big data challenges from the day they are born, traditional companies are now faced with larger data pressures, more complex and more variable data structures. In the industry big Data application, the relational database still is the mainstream, but its technical connotation has the new change, the column storage database, the distributed computation and so on the new technology beginning obtains the widespread application.

Vounie said that from the product point of view, the domestic new database and similar products in foreign countries in the same starting line, and in the cost, local services and customization of the scheme than foreign products more advantages; From the industry trend, "x86+linux" architecture and cloud computing is gradually accepted by industry users, More Chinese companies are starting to find more cost-effective solutions locally, thus reducing dependence on foreign products, and from the perspective of information security and independent innovation, large-scale domestic data solutions are becoming more and more popular in some major projects in government industry.

RELATED LINKS

South General's largest newsql cluster demonstration

March 8, the South big general in Tianjin Hai Tai Green development Base held a "domestic new large data platform open day" activities. South Big General to customers and partners to show their gbase industry large data large-scale cluster processing platform, and conducted a variety of business analysis business scene demo, of which 200TB industry data processing program demonstration is particularly eye-catching. This demo uses 80 high-end servers, 5 million gigabit switches, spanning 7 cabinets. Vounie told reporters that the test platform, whether from the network deployment, testing complexity or the amount of data measured, are called the largest domestic newsql cluster environment. The test results show that the platform can support PB-level data query and analysis, which is a reliable platform for large data analysis in industry.