Wang see big Data-products and technologies

With the advent of the large data age, data generated explosive growth, data equipment so that the real world can be digitized processing, data from the accumulation of the analysis and processing value.

Wang, a professor of information at Renmin University of China and Ph. D., said the big data came mainly from simple data types and structured data from internet companies such as Facebook, large enterprises, telecoms, finance and other industries. Facebook now has a powerful data warehouse. With the development of Internet and e-commerce, database technology is confronted with the challenge of massive data processing, which requires high scalability and highly scalable. The processing of data is handled by transaction processing. With the development of hardware technology, the data processing platform has moved from a single processor platform to multi-core, large memory, cluster and cloud computing platform by the data of sensor network and Internet.

When it comes to big data, Wang points out that the report from the McKinsey study points to the characteristics of the current big data, referred to as 4V (diverse, fast, huge, value) such as sensor data, traffic data Update frequency high, data value of the same time, data by structured, semi-structured, and unstructured data such as text, video, click Stream , and logs.

SQL technology uses a unified data model, strong consistency and so on, especially in the core transaction processing area can not be replaced, it provides users with simplicity, as well as the best combination of compatibility, provides a common shared platform. The extensibility of read-only parsing processing needs to be extended further.

The NoSQL technology of web systems is mainly oriented to unstructured data, using key-value processing, mapreduce processing, and highly scalable and scalable.

The fusion between database and MapReduce is mainly divided into three kinds of solutions, including Greenplum and Asterdata as the representative of parallel database, Hive and pig, correlation-oriented, HADOOPDB and IBM solutions represent parallel database-driven and mapreduce integration.

When it comes to the relationship between DB and MapReduce, Wang emphasizes that DB and mapreduce are not alternative relationships, DB cannot stand still, ignore MapReduce technology, and DB cannot Handan, lose deep accumulation and realize a db based on Hadoop, Mining the subsystem suitable for MapReduce computing model from DB, it will be suitable for mapreduce task from the core algorithm level.