Unicom Huang Wenliang: "hadoop+ Xeon" to deal with large data

July 24, the Intel Large Data Forum was held in Beijing with the theme of "Big Data, wisdom and opportunity". Mr. Huang Wenliang, vice president of China Unicom Research Institute, briefed the Conference on the real case of Unicom's application of the Intel Hadoop release and servers based on the Intel Xeon processor platform to meet large data challenges.

Along with the popularity of smartphones, the vast majority of users use mobile phones to surf the internet and socialize more and more frequently, which brings a new challenge: users want to know exactly where their internet traffic is used? In this situation, Unicom started deploying Internet records for mobile internet users from last year. and other operators in different provinces, Unicom is a nationwide deployment of a set of online records access system, users can access through the online business hall of their own records.

In fact, even a simple visit to a Sina home page, the Internet records Access system will produce more than 20 records, so that the daily production of data is simply the amount of days. Unicom had previously tried to use Oracle database to solve the problem, but found that the relational database to Bai can do nothing, a query may not be a few hours of results.

"Unicom's massive data is hard to solve in traditional ways. For this we have worked with Intel to introduce Hadoop technology to solve the problem. "Huang Wenliang said.

At present, the Unicom Hadoop system is deployed in Beijing, the provinces of the data every 5 minutes to generate a file for transmission to Beijing. The use of Hadoop technology after the user's online record retrieval speed can reach the second level, users enter the number of any city of China Unicom, its internet records in only two seconds can be retrieved.

Huang Wenliang told doit that the background of China Unicom's Hadoop system was 178 two-way blades based on the Intel Xeon 5600 processor, each equipped with 14TB storage capacity. In addition, the system also has 3 Namenode node server, Warehousing service node server 24, Zookeeper node server 7, cluster monitoring node 1 and Web Query Application Service node 20. These are also x86 servers for the IA architecture.

"We've also tried open source versions of Hadoop, but it's really not as good as Intel's Hadoo release, whether it's monitoring power or stability. In the end we are fully using Intel's Hadoop release. "Huang Wenliang said.

At present, there are other large data solutions in addition to Hadoop in the market, Huang Wenliang that China Unicom chose to focus on the sustainable development of products.

Hadoop is a trustworthy solution because of the support of Intel and other industrial eco-chain software. In addition, Unicom also chose large companies as the application of the vane, the current world famous companies such as Facebook have opted for Hadoop, and their successful experience has also boosted China Unicom's confidence.

Huang Wenliang revealed that the Internet record query is only a part of the big data platform, how the future in large data to dig "gold", Unicom has a set of planning. At present, Unicom's large data solution technology prototype has been completed, and by October this year, the entire system will be completed, when Unicom will provide customers and the vast number of users with more large data services.

(Responsible editor: The good of the Legacy)