Hadoop is not a panacea to a clear scene to avoid weaknesses

Ye Qi said Hadoop is not a panacea, can not solve all the big data needs of http://www.aliyun.com/zixun/aggregation/14294.html ">, there are many shortcomings of its own security, real-time, SQL capabilities, Certainly clear demand and use of the scene, with its long and short, in the training he will share Haodop system planning and design, construction, operation and maintenance in the telecommunications industry.

- What makes you research Hadoop technology?

In the world of Hadoop, I encountered the most complex technical problems at the very bottom of various systems, the most motivating force that attracted my long-term study. My career has gone through several different types of products, but most of them belong to the Application level. They do not involve any core technologies such as OS, file system, memory management, and network communications. When I was lucky enough to start working on Hadoop this day and gradually When I looked at its kernel code, I found out that there was all the topic that I was most interested in, so I fell in love with it.

- Hadoop's unique advantages for solving problems?

Hadoop is not a panacea, can not solve all the big data needs, but also their own security, real-time, SQL capabilities and many other defects, so we must clearly demand and use of the scene, with its long-abandoned.

In the telecommunications industry, Hadoop is best suited to deal with three types of data: network signaling, telephone bill, and business log. The three types of data are characterized by large amount (over 1T per day), structured (explicit field meaning ), Storage for a long time (up to 1 year and above), using Hive + MapReduce, HBase and other components just to meet the statistical summary, point queries and other basic needs, and replace most of the IOE products complement each other, so in these scenarios Hadoop Unique advantages.

What is the most difficult enterprise Hadoop application?

I think the most important thing is SQL, the traditional enterprise applications are SQL way to connect to the database, so migrate to Hive QL and HBase will face changes in the table structure and interface adapter problems, followed by personnel skills problems, these are enterprise applications Hadoop biggest obstacle.

- According to your understanding, what is the current development of Hadoop?

Hadoop from the Internet industry has gained broad consensus and great success. This success has become a demonstration effect and spread to other traditional industries, the current turmoil in the telecommunications industry, all kinds of traditional storage technology are faced with the trend of upgrading to Hadoop; The next is the financial sector, the current financial industry is like 2 years The telecommunications industry in the early days was in its infancy. Once the "fire of stars" in the successful case has been set, a "prairie fire" trend will be set off immediately.

- Please talk about your upcoming Hadoop training to share the topic.

At present, there are many good forums and technical blogs in China, and there are lots of analyzes on all aspects of Hadoop's kernel technology. Therefore, I do not intend to repeat these contents but intend to share the planning, design, construction, operation and maintenance of Haodop system. A few years I in the telecommunications industry Hadoop implementation experience.

- Which people should come to attend this training? What will be helpful to them?

IT director or system architect who plans to or is about to implement Hadoop within the enterprise is the subject of my training. These people have some initial knowledge of Hadoop, but are not able to plan and build a production-level Hadoop cluster autonomously, nor do they have the ability to identify which Hadoop distribution or technology providers should help them build this system.

The original link: http://www.csdn.net/article/2014-05-05/2819608-Hive+MapReduce