The job market is grim. The scenery on the hadoop side is outstanding.

Analysts and IT managers at the hadoop World Conference told us how important hadoop is to enterprises. Kobielus, an analyst at the Forrester Institute, pointed out that "hadoop is a new type of data warehouse, a new type of data source within the enterprise ". Compared with traditional relational databases, hadoop can store and manage more structured and unstructured data.

In today's big data era, enterprises need to pay more attention to data storage and analysis in order to develop customers and improve their business level. The data comes from various aspects, such as Weibo and website click rate, social media content. The direct question is, who will analyze the data? How to analyze the data? Apparently, professionals with advanced data analysis capabilities are needed. Specifically, their capabilities should include: data analysis, data mining, predictive modeling, natural language processing, content analysis, text analysis, and social network analysis capabilities. For hadoop, they must also learn to use mapreduce for pre-test and statistical modeling. In addition, for hadoop platform administrators, they must be competent for hadoop clusters, security management, and performance optimization. Kobielus also said, "The database administrators of teradata and Oracle exadata have begun to transform to hadoop cluster administrators. They realize that this is a brand new field ." Of course, these professionals will also bring the existing relational database related policies to the hadoop platform.

 

A sales director engaged in hadoop-related software said, "the current requirements for hadoop professionals can be divided into three types: data analysts/data scientists, data engineers, and it data management professionals."

 

The first type is hadoop data analysts who are familiar with how to select, install, and manage hadoop clusters. They will decide whether to deploy hadoop on the cloud, which hadoop vendors to choose, how to allocate hadoop resources, configure clusters, and how to debug and run hadoop applications. In this regard, data analysts in traditional relational databases and data warehouses are similar.

 

The second type is hadoop data engineers who are mainly responsible for data processing and implementing mapreduce algorithms. As enterprise hadoop applications grow, engineers with Java, C ++, and other programming experience will find more opportunities.

 

The third category is hadoop data administrators who usually have professional data scientists with SAS, SPSS, and programming capabilities. They are familiar with how to create, analyze, share, and integrate Bi in the hadoop environment.

 

The current situation is that there is a shortage of hadoop talents. Some companies have to rely on hadoop vendors to provide deployment and other technical support. Hadoop manufacturers such as cloudera, mapr, hortonworks, and IBM currently provide hadoop-related technical training courses to help enterprises deploy hadoop centers.

 

We recommend the first big data visual analysis software in China-Big Data magic mirror. If you are interested, please check it out.

The job market is grim. The scenery on the hadoop side is outstanding.