You need to master these skills for getting started with big data.

I will dedicate this article to young people who are enthusiastic about data and want to engage in this industry for a long time. I hope to inspire you and adjust your ideas and directions quickly so that you can develop your career better.

Based on the different stages of the data application, this article will discuss the necessary skills of these data personnel from the bottom layer to the final application.

1. Big Data Platform

Currently, it is very popular, with data sources and various new cool technologies. It builds hadoop, honeycomb, spark, Kylin, Druid, copy ~, If you want to know Java, many platforms are developed using Java.

Currently, many enterprises have collected data. Traditional data is sufficient for traditional business data. However, many companies do not know how to handle user behaviors and click behaviors or many unstructured data, such as text, images, and text, due to the large amount of data. Storage.

How to build a real-time, near-real-time, offline big data framework, how to couple and decouple data streams, and how to achieve disaster tolerance, platform stability and availability is what we need to solve.

My feeling is: in the past two or three years, this talent is still scarce, because the concept of big data hype is so fierce that many companies are fooled to say, let's also begin to enter the big data industry. One of the prerequisites for entry is to store data, especially in many aspects of user behavior. Enterprise Progress is obvious. If you can describe users well, it will help your product design, marketing and market development. At this stage, many companies have to take the first step: store more data. This is also the reason for the relatively high mobility of employees.

The most important part of this work is the slow beehive speed, slow SQL query speed, and the cluster is suspended again. How does the data run after the hadoop version upgrade?

If you want to do a good job in this field, you also need to have the design capability of the entire system architecture, strong ability to resist pressure and solve problems, and the ability to collect resources. You can enter the open-source community, in this way, you can keep up with the latest development trends and technologies. GIS is always available.

2. Data Visualization

This is a dazzling job. It is best to know something about the front-end, such as Js. Data Visualization personnel need good analytical thinking and cannot ignore the degree of help to the enterprise to demonstrate their skills. Because I don't have many visitors who come to this article, I don't have any profound insights, but I think this article requires analysis capabilities to do a good job of visualization.

On the other hand, people engaged in data applications should understand data visualization. They should know the order of materials: Pictures> tables> text. An opportunity to describe images should not be described in words, because it is easier for others to understand. You know, when explaining things to big leaders, you need to think of big leaders as a "Data idiot" so that you can speak one thing more vividly. Big Data Learning Group 142973723

3. Data Analyst

Positioning of data analysts: for individuals, it is difficult to become a good data analyst, and there are not many excellent analysts on the market. In addition to the reasons behind data analysis, Conclusion extraction, and data insight, data analysts also need to understand their business and algorithms.

In this way, when a business problem occurs, data analysts can gradually solve the problem and then respond to the policy based on the problem location, such as the first policy test or optimization algorithm application, in this example, SCE nario is used. Can we use this calculation? Resolve the problem according to law.

Excellent data analysts are omnipotent data scientists who are proficient in business and algorithms, rather than just following business needs to pull data, make reports, and do analysis. We all say that the analysis should draw a conclusion. The conclusion of excellent analysts is a set of strategies and responses that can solve the problem. At the same time, many requirements are actively discovered by analysts and mined through data.

From the above description, we can see that the requirements for data analysts are: Writing SQL data, proficient in business, insight into data, proficient in algorithms, strong initiative, and high requirements.

4. Data Mining/Algorithms

You do not need to execute all algorithms from the beginning for skill requirements in this position. There are many existing algorithm packages to call. The basic requirement is to know which algorithm is used in each scenario, such as classification scenario. The commonly used classification algorithm is lr/RF/xgboost/ET. When the model is invalid, imize. It also needs the ability to implement algorithms. Scala/Python/R/Java can be used in languages. We often say: "tools are not important. What matters is that you play with tools instead of tools ."

In addition, for supervised learning algorithms, it is best for algorithm engineers to have a good sense of business. In this way, feature design can be more targeted. Only when feature design is available can we have a good foresight.

So many people talk about it again. In fact, the core is how to use data to create value. If you are not able to use data to create value, you can only wait for data to be drowned, killed by data in the workplace, and reach the peak of your career as soon as possible.

You need to master these skills for getting started with big data.