The carding of the knowledge architecture of big data

Recently handover of the previous Big Data project, the previous project content to do a summary. Also is to comb the structure of the project, to the Prophase is a summary, for the late learning to lay a foundation.

Clean up data

For the traditional industry, come up and say to make big data, generally will be a gimmick, because before the amount of data is not very large, so basically are some statistical analysis of the main content. At this stage, your understanding of the data is especially important! The knowledge involved in this area is data cleansing and related ETL technology. In other words, you have to do data analysis, where the data is very important, when you do not know the location of your data, your analysis will not talk about. There must be a lot of problems with the raw data inside. At this point, your cleanup process is about getting the raw data in-depth. Why it's important to say that a good data analyst must be a good business person. Because only when you know more about the data, you can better complement, replace. To speak and to be popular is that you want to convert raw data into data that your PC can read.

There is also a 4:3:3 principle, your raw data to train your data from testing, training, and validating these three dimensions to make a loop that will make your data more successful in the end. This is also important when your data is being structured or unstructured when it comes to storage. It also determines the speed of your later reading!

Analyze data

This step is to be done in conjunction with the business, how much you understand the business. In combination with business requirements to analyze data, rather than simply understanding the data, different occupations in the same industry have different understanding of the same data. In contrast, business people need to understand the data more deeply. How do you analyze your data and how to understand the special values inside. It is especially important how to find the target data that you are asking for.

Analyze the data, which is also related to the success or failure of your project. This personal feeling is also a place where product managers need to be important. First of all, as a product manager, you can not understand all the industry clearly, in this case, it is bound to require you to maximize the value of the data. In this step, you need to get in-depth with your business to make sure you have a detailed understanding of the data before you can stand out in the next steps.

Algorithm selection

Some people say that this is involved in research and development, as a product manager is not required to focus on. But it is equally important from a personal point of view. Because your initial algorithm selection will result in later errors. It is the equivalent of saying that the basic things you have to choose as soon as you come up.

In the selection of algorithms, the personal feeling is to be implemented in conjunction with the business. First of all, to understand what the main focus of the business is what indicators. The parameters associated with this indicator are those that affect how these indicators are affected. As for the accuracy of the algorithm, this can be improved by thinning the granularity of the data. Different code on the system resource scheduling is different, and if you know the extent of the algorithm to determine the maximum speed of your final product response!

Demand analysis

Some people say that this piece is the most important. Why don't you put it in the first part, but put it in the last part. Because of the deep feeling, in the traditional industry, the user's needs are unclear, or not so clear. Or a user's needs can be guided. All along, the individual needs to be divided into four kinds of users: strong need, weak need, really need, fake need.

Sometimes, you need to distinguish these needs. The product manager is required to have the background of the relevant industry. Because of different industries, different companies have different needs for people. How to tap into the needs of users and transform these requirements into products that can be achieved on the ground. This is a very high requirement for the product manager.

Departmental communication

Big Data products, I divide it into three lines, one is a product, one is a business, and the other is research and development. This involves the communication between the departments. Business has a lot of user needs to go through the product of people to the research and development feedback, and research and development also need the product of people to put their work into the actual project.

Big data, for the top. The leadership may not understand what big data can do. This requires the product staff to the leadership in a popular language to understand. And for the cooperative manufacturers, to have the right guidance, can let the other side see the possibility of cooperation. So as to provide impetus for the development of the project.

Big data projects, in the perspective of a product manager, came to this project, only to find that the content that they have learned to the actual application is so minimal. The traditional industry's thirst for big data is no longer based solely on concept but on real ground, and real ancillary business creates value. In this regard, the requirements for a product manager will only be higher and greater.

I am glad that the previous period of time the paper has successfully passed the problem, big Data road is far away, and the line and cherish it!

The carding of the knowledge architecture of big data