The analysis philosophy of LinkedIn Zhangxi dream: Big Data to do small do fast

In many people's minds, LinkedIn is a very different and mysterious social networking site, but its value is gradually getting the world's attention. At a recent 2013 Teradata Summit, the director of the Business Analytics Division of LinkedIn, Zhangxi, interviewed by the IT specialist network, described how LinkedIn creates value for the enterprise through its use of data analysis technology.

LinkedIn's goal is to connect all the world's professionals so they can be more efficient and successful. Currently, LinkedIn's worldwide users have grown to more than 200 million, and 86% of Fortune 100 companies are using LinkedIn's paid solutions. The "Talent solution" contributed more than half of the 161 million dollar revenue from LinkedIn's last quarter.

Behind this is the Zhangxi dream, with a Business Analytics team of less than 60 people, to support 70% of the existing 4,000 internal staff of LinkedIn through integrated data architecture, BI, data mining and analytics, covering five major business functions, including research and development, product, marketing, sales and operations, Including the company's three major business branches.

Human resources experts even claim that the LinkedIn recruiter paid recruitment service (the flagship of LinkedIn, the core of the talent solution business) is a "nuclear bomb" that will gain an unmatched position in the future recruiting market with a huge database.

How does LinkedIn do that? Zhangxi Dream Cobwebs, in-depth introduction of LinkedIn's analytic philosophy and its underlying technical support.

Zhangxi Dream (Simon Zhang), director of business analysis at LinkedIn Corp.

Analytic philosophy from pyramid to Diamond

The task of analytical work, Zhangxi dream that is "seek to break". Li lost Wei Zheng that section of "Copper for the Mirror," the famous saying no longer repeat, in the Zhangxi dream, seek to break is based on the past and now custom future, the object of course is data.

The three main types of data for LinkedIn are user behavior data, user identity data, and career network data. If 200 million of the user's data is not enough to make the current storage hardware and database pressure, then the interrelated professional network data, is absolutely a well-deserved large amount of data.

"Soldiers care about fine, do not care much", "the Soldiers your speed, not expensive long!" Ancient Chinese military strategist Criterion, is Zhangxi dream Big Data wisdom lies. He put forward two criteria, big data to do small, do fast, because speed determines value and success or failure.

The traditional pyramid structure of data analysis, from data and data quality management, Mr. Cheng Business intelligence and reports, and then specialized analysis, as well as in-depth analysis, the final form of business insights, but Zhangxi dream that, as the great painter will not borrow others to depict the beauty of the heart, analysts do not need to use ready-made reports to create the future, In other words, in LinkedIn, business intelligence reports are placed above the analysis layer.

But after the transformation of the pyramid structure, there are still two major problems, the first is the gap between the functional layer, and, more frightening, the bottom occupies 90% of the resources. The practice of Zhangxi dream, to the bottom of the "Operation", let pyramid Structure "evolution" into a diamond structure, when the pyramid base to achieve a small, the entire analysis process to reduce the area of half, the efficiency of the resources to obtain 100% improvement, and very large data into a very small data, processing speed has been a qualitative leap.

LinkedIn is not satisfied with this, once again the Diamond as a new pyramid "evolution", so repeated, to no longer "small" into the second stage of evolution, the application of spherical snowball spherical, will form a greater value ring.

Zhangxi dreams are delighted with the philosophy: "I've just started to join LinkedIn, work from 8:30 to two or three in the middle of the night, but only 500 reports a year, and less than 200 people, but now my team can help people 10 times a day." ”

The problem, however, is that there is no value in today's data, who can guarantee no value tomorrow? The consequences of asymmetric information, many companies are deeply experienced, so, as far as possible to collect data, is the advice of many experts, this is the big data is "big" one of the reasons.

"Intelligence is never enough." The Zhangxi Dream replied that the increase in data volume also meant that the cost of storage and analysis increased, the rate of analysis dropped, and often the value (ROI) dropped.

Why Choose Teradata

LinkedIn's analytic philosophy has been implemented and the power of it is certainly not. The Zhangxi Dream says technology is the cornerstone of LinkedIn's expansion of analytics. The Linkedin,hadoop, Aster data and Teradata are the three major platforms on which the Business Analytics Department operates.

LinkedIn's collaboration with Teradata is actually starting with the Aster data, which has now been Teradata acquired. Zhangxi Dream Introduction, in LinkedIn based on the social network analysis model, based on the traditional relational database analysis, multi-level relational network calculation, a few days or even one weeks to complete, and later adopted the Aster Data, analysis efficiency has been greatly improved, The current analysis time has been shortened to several hours.

While LinkedIn spends a lot of energy on open source technology and has developed a wide range of open source technologies, Zhangxi dreams are more inclined to use stable business software at the data analysis level. He said open source technology updates fast, more functions, also means instability, closed-source response is slow, but also synonymous with stability.

Zhangxi Dream said that LinkedIn is not a database company, the use of existing mature technology is more conducive to the company's business speed, and Teradata is the most mature enterprise Data warehouse Suppliers, the advantages of its solution has been verified by the market. By contrast, LinkedIn, which uses the Hadoop platform, also needs to add a layer of security in the middle to protect members ' privacy and interests.

Zhangxi Dream of the reason, can be summed up as a professional, authoritative. In fact, the deeper the two are the same as the professional understanding. Zhangxi Dream hope that large data to do small quickly, Teradata Greater China President Singlun just constantly stressed that no biting, but to learn to discard data, only analyze useful data. Teradata Data analysis method is I (integration), D (Explore), a (action). Aster data is the Teradata platform, the design of the concept is to enable people at different levels of demand can carry out a variety of analysis, easy to explore the value of large data, providing SQL, MapReduce, statistics, graphics, paths, time and geographical query tools, is right for LinkedIn's needs. The Teradata platform is primarily used to support BI.

Advice "quasi-data scientist"

In an era known as the "Big Data Age", a new career called data scientist is thought to be in the offing, including EMC, Microsoft, Teradata and other companies that are talking about data scientists, the inevitable need for data analysis in the big data age, and even comments that Data scientists are the "sexiest" occupations of the 21st century.

In the LinkedIn model, data scientists ' precise judgments are particularly important to identify which are the most valuable data, not just software platforms. Zhangxi Dream that the best analyst to understand the product better than the PM, to understand the market more than marketing, to understand the relationship between hardware and software.

It is no exaggeration to say that being a LinkedIn analyst is also a challenge. Therefore, Zhangxi dream of "soldiers care about fine, do not care more" another meaning, but also the analysis of the team's "fine."

So how do you deal with the challenges of the future into this "sexy" career? The advice of the Zhangxi dream is not to choose this business because data scientist is the current hot job, and your long-term goals are more important. One of the things he emphasizes most is interest, and that interest drives you to find a way to become professional.

Opening the Zhangxi dream, we will find an interesting thing: he used to be a neurosurgery surgeon. "I am a competent doctor, but I enjoy the numbers more and enjoy the logic." The Zhangxi Dream said.

Author: Thunder

(Responsible editor: The good of the Legacy)