How to use large data for value cashing is a serious matter

If one day you can predict the future, what is the first thing you want to do? Buy a lottery ticket? What's the second and third thing? We'll talk about it later.

Large data is an industry, in the broad sense is in this information overload era around the mass of information generation, dissemination, collection, processing, create value of the entire product chain, in the narrow sense generally refers to large data storage and processing, data mining related industries. At present, the most common use of large data in the market is analysis and prediction.

According to my 10 years of experience in this industry, large data and the previous two years of cloud computing, and then the grid computing, parallel computing is the same industry chain several links, it is as a concept by the media and practitioners hype up. But it is undeniable that our society is in the era of information explosion, the volume of the various industries in the geometric growth, efficient use of these data can indeed bring about great changes in work and life.

Point 1: The data itself does not produce value, how to analyze and use large data to help the real business is the key

It could, for example, help control America's 17.6%GDP health spending and bring more than $300 billion trillion in cost savings and benefits to the pharmaceutical industry (from McKinsey's report)

Many people have heard the story of Netflix's "card house", saying that the company used large data to analyze users ' favorite plots, favorite actors, favorite writers, and so on, combining them to make a TV show that eventually made a huge success.

Although many people have heard the story, few people notice that the company is at the top of the big data application chain. Unlike the downstream companies that provide analytics services, provide large data storage, and provide data cleansing, Netflix, in addition to its own data generation and analytical capabilities, has the key to the ability to turn analytical results into products that are at the heart of high profitability and competitiveness.

Point 2: Pay attention to inputs and outputs in business with large data

Large data threshold is very low, you can start with an Excel, but with the in-depth research, want to use large data analysis to profit, still need a certain budget and input.

For example, a professional team, such as a large data analyst, existed 10 years ago, called BI (Business Intelligence), and the job was to analyze large amounts of data and to help develop strategies or make business decisions through modeling.

With analysts, there is a need for matching engineers to dig out valuable data from massive amounts of information.

Server: Large data Another resource to be consumed is the server, from storage to computing to bandwidth, which needs constant input.

So business companies have to consider whether there is enough budget before entering the industry, but these points also give birth to a number of new opportunities, such as Amazon is the world's largest cloud computing infrastructure manufacturers, Splunk and the previous listing of Tableau are the analyst to provide data processing services, The equivalent of a part of an expensive engineer's job.

Point 3: Big Data is not the most recent, the data has been there, the technology of data analysis has been a revolutionary breakthrough in recent years

Processing large amounts of data in the technology industry has been a problem, several revolutionary technology in the last 10 years, has laid the foundation of our current big data, including virtualization technology, Map-reduce & Bigtable, NoSQL database, Deep learning technology.

Virtualization has created today's Amazon Cloud services infrastructure, and map reduce has created the Hadoop Open-source software that helps us with high-speed cloud computing, which can now be processed in a few minutes. NoSQL database has been widely used in a large number of data and high access to the site, performance than the traditional database improved a lot.

Point 4: Many people have been quietly profiting from big data

Commercial companies through long-term research, once found through the big data to profit from the secret, in most cases or choose to solo Lele rather than the Lele. Because the analysis method and the data source once the public, the competitor will follow up, will lead to the method homogenization finally manifests in the income reduction.

Precision Marketing is a well-known area, by classifying each person's information and modeling it for different kinds of marketing. Like search engines, you search for new real estate information for a long time, the search engine will be based on your search history to determine that you are likely to be a potential buyer, the United States target department store in accordance with the user's shopping records to determine a girl pregnant and to her home to deliver pregnant women's shopping manual and fame. Taobao search for the travel backpack, on Sina can see the relevant travel supplies ads.

But in fact, big data companies are not content with this direct link to improve their competitive advantage, and they are collecting data through more channels. According to the author of these years, under the premise of protecting industry secrets, share some of the dry goods that readers have not heard before.

1, routers, before only the small box on the Internet, in the data procurement market is extremely hot channel. The reason is that with the development of mobile phone hardware in recent years (similar chips used in mobile devices such as routers and mobile phones), in particular, the geometric growth of processor computing power, small routers can already run a lot of programs, these programs in the user's Internet silently analysis of a variety of data, including you often contact the friend information, Online records.

2, network operators, operators like to browse the Web page when users insert some ads, I believe most people have encountered, and the data analysis principle of routers, operators are not satisfied with the same fixed display ads, but also with the times, using large data for accurate personalized advertising marketing.

3, basic software, such as browser, input method. Not only the computer also includes the mobile phone, you use the input method in each software submitted query request, in the browser mouse to move to a certain product but did not click and so on, these large data will be stored in the cloud, for manufacturers to analyze.

4. Financial companies.

When it comes to revolutionary technology, we have to mention the financial sector. The big data-cost problem we mentioned earlier in the paper is not a problem in the financial sector, because the benefits of new technology in this industry are much higher than the cost, so we can see that every new technology will often be used in the financial industry for the first time. Big data is no exception, in fact, many years ago, large data mining has been widely used in the financial field. Here we want to classify the gaming industry as a financial industry.

It was not until this year that the United States had a "shopping mall traffic Data monitoring system" that was applied to casinos in Macau and the United States Las Vegas many years ago. A casino is equipped with thousands of cameras, starting with a face recognition technique, starting with a customer's door, and data mining with each individual's transaction. In addition to finding unpopular people such as the old thousand, we can also pick up big clients and encourage irrational gamblers to increase their bets and earn more profits.

The stock market is another big data market besides betting industry, the stock market of a few decades ago is one to take advantage of asymmetric information to profit, now big data analysis becomes a new generation of effective tool. The stock market has a huge amount of trading information all the time, and large data analysis technology has been accompanied by the development of the securities industry.

Many of the previous articles on big data mention the Indiana University researchers found that by analyzing the mood of Twitter's messages can accurately predict the rise and fall of the stock market, hedge funds that use Twitter data to trade with Google will only find the DCM in London. The reason, as mentioned in the previous paragraph, is that commercial companies, especially hedge funds, do not easily expose their computing logic, which is the same as the "dark forest" logic in the three bodies.

But in fact, we can find that Twitter information has been widely used by hedge funds in the market. For example, as long as Hollywood actress Hathaway appears in the headlines, the shares of Warren Buffett's shares will rise. The reason is simple, actress Hathaway's name is Anne Hathaway, Buffett's company is called Berkshire Hathaway (Berkshire Hathaway), both include the word Hathaway, Shows that many hedge funds use the technology of real-time analysis of Twitter and news-Big data.

Another example: A fake message was posted on Twitter: Two explosions in the White House, President Barack Obama injured in the incident, led to the stock market, the euro and other related lines fell in the first time, the Dow 2 minutes down 100多 points, the euro is also a strong downward, U.S. stock market market value briefly evaporated about 140 billion U.S. dollars. An accidental hacking prank has led to the exposure of many hedge funds using Twitter data.

To give an example of what happened around us, we know why you should fill out a detailed form for a brokerage account. It will require you to fill in your income, investment experience, etc., because on Wall Street, there are very mature models that can predict your future earnings through the form you fill out, while analyzing a large number of transactions for different kinds of users of the corresponding marketing and service, the goal is also to improve profits.

Another big area of finance is the credit market, which has been unusually hot in the country, including Alibaba and other companies. The new peer-to-peer industry, in addition to the business model, efficiency, user sources and other traditional banks, one of the key factors is large data. Through large data can solve the credit industry's core bad debt rate problem. Alibaba through its platform users of the transaction data, ratings and other complete structured data can be very easy for users to credit evaluation, but other platforms do not have the data advantage of Alibaba, it needs more data mining to reduce the bad debt rate and profit, foreign largest peer-to-peer companies such as lending Club is after 6 years of continuous improvement to achieve the first profit, visible in this huge market, we face the challenges of the big.

(editor: Heritage)