Large data: From technical phenomena to business and social change

What is the most interesting word in the business world of the 2013? All walks of life, every corner has a large number of traces: electric business platform, shopping malls, banks, hotels ... Even if you stay indoors at home, it will also provide insight into your needs and timely delivery of the information, goods or services you want.

This is almost exactly the same as Shiji's expectation. In July 2012, the American information management expert, a former Data Warehouse programmer, published a book "Big Data: The Coming Data Revolution" ("Big Data"), with the "Chinese story" to remind everyone: Big Data! The 2013 wave of big data swept across China.

"It changes too fast and too much. This is Shiji this more than a year since the most often sigh with a word, he put these new phenomenon all boil down to the novel "Top of the data."

In his own words, "big data" mainly focus on new phenomena and challenges, "the top of the data" from the depth of history to trace the ins and outs of the data, in this kind of talk to China and the United States in horizontal contrast, "China now has the advantage of the post, The advent of data and software has provided new possibilities for us to achieve business change and leap-forward development.

Change

First Financial daily: early last year, you said in an article published in this newspaper that hope 2013 is the big data year of Chinese society, now, how do you evaluate this expectation?

Shiji: I can feel that the Chinese society has a very urgent passion and need for new technology and new ideas.

2013, large data occupy the major news media, as long as the High-tech conference will always talk about large data, even the two sessions have large data analysis. I made dozens of speeches at home last year, for companies, financial institutions, universities and governments, and I have even been invited by many small and Medium-sized City district education directors. Every forum can receive a good response and feedback, let me experience the Chinese society to new technologies, new ideas of the desire.

This also shows that big data is not just a technical phenomenon, it is a business change and social change.

Daily: How is the development of large data in domestic applications a process?

Shiji: The application phase of large data is actually a development curve of it.

The development of any new technology and idea is a gradual process, many people are constantly aware of it, and then suddenly appear a burst and rise, everyone began to talk about it and say its benefits. But in fact, the "hot word" must also be a bubble, we are now in this situation.

But at a certain stage we will find that the original thing is not as strong as we think, it also has problems. Then there will be a voice of criticism, from deified it to demonizing it. At this stage in American society, there is a constant criticism of big data, and our Chinese society is almost there, which means we are moving on to the next stage.

This new phase is more sensible, and we can be more rational and calm in dealing with large data and make it a part of our lives. At this point in the technical level will begin to develop and gradually rise to a certain level. From this perspective, big data is not like cloud computing, which is a purely technical issue, and the former is a lasting problem involving business management, business change and even social change, at least for the next few decades.

Daily: In summary, there are five stages altogether.

Shiji: There are five stages: at first it is a cognitive period, everyone is climbing and understanding. The second is a period of overheating in which everyone swarmed and participates in the discussion. Next is a cooling period, people find that large data is not omnipotent, start to cool down. Then there will be a smooth development period, the technology applied to all aspects of commercial operations and life in every corner. Eventually the idea of big data has matured and become a habitual part of our lives.

Progress

Daily: How do hardware and software improvements play a role in the development of large data?

Shiji: The role of hardware is to provide a physical basis, which can be explained by Moore's law, which greatly reduces the cost of saving data. For example, 1TB of capacity is enough to keep the entire library content, now 1TB hard drive about 45 U.S. dollars, that is to spend 300 yuan or so to save the entire library copy. The price is still falling, and may be bought in the next 20 years with a cup of coffee. Therefore, the development of hardware provides a foundation for large data applications.

Software is the tool for performing calculations. The only way to use data is to compute, and the value of the data is embodied by software. For large data, the hardware is the scale of the capacity, and software is the yardstick, the big data will eventually fall to this value. In addition, because of the large amount of data, the function of the software itself will be improved a lot. Modeling, for example, allows you to build a better model with more data. Software is the mission of mining data and making data valuable, and the appearance of large data promotes the improvement of software function.

Daily: What emerging tools have emerged in the analysis and use of data?

Shiji: The main data mining and machine learning, there is a big difference between them.

Data mining is to build a fixed model to analyze data, but the machine learning model is flexible and variable. In other words, the more data, the machine learning model can adapt itself to make it more accurate.

For example, one of the most difficult problems in graph mining is face recognition. If you do it in the way of data mining, you can build the model with the characteristics of the face. But if it is machine learning, initially just build a preliminary model, build and then take thousands of facial images to the machine, it will identify each face characteristics, and then improve the model. The original data mining in the fixed model, discriminant error rate is very high, and the machine saw more face, its recognition rate is higher, the more accurate judgment.

Data mining is a fixed pattern that does not change easily. But the machine can be in the recognition and processing of human face more and more on the basis of adjusting parameters, so that their models more and more accurate, so called "machine learning." In this case, machines and people have the same intelligence, even can produce discriminant.

One of the most important meanings of large data is integration and the other is automation. Through the integration of a large number of data, people can find the knowledge that could not be discovered before, thus producing value. Second, the machine can be made intelligent by large data, automation, which is the peak of human use of data.

New mode

Daily: How does Big Data change the business model of traditional industries?

Shiji: This time I visited the company, this is an emerging start-up that uses large data to develop a new generation of fund management and trading platforms, by collecting large, multi-source, real-time data to provide data analysis, inquiries and judgments to fund managers and investors. Communications data are also thinking about how to measure the creditworthiness of small businesses with large data, perhaps creating a new business model.

In addition, Silver Electronics AG in Zhejiang is also thinking about how to use large data technology to help Zhejiang provincial government to solve the fraud in medical insurance. Such innovations rely on rich data in the medical field. Education field also has a lot of data can be used, such as the college Entrance examination data open to let folk to dig, will certainly be able to come up with a lot of help candidates to fill the volunteer, optimize the admissions process conclusions.

In addition, there are some new applications in the public domain, and the government is trying to break the ice. For example, Guangdong Province is using large data to catch fake license plate, I learned from the Guangdong Province's letter committee, only 2013 years to catch more than 50 sets of vehicles.

This competition based on large data is a kind of refinement of competition requirements and performance. But overall, the domestic business competition is relatively extensive, many decisions are also patted the head. So in the "top of the data" I proposed that the data from "Made in China" to "China to create" the hand, but also determines the business form from extensive to fine transition.

Daily: What new business models are available for data use?

Shiji: The use of data to accommodate individual privacy protection, there has been a new business model: The user authorization to use.

There is no doubt that the future protection of consumer privacy is to put the right back to the user's hands, to user authorization to query the individual generated data.

A company that runs small micro-business loans in the United States, called Kabbage, collects much of the company's data as a basis for lending, one of which is how many express deliveries are made with UPS. But there is a problem: Kabbage to the UPS to inquire about the company's Courier records, but UPS needs the small micro-enterprise authorized permission to do so, even if the company agreed, UPS can also refuse kabbage requirements. The end result can only be that, even if the user authorized, Kabbage need to check the data to pay ups. For UPS, the data is the asset, and this is the new business model.

On the other hand, small micro-enterprises can also directly find ups, hoping to get their own data. But by its own hands, kabbage can question the authenticity of the data and refuse to accept it. Therefore, these enterprises can seek the data authentication of UPS, and then hand it over to Kabbage. As a result, UPS can charge both small businesses and kabbage.

From this example you can see the user authorization, privacy protection, data assets, third party framework how to use the data and so on, this is the future data business model.