International IT giants accelerate construction of "Big Data Empire"

In the late 80 and the early 90, an intriguing phrase was widely circulated at home and abroad: we are immersed in the ocean of data, but thirst for the fresh water of knowledge. This sentence vividly depicts people in the face of the vast numbers of confusion and helplessness. Just then, Wal-Mart, the world's business giant, deduced a "beer and diapers story" from its vast trading database. Reveals a pattern of behavior that is hidden in the vast numbers of Americans: Young Fathers aged 25-35 often go to the supermarket to buy diapers for their babies after work, and 30%-40% People to buy themselves a few bottles of beer. Inspired by this simple customer behavior model, Wal-Mart adjusted the layout of its goods and plotted promotional prices, resulting in a significant increase in sales. The phenomenon drew the attention of the scientific community, which extended the "beer and diapers Story" to "Association rule Acquisition", in turn, the process of extracting information and knowledge that is implied in a large, incomplete, noisy, fuzzy, random data, which is implicitly unknown to the person, is defined as Data mining.

Demand is the source of success, so Western developed countries have a storm of data mining. The business community has discovered the secrets of Wal-Mart's rapid development and has followed suit. The telecoms industry is also boiling, with companies scrambling to use data to tap this sharp weapon to solve the most pressing problems they face. Industry has also acted by digging out the rules of decision-making that govern production and management from the mountain of data.

Now, the world has entered the "Big Data" era. The US Internet Data Center points out that data on the internet will grow by 50% a year, doubling every two years. The value of large data being excavated through processing and analysis is like Solomon's treasure that attracts all walks of life. So how will this new cutting-edge technology in the future change the world? How will internet trolls lay out the world's big data markets? What potential has not yet been developed for big data?

Tech giants Gamble Big data

"There is no doubt that the potential of the Chinese market is huge and that China will grow faster than the rest of the world in big data areas."

--boyd Davis

As the commercial value of large data becomes more apparent, more and more companies join in the large data strategic layout, and the Internet giant with the advantage of data development will not be satisfied with the existing Data warehouse.

To burn money to compete

As a large technology enterprise, it is necessary to burn money. Google, Amazon and Microsoft need to pay a hefty price to ensure their global network is functioning. According to statistics, in the first quarter of this year, the capital expenditure of these three technology companies amounted to $4.6 billion, which grew by about 65%. In particular, Google, in order to build its "Big Data Empire", burning money is crazy. In 2013 alone, Google's spending on big data rose by more than 2012, to $2.3 billion trillion, according to statistics.

The Wall Street Journal reported May 8 this year that Google VCs announced a $130 million trillion investment in Flatiron Tiyatien, a cancer patient data provider, to big data companies. It will be the biggest investment in the pharmaceutical software industry ever made by Google VCs.

Google VCs is an important part of its portfolio as a large medical data-field company. They have already invested in DNAnexus and foundation dentistry, who are working to build the world's first genome database to search for, while using large data-analysis tools to help treat tumours.

And just this April, another international it giant, Intel, also announced its $740 million trillion investment in big data software rookie Cloudera, the largest single investment ever made by Intel in the data center area. According to Reuters, with Intel investment of 740 million U.S. dollars in the 18% calculation, the investment will be ready to be listed in the valuation of the Cloudera to 4.1 billion U.S. dollars.

Layout China Market

Behind the rush of the IT giants to burn the money is the growing demand for large data markets. The world's big data market has reached $6.3 billion trillion in 2012, according to a report published by a US business information service company called Transparency in market research. This is a young market, with strong growth momentum, an average annual growth of 40%, the market size is expected to reach 50 billion U.S. dollars in 2018.

Industry experts pointed out that if the big data in the 2012 and 2013 in the communications, financial, medical, media, culture and other areas of performance is not good enough, has not become an important direction of enterprise investment, then large data applications will be in 2014 in the mainstream of corporate investment. For China, the world's second-largest economy, the big data wave is also strong. While the big data is at an early stage in China, it also means that China's big data market has huge potential.

Sadie Consultant, who focuses on China's IT industry and market development, published in 2013 the White Paper on the application market research of China's big data industry, pointing out that China's large data It application market will be expected to reach 14.496 billion yuan in 2015, where smart cities, Telecommunications and finance three large-size data application market from 2013 to 2015, the annual composite growth rate will reach 114.4%, 97.3% and 104.1% respectively. The goal of many it vendors is to build more efficient, easy-to-use large data solutions that make it easier for businesses of all sizes to discover more business value from massive amounts of data.

As a leader in the big data world, it is unlikely that international it giants will allow this tempting big cake to be ignored. Just one months after Intel and Cloudera announced their cooperation, executives from the two companies appeared in China. Both sides announced that they will further promote China's large data technology and industrial development, and better serve the Chinese market and users. Among them, Cloudera is planning to set up a business organization in China this September, the business will cover direct sales, professional services, customer support, training and research and development work.

"The potential of the Chinese market is enormous, because many of the economic phenomena associated with China are huge, and there is no doubt that there are the world's largest mobile operators and the largest banks in the world." At the same time, we can also see that China now has a lot of innovative big data applications, and this huge potential makes us believe that in the Big data field, China will grow faster than the rest of the world. "Intel's previous practice and success in China's large data industries and markets, as well as China's industrial environment that encourages the development of large data technology and applications," said Boyd Davis, vice president and Data center software general Manager of Intel Corporation data Center, in an interview with the International finance daily. is a key factor in our determination to work with Cloudera to further expand the Chinese market. ”

A forecast by IDC, a market-research firm, also explains why international It giants value the Chinese market. According to the agency's projections, 2020 global new and replicated information will reach 40ZB (ZB for ze byte abbreviation, 1ZB equals 10 trillion bytes), which will be 12 times times in 2012, of which the amount of data from China will be 8ZB, compared with China's 2012 years of data volume, growth of up to 22 times times, That would increase the global share of China's data from 2012 to 2020 from 13% to 21%, which would also mean huge opportunities for China's big data and data analysis.

Big data development requires a "discipline"

"In practice, the value of data analysis and processing is to produce some practical, good results, rather than too much data acquisition and mining." ”

--kord Davis

For companies scrambling to get involved in big data marketing, the road to gold is not all risk-free. Target Corp., the US chain store group, April 29 this year appointed Bob DeRodes as the new chief information officer to replace the chief Information officer, Beth Jacob, who resigned after the data leaked.

Data Mining double-edged sword

At the end of 2013, Target admitted that a total of 110 million user information had been leaked, including 40 million users ' credit and debit cards, and 70 million additional user data. It is reported that the data leak almost covers Target's 1797 stores in the United States, which is also following the 2007 U.S. department store retailer TJX COMPANIES9000 million credit and debit card data leakage incident after the largest data leakage incident.

It was due to data leaks that, as of February 1, 2014, Target generated 61 million of billions of dollars in extra spending, with insurance compensation offsetting $44 million trillion, or $17 million trillion in actual expenditure. In the fourth quarter of fiscal year 2013-2014, Target department store net profit was 520 million U.S. dollars, compared with 2012 in the same period of 961 million U.S. dollar significantly reduced by 46%, diluted earnings per share from 2012 U.S. dollars in the same period of 1.47 dollar to 0.81 U.S. dollars, net sales fell 5.3% To $2.152 billion trillion, same-store sales in the US were down 2.5% from the same period in fiscal year 2012, due to data leaks.

Target's case is the other side of the double-edged sword of big data. Kord Davis, author of Large data Ethics: balancing risk and innovation, said: "In practice, the value of data analysis and processing is to produce some practical, good results, rather than too much data acquisition and mining." ”

Due to the excessive data mining, the enterprise's behavior has caused the customer dissatisfaction, has been criticized by the customer the accusation and the legal lawsuit. Data privacy management solution provider TRUSTe A survey released in September 2013 found that 1/3 of Internet users have stopped using a company's Web site for privacy reasons or completely terminated their dealings with a company.

Legal norms are also required

As a result, some enterprises began the big data "self-discipline". For example, Nordstrom, an upscale American chain store, used sensors from Euclid technology companies to get shoppers ' information when customers use smartphones to connect to the store's WiFi service. But the act has sparked criticism from privacy advocates, which are no longer used by Nordstrom.

However, on the issue of privacy and information security, only enterprise self-discipline, is clearly wishful thinking. In January this year, U.S. President Barack Obama instructed the president's advisers to conduct detailed research on big data and privacy issues, hoping that the brutal growth of data companies could be effectively constrained by legal regulation and a game of public opinion. The President's Science and Technology Advisory Committee also conducted a study to explore the technical aspects of the relationship between big data and privacy leaks.

Michael Walker is a managing partner for system integrators and information Technology service provider Rose Business Technologies in Denver, USA. He has now drafted a 12-page code of Conduct on data analysis and use to regulate the behaviour of data scientists, which encompass everything from the role and responsibilities of data scientists.

"Companies are also beginning to understand the dangers of two use of data information and misuse of personal data. "Once companies begin to have this idea, they would very much like to have a code of conduct to regulate them so as not to cause unnecessary trouble," Walker said. ”

Michael Walker says data scientists should also use a number of moral codes to discipline their behavior, as well as doctors or lawyers. To this end, he developed a set of guidelines and norms for those who manage and analyze data to regulate the conduct of these data scientists and to protect the privacy of customers.

Super Artificial intelligence constructs the future

"Large data may be defined at some point in the future as a true restoration of the human world and to continually satisfy any of our desires." ”

--Huai Jin Peng

The popular British drama "Black Mirror" has such a thought-provoking plot: Martha's boyfriend Ash is a social media fan, almost every day on Twitter, Facebook such websites. Worst of all, Ash died. At Ash's funeral, Martha's friend Sarah said they could use ash to restore ash from the "life track" he had left in social media.

So, through the data analysis and mining, statistical comparison and re organization of Ash in the Internet world, then discovered the law, constructs the model, finally realizes the accurate "forecast", the Martha side appeared a very powerful "big Data + Cloud computing" "The reconstructed Person", A robot that even thinks personality can and dies prematurely because of a car accident [0.6% Capital Research Report]. "Robot ash" can talk to and react to and even learn from living people like "real ash."

In the future, will big data bring us such subversive changes?

In this year's Baidu large data opening conference, academician of the Chinese Academy of Sciences, the current Beijing Aerospace University President Huaijin Peng's speech "Super Artificial Intelligence: the Future of large data" seems to find the future of artificial intelligence can reach the possibility.

To achieve super Artificial intelligence, Huai Jin Peng first mentioned the "crowdsourcing" concept. This was proposed in June 2006 by Jeff Howe, a journalist at Wired magazine in Jeff Hau. Jeff Hau's definition of "crowdsourcing" is: "a company or organization that has outsourced tasks previously performed by its employees to a neutral (and usually large) mass network in a free and voluntary fashion." Crowdsourcing tasks are usually carried out by individuals, but it is possible to take the form of individual production that relies on open source if it involves tasks that require more than one person to collaborate. ”

Huai-Peng assume that at present we can do large data software development, so large data crawling machine can read such as Sina Weibo, Baidu Index, Baidu Bar or Taobao transaction data, found that the user's various emotions and needs of the curve, Software developers then based on the presentation of these data to develop a software model and then to the operator placed in the cloud, and the user to participate in the various cloud generated by the software, which produced a variety of behavior, so the machine based on the behavior of these users, for the software modeling, planning.

This is a highly precise interactive data mining technology, provided that the solution and storage problems, all possible. The future of large data software will not be an inherent form, but a constant based on the data automatically changes in the super ecology, may not rely on product managers to promote, but rely on algorithmic engineers to promote, so that the user's needs naturally exposed, and then for them to implement some functions.

If it is true that such large data software can be constructed, the bosom enters the view that the current definition of large data will be completely overturned. "Big data may be defined at some point in the future: The true restoration of the human world, and the constant gratification of any of our desires, once we relied upon it to make decisions about something, and now we rely on it to get directly to what we want to do, and all our actions have become part of our decisions. ”

Whole army prepares for China Big data

--Interview with Intel company data Center vice President and Data Center software Department general manager Boyd Davis

Cloudera company co-founder, chairman, chief strategist Mike Olson

The International Finance newspaper: Why did Intel and Cloudera choose each other as their strategic partner?

Boyd Davis: We are committed to technology to achieve and promote economic development, Hadoop is such a very basic technology, it represents a new way of using information, can be used to promote economic development. By working with Cloudera, Intel is able to participate in the process by allowing a large open source data software platform such as Hadoop to better fit our chips, in line with our growth strategy.

Mike Olson: The era of Internet of things is going to be huge, capacity is much larger than it used to be, and we need to have a strong ability to capture and analyze the data. As a global leader in this field, the collaboration of Intel and Cloudera will give rise to more innovation in chips and software that will help more business users do risk control and data analysis, which can drive the development of large data industries and applications.

International finance: What is the main attraction of China's current big data industry and market?

Mike Olson: China's health care industry, Smart city, intelligent transportation, financial services, telecommunications, etc., has shown a large number of opportunities in China's big data market, I believe that from a broader perspective, the Internet will also be a huge growth engine, the relevant applications may involve more industries and areas, such as transportation and manufacturing. It is believed that the Internet of things will bring about a larger growth effect or create a greater market opportunity.

International finance: What are the unique challenges that big data applications will face in the Chinese market?

Boyd Davis: In the Big Data application model, China and the rest of the world are relatively similar, the main difference is that China's population and the economy, the market and the size of the enterprise is larger. Therefore, both enterprises and government leaders need to exert imagination and creativity as much as possible, and to make better use of the vast amount of data in order to continuously improve and create new services and experiences. In addition, the relevant laws and regulations and policy challenges are objective.

International financial newspaper: How should we protect the security of user data in the Big Data age?

Boyd Davis: The protection of data privacy or personal information is critical to many industries, especially the financial sector or the healthcare industry. When Google, Facebook and Yahoo are just starting to develop big data platforms, security is not doing a good job. But over the past few years, the level of data security, the level of privacy protection has improved very quickly, and related security technology, in our financial services and medical industry users have a lot of deployment and application. For Intel, we have a rhino open source project, and its security and encryption technology is doing very well across the platform. For Cloudera, there are also sentry projects that are focused on security, which can also be tightly controlled for data protection. Now our cooperation will be able to achieve better data security and personal privacy protection capabilities. Data security throughout the industry is the need for all manufacturers always strive for the goal, our efforts have made it a lot better than the past 5-6 years.