"Dumbbell" phenomenon in large data applications

If the software is devouring the world, as Mark Andriessen, a well-known VC, says, the big data should be saving the world, right?

In the past two years, the word "big data" has been abused in newspapers. It generally refers to a series of techniques used to analyze massive amounts of data that exclaiming conventional tools. The "Big Data" boom has left many executives wondering if their companies are going to come up with something like that. This phenomenon in many ways looks much like the last century in the 60 's, while the computer is still in its infancy, although unusually expensive, but its futuristic color still makes many big enterprise heart fold unceasingly, hence it is regarded as a kind of competition-friendly tool. So where do companies now face the big data wave? Should fear it or embrace it bravely? Besides, who really needs this thing?

In order to see through the lively doorway, Fortune magazine called Gaurav Dillon's office in San Mateo, California. If you think Dillon's name is familiar, it's because Dillon was the founder and CEO of Informatica. Informatica is headquartered in Redwood City, California, with a market value of nearly 4 billion U.S. dollars, the main business is for large enterprise management database.

Dillon became chief executive of data integration company Snaplogic in 2009. He sees big data as a rich business opportunity for big companies--but only for certain industries. He described the situation as a "dumbbell" phenomenon in large data applications. The following is a transcript of the phone interview, partially edited and streamlined for clarity.

Fortune: There may be no more fire than "big data" last year, and it's almost everywhere-for example, in the keynote speeches at the Tech summit, in a variety of briefing materials and panels, in news articles about various industries ... Everyone thinks they need to get big data. But big data is a very specialized type of computing, right? Or is it just a gimmick?

Dillon: I'm in the information technology industry for 22, and I have some ideas. In 2002, I used the term "information tsunami" to describe it. Now we have a term.

I think the amount of data that needs to be managed now is really getting bigger. The industry first started in the last century, and before the internet was invented, it was first to deal with the barcode and UPC code data of the retail industry. Early analysis of these data gave birth to the later data storage industry. Later, the industry led to market decisions, pricing decisions, retail forecasts and so on.

The hot trend of big data will continue, and there will be no sudden change. One scientist once said, "Science buries a bit of the past in advance every time." "So I think we can continue to enjoy the benefits of using data for decision-making and making more rational decisions with big data."

The data we need to deal with is really "getting bigger". Of course, there are more things in my garage than we did 10 years ago, and as time goes on, things are going to be more and more.

But interestingly, big data has elements of data science, which I think is more important. First, it retrieves small data from large data and then looks for signals in small data to understand what we should do next-for example, who will win the election? What is the correlation between climate and language? That is, we can now do something that we can't handle with the computational power of the last century. And now Hadoop and other tools have made big data popular. Therefore, the price and performance of large data calculations have undergone fundamental changes.

In some cases, the benefits of large data are obvious; in other cases, the effect of large data is exaggerated and its benefits may not be so obvious. As many things become more electronic--such as supermarkets, bridges, automobiles, highways and so on--you get a lot of information when you have their sensor data. But more data doesn't make people smarter, it just means spending more money to store the data. It is this aspect that will allow some companies to be thrown out of the market-that is, the benefits of large data.

In some areas, such as retailing, pricing, and finance, the benefits of large data are obvious. But in some industries, the answer is not obvious, if you invest money in big data or in research and development and in the market, which brings more benefits. I'm not telling you that big data is a panacea, but telling you to manage the data ... Different people get the same benefits.

In Mad Men, a newly updated episode last week, the Sterling Cooper & Company bought a new IBM 360 mainframe and placed it in one of the original conference rooms. Some characters in the play want to buy the computer in order to gain a competitive advantage, and some people support buying the computer because they think of it as a future trend. Others worry that the computer will replace their work. Is that what people think of big data?

The fear of computers is not just that they have. Just graduated college students, 2000 years after graduating and my children (a 13-year-old, 6-Year-old) This generation, they are not afraid of computers-although they may not be programming, but they are very fast in technology, all are folk masters. And the "Insight program" of nine snakes plotting to subvert the world in Captain America 2 (captain America:the Winter soldier) renders all the darker side of the big data. In fact, the big companies now think of "We can't be left behind," so we have an arms race in this field. Although there are concerns in the community that big data could lead to a conspiracy such as "Insight Plan", the business community has no such concerns. But there are also problems in the business world that have either acquired the wrong data or failed to really understand the meaning of the data--exactly the same as it was fifty or sixty years ago. At Snaplogic, we are now trying to complete some unfinished business. Why is it so difficult in 2014 years?

I feel that many industries are embracing big data--manufacturing and financial services, for example--because people have the ability to operate computers skillfully. But I think people are anxious to see what benefits large data applications will bring to them in their lives. They are more concerned about big data, and they really just want to enjoy the benefits of big data. This requires a lot of work. And now there are too few data scientists, such as hadoop companies are rare, you need a computer science professional graduate to build these things up. Large data has fundamentally changed the unit bit cost of data storage, which is a structural change.

Now around the big data already can clearly see a "dumbbell" posture formation. Industries with a large number of knowledgeable employees, such as services and information-intensive industries, are clearly gaining significant benefits from large data. There are retail, hospitality, stock trading ... If you have the ability to find trends, you will be able to identify the boundaries of your business and then take appropriate action. If you find out how to use some of the events in the market to move, you can definitely turn it into cash. This is one end of the dumbbell.

The other end of the dumbbell is the industrial Internet. I find it particularly interesting. "You have to sell not only the aircraft engine but also the peripheral value of the aircraft engine," a General Electric company article wrote. So we need to inspire some action around big data. For General Electric, for example, it is a preventative maintenance of an already-sold engine. The idea is of great importance to GE, Siemens and any other manufacturing company. You might think that big data is just a matter of knowledge, but it's also a very interesting "dumbbell" in industry.

But for other industries ... Can you predict trends and popular colors in the fashion industry? What elements can make a season's fashion a success? Maybe big data can do it. Another good movie is a good movie, the big data itself makes a good movie. Sometimes you can only create something step. A good book, a movie, only know when it's on the shelf. The theory of "dumbbell" seems very tenable.

So should we tell some companies: "Big Data isn't for you"?

We should find out the problem. Because if we don't figure it out, people will be very unhappy. You can't have a problem hanging over your face, but to get a problem done. A lot of people just want to start making big data right away, but if you don't want to invest in an effective level--it takes a big investment--just a little bit of money and you expect to reap huge rewards, that's not going to happen. So if you don't have enough budget this year, maybe you should wait, because the technology will get cheaper. So you might as well sit down, it's best to use software as a service (SaaS) and cloud applications to cheer your company up and let your marketing department go hard.

Fundamentally, corporate executives are investors. What do executives do? As one of our investors, Ben Horowitz, says, they are not doing things, they are making decisions. Nothing is more tragic than a half-baked big data project. Doing so will only make you resent and distrust the real benefits of big data.

What are some of the space available in the Big data market? What areas or industries are big data that can easily be conquered but are still completely open?

All these changes are turning the so-called negative space (the space between the connected things) into a battlefield. If these areas don't communicate with each other, it doesn't make sense to spend as much money as you yourself. But because of the great changes, we see a lot of negative space. For example, people are shutting down traditional data storage warehouses, and we find that many business applications are moving to the cloud. Salesforce is doing this, workday is doing well, there are APIs, Internet of things, data ... Big data is still at an early stage of development, but it is likely to be the best source of information ever. How many barcodes can you have? You will certainly see the ability to make big data in industry.

Combining negative space is a big problem. But they are still blank. We still have a long way to go.