Matthew: An Analysis of the Evolution of Big Data

Some time ago Gartner released the magic four-quadrant on BI and analysis in 2013, while Wikibon also released its estimate of the big data market in 2013. Both reports made it clear that as analytics is becoming enterprise IT Core, former BI-ETL-EDW analysis paradigm has completely outdated, no longer applicable.

Not long after the start of 2013, a series of major events marked the rapid evolution of big data and analytics. For data analysts and business executives, 2013 is a crucial year for big data into enterprise applications.

Recently, Alteryx President George Matthew (Twitter account @ gkm1) and Maybank Bawa, Mike Olson and Scott Yara, prominent experts in big data, are about to analyze the traditional paradigm of data analysis (BI-ETL-EDW) Paradigm Replacement Some experts agree that the new data analytics platform will eliminate the delays and inefficiencies in the design and implementation of current analytics software and fundamentally rethink and define three key issues blocking enterprise data analytics applications: Data Manage, analyze transparency and user applications.

The following is an explanation of Matthew's blog in the new data analysis paradigm three evolution direction compiled compiled as follows:

First, data management

Hadoop has become the foundation of enterprise management big data technology. With recent releases of Greenplum Pivotal HD, Hortonworks Stinger, and Cloudera's Impala, Hadoop's technology innovation is accelerating and the Hadoop project sends a very clear signal that major Hadoop publishers want to deliver real-time, Interactive query service. This trend brings together two areas of masterpiece: the well-known SQL query processing and exponentially scalable HDFS storage architecture. Reference reading: Hadoop distribution upgrade, the future of NoSQL is SQL?

Second, to black box

Predictive analysis is the key for managers to make data decision. There are already many technologies available in the field of forecasting and statistical analysis to help companies understand the near future. However, the biggest problem now facing the predictive analysis is "black-box "ization. As business leaders rely more and more on predictive analytics to make major business decisions, forensic analytics need to be black-boxed: applying self-descriptive data line-ups and adding underlying mathematical and algorithmic explanations. "To black box" is conducive to business managers learn to thoroughly control data analysis tools, not only to see the results of data analysis, but also know how to get the analysis of the design principles of tools, which will help managers increase the forecast analysis Rather than rely entirely on "faith" as it used to be.

Third, the popularity of applications

Deploying enterprise data analytics applications in the enterprise still faces challenges in several areas, such as the rollout of reusable applications, the creation of best practices, organization-wide horizontal collaboration, seamless reorganization models, etc., even when the analysis goes black-box. . The popularity of applications in the end user (employee) is the key to successful data analysis. For example, building an App Store, an enterprise mobile app store that specializes in analytics applications, can often dramatically accelerate the popularity of data analytics applications.

Important features of the new data analysis paradigm:

The new data analytics paradigm is goal-oriented and does not care about the source and format of the data, allowing it to seamlessly handle structured, unstructured and semi-structured data. The ability to output valid results; the ability to provide off-box forecasting analytics to rapidly deploy analytics applications to a wider range of general employees.

Recently Gartner released the Magic Quadrant for BI and Analytics 2013 and Wikibon also released the 2013 Big Data Market Forecast, both of which make it clear that as analytics is becoming the core of enterprise IT, the old BI-ETL- EDW analysis paradigm has completely outdated, no longer applicable. The new paradigm of analysis is on the rise. Here are some future trends we can see:

Hadoop (and NoSQL) is revolutionizing how we manage big data at the petabyte level.

The rise of R and Stata is impacting the black box approach to traditional analytics academia, which also represents the trends in the business world.

Analytical applications will no longer be patent-pending to data scientists, and more analytics applications will be delivered to analysts and employees in pre-packaged content and applications.