Five trends to dominate big data technology in the 2015

Big Data technology has evolved at an unusually hot pace since its inception, and there are indications that this trend will continue in 2015. John Schroeder, co-founder and CEO of MAPR, predicts that there will be five major trends leading to big data technology in 2015, and MAPR is a professional company dedicated to Hadoop distribution.

In just a few years, big data technology has evolved from a pre-hype stage to one of the core technologies of the new digital age. In the 2014, the big data plan was slowly moving from the testing phase to the development and production of the company. Schroeder said that in 2015, the company's big data technology will be further advanced and forward, and even generate more user stories, especially real-time user cases.

"Next year, companies and related organizations will be deploying big data and moving forward to real-world applications," Schroeder said. This is largely driven by a major shift in the industry today, where leaders in the current industry and new leaders in the industry are already consolidating new big data platforms with their "run" data analytics to have a certain impact on their business. ”

The five trends that dominated big data technology in the 2015 are as follows:

1. Data flexibility becomes the focus

As traditional databases and data warehouses run slower and more difficult to meet the needs of enterprise business, the flexibility of data becomes an important driving force for the development of Big Data Technologies (warehouse). Schroeder points out that data flexibility will become increasingly important in the 2015 as businesses move from simply collecting and managing data to truly using it.

"Traditional databases and data warehouses are expensive to run, so DBAs (database administrator) resources need to flatten and structure the data," Schroeder said. High DBA upfront costs hinder access to new data resources, and this rigid architecture is difficult to change over a short period of time, with the result that traditional databases become inflexible and do not meet the needs of most enterprises or organizations today. ”

Early big data projects focused on the storage of target data resources and were not very concerned about how much data was being managed, and the enterprise needed to shift its attention to the flexibility of measuring data in the future. How do you process or analyze data to have a certain impact on your operations? How can companies adjust and respond when user preferences, or market conditions, competitive trends, and operational status change? These issues will lead to the investment and development of big data projects in the 2015.

2. Enterprises gradually turn from data lake to the development of processing platform

In some cases, the big data area of the 2014 is actually a "data Lake" state, an object-based data store that stores collected data in its most native format (structured, unstructured, or semi-structured) for future use. The data lake has a high value proposition that represents an extensible infrastructure that is very economical and super flexible.

Schroeder said that the 2015 "Data Lake" will continue to evolve as more computing and execution capabilities drive companies to transform their data from a large collection of data into appropriate processing. This will not only lead to greater efficiency, but also a single point of management and a single point of security.

Next year, as enterprise data is processed in batches to real-time processing, and as enterprises integrate Hadoop and databases into their large-scale processing platforms, the "Data Lake" will be further developed. In other words, this trend does not mean that the data lake is massively used to support larger queries and reports of data storage, but to continue to access and process data, while businesses will focus more on real-time data and take proactive measures.

3. Self- service Big data becomes mainstream

Schroeder points out that with the development of big data tools and services, the IT industry will gradually ease development bottlenecks in 2015, and many business users and data scientists will be accessing large amounts of data with tools and services.

2015, self-service big data becomes a trend in the IT industry that allows business users to reach big data through self-service. Self-service can also help developers, data scientists, and data analysts work directly with data exploration and processing.

Prior to this, IT industry required a centralized data structure, but it was very time-consuming and cost-intensive. For some user cases, Hadoop can help the enterprise adapt to the structural pattern of structure-on-read. Some advanced organizations will meet continuous demand through data-bound operational patterns rather than a centralized structure. This self-service model will enable enterprises to make better use of new data resources while capturing new market opportunities to address problems and challenges.

4. Hadoop Supplier Integration new business model appears

In 2013, Intel released its distribution of Hadoop, which caused a stir in the industry. In this distribution, Intel says, they have adopted a completely new approach to integrating Hadoop directly into the chip. But only a year later, Intel abandoned the distribution and was quickly replaced by a Hadoop distribution vendor, Cloudera.

At the time, Intel said users were looking at how the Hadoop market would evolve and grow. And the vendors of Hadoop products are diverse and mixed. Schroeder believes that with the failure of the enterprise to continue to develop and launch its Hadoop distribution, instead of focusing on other areas, in 2015, the integration of Hadoop supplier will become a trend (+ focus on the network World), continue.

"Open source software (OSS) has been used for more than 20 years now, and it has brought great value to the market," Schroeder said. "The development of technology requires several stages. The life cycle of technology begins with the innovation and development of highly differentiated products and ends with the final commercialization of the product. Edgar Frank Codd, known as the "father of relational Databases", introduced the concept of "relational database" in 1969, and in 1986, Oracle successfully went public with the development of this innovative concept. At the same time, this innovative concept has facilitated the commercialization of the first MySQL release in 1995. Therefore, from the historical point of view, the database platform technology from the initial innovation to the last commercialization, has gone through 26 years to mature.

"Today, Hadoop is still at an early stage of the technology maturity cycle and it's only been 10 years since Google released the influential mapreduce white paper," Schroeder said. Since the initial concept was developed over the past 10 years, Hadoop has been deployed and used globally, and vastly surpassed other data platforms. "At the moment, Hadoop is still in the innovation phase, so the vendor mistakenly takes the" Red Hat for Hadoop "strategy that still exists in the market, most notably Intel, followed by EMC Pivotal.

Schroeder that the industry will see a new, more subtle evolution of OSS models in 2015, and that the evolution and development of this model combine community development with deep innovation. The open source community is an important area to create standards and reach consensus, and competition is an accelerator that facilitates the development and evolution of Hadoop from the original analytics processor to the full-featured data platform.

5. Enterprise Architects no longer hype Big data

In 2015, as Enterprise architects learned more about the Hadoop technology stack, they also slowly played an important role in better defining and more complex representations of the needs of big data applications, some of which included high availability and business continuity.

"As companies move quickly from the pilot phase to the actual deployment phase in the data center, the Enterprise Architect starts to come to the front desk and plays an important role in the big Data deployment," Schroeder said. "It leaders also play a critical role in choosing the infrastructure that needs to meet SLAs, with high reliability and business continuity, while meeting critical business needs." In the 2014, the Hadoop ecosystem flourished and generated a large number of applications, tools, and components. The market is expected to focus on cross-platform differences in 2015, while the enterprise infrastructure will integrate Hadoop into the data center and generate some business value.

(Source: China index net)

Five trends to dominate big data technology in the 2015