Recognize "fast data"

The development of Internet business and technology ushered in the new climax of the information revolution, which brought about explosive growth of data in addition to more efficient production and consumption patterns. The scale and speed of data in the wave of mobile internet and IoT are accelerating faster than ever before, and the technologies and tools for data storage, processing, analysis and presentation are emerging. Traditional enterprises already have a more mature production and acquisition of existing system processes and data, but how to the once neglected and discarded data, the use of new technologies such as distributed, memory-based, and integration of existing enterprise data and analysis results to achieve business innovation and enhance, is an urgent problem to solve.

In the initial stage of big data, the value tends to be sparse, and enterprises often need "haystack". In today's era, the cost of storing massive amounts of data has been reduced, but it is expensive to get value from massive amounts of data, and it is more expensive to get the value in time. Therefore, more and more enterprises choose to build real-time computing framework of big data, in order to get real-time data insight, the concept of "fast data" came into being.

What is fast data?

The concept of big data itself is rather abstract, and a more representative 4V definition is that big data needs to meet 4 features: scale (Volume), Diversity (Variety), high speed (velocity), and value. Fast data is created to achieve high speed (velocity). "Fast" comes from a number of well-known laws: Time is money, the value of data is also time-sensitive, and the value of the data is depreciated more quickly. The following table gives a comparison of the speed of different business processing data.

From the perspective of data analysis technology, the current big data processing can be divided into the following three types:

• Complex bulk data processing (batch processing), a common implementation framework such as hadoop/mapreduce, data processing time spans between 10 minutes and several hours.
• Enhanced historical Data Interactive query (inter-active query), a common implementation framework such as Dremel/impala, data processing time spans between 10 seconds to several minutes.
• Data processing (event streaming processing) based on real-time incident data streams, common implementation frameworks such as Oracle CEP, Strom. The data processing time spans between hundreds of milliseconds and a few seconds.

The above three ways, the most consistent with the fast data definition is the third. Data processing based on real-time event traffic is not only able to provide faster data processing efficiency, but rather a completely different mode than offline batch processing. These two processing modes, batch processing, are stored post-processing (store-then-process), while stream processing is directly processed (Straight-through processing). These two modes of processing complement each other and are important when enterprises build large data processing frameworks. Some mature enterprises divide the data processing service into several layers when they are processing it. On the one hand is the importance degree, on the one hand is the processing time requirement, for example "fast data urgent", "fast data not urgent" and "slow data important" and so on. Fast data essentially means the ability of data processing to close real-time decision-making, the time spent in improving business decision-making, and the flow-processing mode brings more imagination and innovation space to system business innovation.

What are the application scenarios for fast data?

The way the data is produced in human society has gone through 3 stages, and it is the great change in the way that data is produced that ultimately leads to big data generation:

1, the operating system stage. Most of the data is generated by operating systems, and data are mostly operational data. This data is produced in a passive nature.

2, the user creates the content stage. The development of Internet, especially the development of e-commerce and Web2.0 brings about a new stage of data explosion. The development of Internet e-commerce produces a lot of user behavior data, which is completely different from the data generated by passive operation system. The most important sign of Web2.0 is user-generated content (UGC, user Generated contents). This phase of data generation is characterized by initiative.

3, perceptual system stage. Today we are at the beginning of this phase. The core reason for this phase is the rise of mobile internet and IoT. With the development of technology, smart phones, wearable devices and small sensors with processing capabilities are maturing, all kinds of original "dead" devices, can now automatically generate, collect data, even we humans, but also because of the carrying of smart devices, every change in location and equipment use will produce a large number of data can be used for analysis. The characteristics of this data are automatic.

Simply put, data generation is a passive, active and automatic three stages. These passive, active, and automated data together form the source of data for big data.

Fast data can help corporate customers in four broad areas:

• Help businesses improve their customer experience. Traditional passive systems discard many data unrelated to business storage and statistical analysis. And fast data can organically and traditional data and Hadoop type Big Data organic combination, help enterprises to better build a full view of user views, the development of user-driven products, providing customer-oriented services. Fast data makes up the point at which a full-view user view is built with a single database and Hadoop platform.

• Help businesses optimise their operations. Traditional enterprise Big data analytics IT operations often lack the ability to seize online marketing opportunities. Marketing in the form of the best understanding of the customer in the right time when the customer's behavior event occurs. At the same time, in social media to track information about locations, users and products, to analyze the links between products, users, and brands to optimize the accuracy of their in-house product and service offerings, and to conduct targeted, online and offline (o2o[note]) product recommendations are the most important innovation in operational capabilities of traditional enterprises.

• Help enterprises optimize resources. Through the intelligent Device data acquisition technology, the enterprise can realize the accurate optimization of the required resources and optimize the efficiency of resource use. In the enterprise in the operation process, the user product needs each kind of resources the specific use situation and the distribution and so on, the enterprise can carry on the collection analysis, just like "the electronic cockpit" general, realizes "the point to the point" the data, the image display. Fast data enables the management of enterprise managers to optimize the way of enterprise resources from "t+1" to "t+0", can be more intuitive and efficient management of their own enterprises.

• Help businesses expand their services. Fast data enables businesses to extend their time and place of service to every point in the customer's life cycle. Enterprises can also take advantage of the large amount of data exposed in social media, through the fast data trend analysis of public opinion listening technology, analysis of the correlation between the content of data, and then to the social users to carry out refinement services.

How to tell if the enterprise needs "fast data"?

As big data has become a hot topic in the IT industry (+ focus on the Web), all companies want to deploy their big data strategy as early as possible, and also want to get real-time data analytics capabilities through "fast data". However, how do you tell if a business really needs "fast data" or just a narrow, big data solution? Here Oracle proposed FAST Data Project three ask help enterprises make the right decision:

• How does the results of data analysis react to the application system?

• Data is continuous and sequential, window, timing and other elements, is the source of a large number of different formats, data volume is big but do not care about storage?

• Whether to consider three first: Can the first time, the customer's first contact point, make the first reaction?

If the enterprise can correctly understand the above problems and according to their actual situation to give a positive answer, then "fast data" is a good choice for enterprises. Common fast data processing scenarios include event-driven marketing and recommendations, location-based services, interactive marketing, customer retention, risk forecasting and evaluation, performance management, media listening and response, intelligent device acquisition and analysis, financial quantification transactions and risk management.

"For more information on business intelligence, business intelligence solutions and business intelligence software downloads, visit Finebi Business Intelligence official website www.finebi.com"

Recognize "fast data"