Cloud computing Architecture technology and practice 20:2.4.5 Big data Analytics Cloud

2.4.5Big Data Analytics Cloud

Cloud solutions for Big data analytics based on the overall architecture of cloud computing, as shown in2-33 .

Figure 2 - - Big Data Analytics Cloud Solution Architecture Subsystem Portfolio

The Big Data Analytics cloud solution provides support for enterprise and industry scenarios where massive amounts of static data batch processing and high-traffic dynamic flow data processing are key features, enabling business appreciation through automated extraction and induction of value information. The Big data analytics cloud is supported by a parallel data analytics and mining platform for cloud computing that maximizes the value of cloud-based underlying capabilities.

In the large-scale static data batch processing scenario, the big data analysis platform needs to fully analyze the historical data which has accumulated over a long period of time and has huge storage capacity (such as the word list, log, and so on). The parallel data processing engine of the big analytics platform relies further on elastic compute clusters, resilient storage services, distributed structured storage services, and distributed Message Queuing services for Internet e-commerce website users, Bss/oss systems for telecom operators, video entertainment sites, The Search class site provides services. The types of services provided by the Big Data analysis platform include: Information base refinement Search, user behavior log analysis, System operation log analysis and centralized monitoring signaling information intelligent analysis and mining. These big data analytics services provide decision-making support for precision-targeted ad push, network operations optimization, and sales strategy optimization based on consumer trend analysis.

In the scenario of high traffic dynamic Data stream processing, its key feature is the dynamic event and dynamic data (such as the signaling information from the real-time detection of the telecommunication network, the positioning information from the GPS of many vehicles, the real-time information collected from the IoT terminal), In a relatively short time window, the Dynamic Data pipeline automatic correlation analysis and processing, and the timely, accurate and intelligent execution strategy decision-making, for specific business objectives (such as large-scale intelligent traffic cloud, logistics Cloud network construction). Compared with the large number of persistent storage I/O interactions involved in the segmentation, merging, and blending of the previous data batches, the most significant difference is that the data flow processing process is more time-sensitive and agile-controlled, so the process is mostly done in memory. Stream processing and batching can be unified under the same framework engine.

To facilitate the development of a wide range of third-party application programmers and cloud computing platform Ecosystem Partners fully independent Yu Hai data batching and the internal implementation of the stream processing business architecture details, you can set the sql/ class SQL between the parallel data analysis engine and the concurrent application The adaptation and translation layer provides a vast amount of data manipulation in SQL or Class SQL specification languages that are well known to developers.

Cloud computing Architecture technology and practice 20:2.4.5 Big data Analytics Cloud