Real-time query processing based on mass data

Real-time query processing based on mass data

Wuhan University of Caofang

Based on the S4 system, this paper discusses the development of Java Library by introducing the query feature and the mechanism of processing data node, and increases the usability of S4 system, which is called s4_s in the thesis. S4_s replaces the original system's code extension mechanism, the user can create a new streaming application by writing a continuous declaration statement. At the end of the paper, we will prove the ability to improve the system processing data, and use the Network Service monitor and forecast Sina Weibo topic two applications to prove how to use this system to create new complex flow applications quickly and easily. Finally, an experimental example is used to analyze the scalability, resource usage and fault tolerance of the S4S system, and the experimental results prove that the scalability of the system satisfies the requirements, and the fault tolerance is not in accordance with the expectation. Combined with the results of the experiment and the comparison of the S4 system, the next research direction of the paper is how to make the S4S system simpler and more stable, and to respond intelligently to the changing environment of the system.

Real-time query processing based on mass data