Vi. analysis and design of real-time statistics of user log escalation in Kafka

I. Overview of the project as a whole

• Outline the background of the project

Background:
User whereabouts
Enterprise operations

• Purpose of the Analysis project

Through the analysis of the project, we can get the following objectives: • Real-time user dynamics • Based on real-time statistical results, moderate promotion and statistical analysis results, rapid and reasonable adjustment of two, Producer module analysis

• Analyze production data sources

In the user escalation log, each log record represents a user's active state, and the sample data is as follows: 121.40.174.237 yx12345 [21/july/2015 13:25:45 +0000] Chrome appid_5 "http://www.*** . cn/sort/channel/2085.html "The amount of information that the sample data contains, as shown in:

• Flume to Kafka Process analysis

Data source production flow analysis, as shown in:

Three, Consumer module analysis

• Analysis of consumer data sources

KPI metrics for consumption data source statistics, as shown in:

• Kafka to Storm Process analysis

Data source consumption process analysis, as shown in:

Iv. overall design of the Project 1. Detailed project design process

The project detailed design process, as shown in:

2.KPI indicator Design KPI indicator design, as shown in: This section of our project for the overall analysis, and to guide you to analyze Kakfa Producer module and Consumer module, as well as to help you to design the project development process knowledge, should grasp the following knowledge: • Project background and purpose · Analysis of Producer modules and Consumer modules • Overall development process of the project KPI indicator design

Vi. analysis and design of real-time statistics of user log escalation in Kafka