Clickstream log data mining overview
Many websites (such as e-commerce) will collect the user's browsing behavior, and then analyze it to achieve functions such as advertising recommendations, which can improve website design and increase corporate profits.
data collection
Embed some js code to add binding events for some tags on the page. When the event occurs, you can use the ajax request to the background. These event information will be recorded in the web server's log, so as to obtain a growing log file.
Data processing flow
Generally, offline analysis is this process.
Overview of each process:
Data collection: Customize the development of the collection program, or use the open source framework FLUME
Data preprocessing: custom-developed mapreduce program runs on the hadoop cluster
Data warehouse technology:
Hive based on
hadoop Data export: sqoop data import and export tool based on hadoop
Data visualization: customized development of web programs or use of products such as kettle
Process scheduling of the entire process: oozie tools or other similar open source products in the hadoop ecosystem
After the data collection is completed, because the amount of data is very large, the subsequent operations are processed using distributed programs.
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.
