Event Log Database

After more than eight years of practice, from Taobao's collection business to today to support all of Alipay's core business, and in the annual "Double Eleven Singles Day" continue to create a world record for the transaction database peak processing capacity.

Hadoop technology in telecom operators online log processing application architecture Fang Jianguo First, the telecom operators Internet log processing status Today, so popular in the mobile Internet, every day will produce a lot of Internet log, these Internet log due to the huge amount of data generated Can only be retained for 3 days, because of storage space and other reasons are discarded. At present, telecom operators may only have a lot of useful information on customer behavior missing from the analysis of customer behavior mainly based on CDRs (call logs). For example, two people with similar phone conversations may be completely different types of customers ...

Hello, I'm brother Tao. The event has many webmaster from the strategic level to share the idea of the site operation, just found a lot of friends once said the idea is difficult to explain the example, so I took a last year's example, to share with you how we find the problem from the log analysis, solve the problem to the final summary of lessons and optimize the operation of the site process,   At the same time I will detail the details of the way to popularize the log analysis, I hope to help friends. Website operation has a link important, that is data monitoring and data analysis, otherwise the problem does not know ...

The scalability of the system is the main reason for promoting the development of NoSQL movement, including distributed system coordination, failover, resource management and many other features. That makes NoSQL sound like a big basket that can be stuffed with anything. Although the NoSQL movement does not bring fundamental technological changes to distributed data processing, it still leads to extensive research and practice on protocols and algorithms. It is through these attempts to gradually summarize some effective database construction methods. In this article, I will focus on the distributed features of the NoSQL database ...

The scalability of the system is the main reason for promoting the development of NoSQL movement, including distributed system coordination, failover, resource management and many other features. That makes NoSQL sound like a big basket that can be stuffed with anything. Although the NoSQL movement does not bring fundamental technological changes to distributed data processing, it still leads to extensive research and practice on protocols and algorithms. It is through these attempts to gradually summarize some effective database construction methods. In this article, I will focus on the NoSQL database distributed special ...

We will combine case studies to see how to use data analysis to actually improve the game. This time we have an example of a mobile end of the card games, simulation board game 21 points. The popular version of the game is free of charge, and users can obtain a version without advertising and additional features after paying a certain fee. The problem is that this 21-point game does not generate the expected revenue. The expectation is to solve this problem, increase user participation and consumption. If the designer of the game does not have the concept of data analysis, and does not collect enough data for analysis, it can only pat the head to make policy changes. And if the idea of data analysis in the design ...

Loganalyzer is a Web front-end for syslog logs and other network event data. It provides a simple view of the log, search, basic analysis, and some chart reporting capabilities. Data can be obtained from a database or a generic syslog text file, so Loganalyzer does not need to change the existing record schema. Based on the current log data, it can handle syslog log messages, Windows event log records, support troubleshooting, and enable users to quickly find the solution to the problem in log data. Lo ...

Loganalyzer is a Web front-end for syslog logs and other network event data. It provides a simple view of the log, search, basic analysis, and some chart reporting capabilities. Data can be obtained from a database or a generic syslog text file, so Loganalyzer does not need to change the existing record schema. Based on the current log data, it can handle syslog log messages, Windows event log records, support troubleshooting, and enable users to quickly find the solution to the problem in log data. Lo ...

Loganalyzer is a Web front-end for syslog logs and other network event data. It provides a simple view of the log, search, basic analysis, and some chart reporting capabilities. Data can be obtained from a database or a generic syslog text file, so Loganalyzer does not need to change the existing record schema. Based on the current log data, it can handle syslog log messages, Windows event log records, support troubleshooting, and enable users to quickly find the solution to the problem in log data. Lo ...

Editor's note: Jay Kreps, a chief engineer from LinkedIn, says that logs exist almost at the time of the computer's creation, and there is a wide range of uses in addition to distributed computing or abstract distributed computing models. In this paper, he describes the principles of the log and the use of the log as a separate service to achieve data integration, real-time data processing and distributed system design. Article content is very dry, worth learning. Here's the original: I joined the LinkedIn company at an exciting time six years ago. From that time ...