Application System Log collection solution and log collection solution

Application System Log collection solution and log collection solution
Overview

Use Flume + MongoDB to collect logs from multiple existing application systems. Feature data flow diagram Note: The first architecture is to directly write log data to Flume in the application system. However, existing application systems are non-Maven, you need to add more than 20 jar packages to each application system. In order to avoid this situation, a layer of Log service is provided to open the webservice Service for application system calls, and the above system is formed. Log storage 1. problem to be solved 1.1 Use Flume to write logs to MongoDB reference: Flume Learning application: Java to write log data to MongoDB-Internet Reference: Flume Learning application: Java to write log data to MongoDB1.2 release webservice reference: release jaxws-Internet reference in a web project: Release jaxws2. Log service in a web project to implement a simple web project. Release a webservice to write logs to Flume. 2.1 file structure

Src/main/java | ---- cn. sinobest. asj. log | ---- ISALog. java # Log Service Interface | ---- SALogImpl. java # Log service implementation class | ---- cn. sinobest. asj. log. exception | ---- InvalidGradeException. java # indicates invalid Log Level | ---- InvalidFormatExceptioin. java # indicates Invalid Message format (JSON string is required) | ---- cn. sinobest. asj. log. util | ---- ValidGrade. java # enumeration, all valid log levels (DEBUG, INFO, WARN, ERROR) | ---- MessageTemplate. java # message template src/main/resources | ---- log4j. propertiessrc/main/webapp | ---- WEB-INF | ---- sun-jaxws.xml | ---- web. xml | ---- index. jsp # This can ignore pom. xml

 

2.2 you can directly obtain the source code from log-service and skip the content in this section. Application System Group 1. question to consider: 1.1 interception can be achieved by using Filter. 1.2 The organization log content depends on the requirements. Currently, only some information in the request is collected. 1.3 formatting log information must be formatted as a JSON string to be correctly written to MongoDB. 1.4 request for the webservice Service reference: clients that generate code based on wsimport-Internet Reference: clients that generate code based on wsimport 2. demo2.1 file structure

Src | ---- cn. sinobest. asj. log | ---- LogFilter. java | ---- cn. sinobest. asj. log. wsimport # store wsimport generated code # omitted basic | ---- WEB-INF | ---- web. xml

2.2 file content test Appendix Related Articles