Installation and full usage of goaccess under Linux

Goaccess user documentation One, goaccess overview 1.1 goaccess Overview

Goaccess is a real-time Web Log Parser, as well as an interactive viewer that runs on Unix-like systems (terminal) and is a GPL-based free software. Provides fast and important HTTP statistics for system administrators who require visual server reporting. First it parses the Web log file, collects the data from the parsed file, and then displays it on the console or X terminal. The information collected is presented to the user in a Visual/interactive window, including:

General statistics

The Panel provides a summary of several indicators, some of which are valid and invalid requests, the amount of time spent analyzing datasets, unique visitors, requested files, static files (css,ico,jpg, etc.) HTTP references, 404, parsing the size of the log file and bandwidth consumption.

Uniquely identifiable visitors

This panel displays metrics such as the cumulative bandwidth of matching, unique visitors and daily dates. HTTP requests that contain the same IP, the same date, and the same user agent are considered unique visitors. By default, it includes web crawlers/spiders. Optionally, you can use-date-spec= hr to set the time attribute to the hour level, which displays dates such as 05/jun/2016:16. This is great if you want to keep track of your daily traffic at the hour level.

The requested file

This panel displays the highest requested file on the Web server. It shows hits, unique visitors and percentages, as well as cumulative bandwidth, protocol, and request methods used.

The requested static file

Lists the most commonly used static files, such as: Jpg,css,swf,js,gif, and PNG file types, using the same indicator as the last panel together. Additional static files can be added to the configuration file.

404 Or Not Found

Displays the same metrics as the previous request panel, but its data contains all pages not found on the server, or is commonly referred to as a 404 status code.

Host

This panel details information about the host itself. It is great to find active crawlers and identify who is eating your bandwidth.

The expansion panel can display more information, such as the host's reverse DNS query results, country of origin and city. If-a enables the parameter, you can display the list of user agents by selecting the desired IP address and then pressing the ENTER key.

Operating system

The Panel reports the operating system that the host is using on the server. It tries to provide the most specific version of each operating system.

Browser

This Panel reports the browsers that the host uses on the server. It tries to provide the most specific version of each browser.

Number of visits

This panel displays the hourly report. This option displays 24 data points, one per hour per day.

Optionally, the hour specificity can be set to one-tenth seconds using-hour-spec= min, which will show an hour of 16:4 if you want to find traffic spikes on the server, this is very good.

Virtual Host

This panel will display all the different virtual hosts resolved from the Access log. This panel is displayed if you use%v in the log format string.

Referral Source URL

If the host that has the problem has access to the site through another resource, or if it is accessed from another host, the URL of the reference is provided in this panel. See the '--ignore-panel ' configuration file to enable it. (Disabled by default)

Referral Source Site

This panel displays only the hosts section, but does not display the entire URL. The URL from which the request came.

Key phrases

It reports Google search, Google cache and Google Translate using key phrases that lead to your Web server. Currently, it only supports Google search queries over HTTP. See the '--ignore-panel ' configuration file to enable it. (Disabled by default)

Location

Determine where the location is. Statistical data are disaggregated by continent and country. It needs to be compiled with geolocation support.

HTTP status Code

The value of the numeric status code for the HTTP request.

Remote user (HTTP authentication)

This is the user ID of the person who identified the request document through HTTP authentication. If the document is not password protected, this section will be "-", just like the previous one. This panel is not enabled unless%e is given in the log format variable.

Second, installation goaccess

Refer to official installation steps: Https://goaccess.io/download

2.1 Dependent Installation

Goaccess relies on the following components when installing with source code.

2.1.1, to display the IP location for convenient final log statistics, you need to install the dependency geoip-devel:

Execute command:yum installgeoip-devel.x86_64

2.1.2 Installing the Ncurses-devel Development Library:

Execute command:yum Install ncurses-devel

2.1.3 Installing the Tokyocabinet-devel Development Library:

Execute command:yum Install tokyocabinet-devel

2.1.4 Installing the Openssl-devel Development Library:

Execute command:yum Install openssl-devel

2.2 Download and install Goaccess:

Switch to the/USR/LOCAL/SRC directory by using the CD command, and then execute the command with root privileges:

wget http://tar.goaccess.io/goaccess-1.2.tar.gz

After downloading, perform the decompression command:

Tar-zvxf./goaccess-1.2.tar.gz

Then execute cd./goaccess-1.2 to enter the directory.

Execute command:./configure--enable-utf8--enable-geoip=legacy

Execute command: make

Execute command: Makeinstall

After installation, you can execute the command:ln/usr/local/bin/goaccess/usr/bin/

After the installation is complete, the relevant directory:

Executable Program:/usr/local/bin/goaccess

Configuration file:/usr/local/etc/goaccess.conf

Other documents:/usr/local/share/doc/goaccess

Brochure:/usr/local/share/man/man1

Iii. Self-Discussion Log/date format 3.1 How to use

The simplest and quickest use of 3.1.1:

# goaccess Access.log

3.1.2 Generates an HTML report:

# goaccess access.log-a-O report.html

3.1.3 generates a JSON report:

# goaccess Access.log-a-d-o Report.json

3.1.4 generates a CSV report:

# goaccess Access.log--no-csv-summary-o report.csv

3.1.5 If we want to add more flexibility to goaccess, we can use a series of pipelines. For example:

Another useful pipeline is to filter web logs by date.

The following command will get all HTTP requests for July 31, 2017 in the log file:

# sed-n '/31\/jul\/2017/,$ p ' access.log | Goaccess-a

If you want to resolve only a specific time-bound log from date A to date B, you can:

# sed-n '/30\/jul\/2017,/31\/jul\/2017/p ' Access.log | Goaccess-a

Note that this command relies on the speed of the SED and may require longer parsing times.

It is also worth noting that if you want to run goaccess with a lower process scheduling priority, you can do this:

# nice-n Goaccess-f access.log-a

3.1.6 real-time HTML output

# goaccess Access.log-o/usr/share/nginx/html/site/report.html--real-time-html

Or, for httpd:

# goaccess Access.log-o/usr/www/html/report.html--real-time-html

By default, the Goaccess listening port is 7890, and you can modify the port through-port=xxx.

# goaccess Access.log-o report.html--real-time-html--port=xxx

3.2 Interactive Menu

• F1 main help Page
• F5 Redraw the main window
• Q Exit program or current {detail view} (window)
• Open a detailed view of the current activation module
• C Set or change the color scheme
• TAB forward Iteration module. Start with the current activation module.
• SHIFT + TAB to iterate the module backwards. Start with the current activation module.
• Right ARROW opens a detailed view of the current activation module
• 0-9 Activate the module so that users can open {verbose view} using ^o^ or ^right arrow^
• SHIFT + 0-9 activates more than 10 modules
• s separate visitor sort by date. Valid only in the standalone guest module (1).
• S individual visitor sort based on number of clicks. Valid only in the standalone guest module (1).
• /forward in any {verbose view} window to search for Input mode (pattern).
• N Find the next occurrence in any {detail view} window.
• T jump to the first entry or the top of the screen
• B Jump to the last entry or the bottom of the screen

3.3 Date format (date_format)

A date_format variable after a space specifies any combination of regular characters and special format specifiers. These characters start with a percent sign (%). See: Man strftime

• %a abbreviated form of the week
• %A full name of the week
• Abbreviated form of%b month
• Full name of the%B month
• %c Date and time
• Date in%d months, 0-31
• %H hours, 00-23
• %I 12 hours of input, 01-12
• %j dates in year, 001-366
• %m Month in year, 01-12
• %M min, 00-59
• %p Morning or afternoon
• %s seconds, 00-60
• %u weeks, 1-7
• %w Week, 0-6
• Date in local format%x
• Time in local format%x
• The last two digits of the%y year, 00-99
• %Y year
• %Z Geographic Time zone name

3.4 Log Format (Log_format)

Log Format (Log_format)

The Date_format variable after the space specifies the log format string.

• %x date and Time fields that match the time format and date format variables. Use this when using a timestamp instead of a date and time in two separate variables.
• The%t time field matches the Time_format variable.
• %d date field that matches the Date_format variable.
• %d The time, in microseconds, that the service request took.
• %e This is the user ID of the person who determined the request document through HTTP authentication.
• %H Host (client IP address, IPv4 or IPV6).
• %r the request line from the client. This needs to be resolved around the requested specific delimiter (single quotes, double quotes, and so on). Otherwise, use a combination of special format specifiers, such as%m,%u,%q and%h, to resolve individual fields.
• %m Request method.
• The URL path of the%u request.
• %q the query string.
• %h request agreement.
• The%s server returns a status code to the client.
• %b the size of the object returned to the client.
• %R refererhttp request header.
• %u User Agent HTTP request header.
• %^ ignores the domain.
• The time, in seconds, that is required to%T the service request, in milliseconds.
• The time, in milliseconds, that is required for the%L service request to be a decimal number.
• %~ moves the log string forward until it finds a non-whitespace (! Isspace) characters
• The host (client IP address, IPv4, or IPV6) in the ~h x-forwarded-for (XFF) field.

Installation and full usage of goaccess under Linux