Use GeoIP2 to analyze visitors' IP addresses and obtain geographical location information

Use GeoIP2 to analyze visitors' IP addresses and obtain geographical location information

MaxMind GeoIP2 is widely used to identify locations and other features of Internet users, this includes customized content, fraud detection, advertising targeting, website traffic analysis, execution rules, geographic target positioning, geo-fencing, and digital copyright management. Currently, GeoIP is used with the Nginx or Apache server for log analysis to obtain the regional distribution of Website access traffic.

GeoIP is divided into commercial and free versions. The free version is much less accurate than the commercial version. It is tested that there is indeed a gap between city positioning and whether you can accept your precision requirements!

I. Introduction to the free version:

1. The GeoLite version is widely used on the Internet. The database type is dat files, and the accuracy test is not carried out if the database files are small.

2. GeoLite2, the latest version. The database file is in mmdb format. If you are interested in mmdb format, click here.

Compare the two database files. Click here for the GeoLite2 feature

$ du -sh *32MGeoLite2-City.mmdb2.3MGeoLite2-Country.mmdb18MGeoLiteCity.dat732KGeoLiteCountry.dat

The City file contains the City information database, and the Country file is the national information database.

2. Download The GeoLite2 Database
There are two download methods: the first is to download the gz compressed package, and the second is to download the update program provided by the official website. The second is recommended. The official website says that the database is updated on the first Tuesday of each month, if you want to update a scheduled task every month, we recommend that you select the second type! For detailed updates to GeoIP2, click here.

Both methods are mentioned here. In this stage, we only talk about how to download the database. For more information about the calling method, see the API call section of stage 3!

1. First, download and decompress the gz file.

GeoLite2 only provides the City database and Country database for download. Click here to view details. The database files are classified into Binary and CVS. Binary files are used here.

$ sudo mkdir -p /mnt/data/geolite2 && cd $_$ sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz$ sudo gzip -d GeoLite2-City.mmdb.gz$ sudo wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz$ sudo gzip -d GeoLite2-Country.mmdb.gz

2. The second method requires installation of the official download update program geoipupdate.

A. Download geoipupdate from GitHub. The latest version is 2.1.0. You must be patient with the GitHub connection speed. You can download the latest version! The libcurl-devel package is required for file compilation. You must download and install the package in advance.

$ sudo yum install libcurl-devel -y$ sudo wget https://github.com/maxmind/geoipupdate/releases/download/v2.1.0/geoipupdate-2.1.0.tar.gz$ sudo tar xzvf geoipupdate-2.1.0.tar.gz$ cd geoipupdate-2.1.0$ sudo ./configure$ sudo make$ sudo make install

After compilation, you only need to pay attention to two files.

Update the execution file/usr/local/bin/geoipupdate

Account Information File/usr/local/etc/GeoIP. conf

B. Configure the account information GeoIP. conf. modify the configuration file as follows. This configuration file downloads the mmdb file by default. If you want to download the dat file, cancel the annotation!

# The following UserId and LicenseKey are required placeholders: UserId 999999 LicenseKey 000000000000 # Include one or more of the following ProductIds: # * GeoLite2-City-GeoLite 2 City # * GeoLite2-Country-GeoLite2 Country # * 506-GeoLite Legacy Country # * 517-GeoLite Legacy ASN # * 533-GeoLite Legacy City # dat format database # ProductIds geoLite2-City GeoLite2-Country 506 # mmdb format database ProductIds GeoLite2-City GeoLite2-Country 533 132

C. Execute updates

View the geoipupdate help file to find out which parameters can be used! The-d parameter downloads the file to the specified directory, and the-v parameter displays the download process details.

$ /usr/local/bin/geoipupdate -hUsage: geoipupdate [-Vhv] [-f license_file] [-d custom directory]  -d DIR   store downloaded files in DIR  -f FILE  use configuration found in FILE (see GeoIP.conf(5) man page)  -h       display this help text  -v       use verbose output  -V       display the version and exit

Run the update command to check the download speed and network conditions. This document downloads the file to the/mnt/data/geolite2/directory.

$ Ll/mnt/data/geolite2/& cd $ _ total usage 0 $ sudo/usr/local/bin/geoipupdate-d/mnt/data/geolite2/-v $ ll total usage 34088-rw-r -- r --. 1 root 32553611 December 19 18:14 GeoLite2-City.mmdb-rw-r -- r --. 1 root 2349406 December 19 18:14 GeoLite2-Country.mmdb

How to configure scheduled tasks to regularly update the GeoLite2 database.

3. Install the GeoLite2 API caller

NET (C #), C, Java, Perl, Python, and Apache API calls are officially provided. Other third-party interfaces are available, but official technical support is not provided. Click here for details.

This article uses the C language API to call and test. For other languages, please refer to the official instructions for your own solutions! C language API GitHub

$ sudo wget https://github.com/maxmind/libmaxminddb/releases/download/1.0.3/libmaxminddb-1.0.3.tar.gz$ sudo tar xzvf libmaxminddb-1.0.3.tar.gz$ cd libmaxminddb-1.0.3$ sudo ./configure$ sudo make$ sudo make install

View help documentation

$ /usr/local/bin/mmdblookup --help  mmdblookup --file /path/to/file.mmdb --ip 1.2.3.4 [path to lookup]  This application accepts the following options:      --file (-f)     The path to the MMDB file. Required.      --ip (-i)       The IP address to look up. Required.      --verbose (-v)  Turns on verbose output. Specifically, this causes this                      application to output the database metadata.      --version       Print the program's version number and exit.      --help (-h -?)  Show usage information.  If an IP's data entry resolves to a map or array, you can provide  a lookup path to only show part of that data.  For example, given a JSON structure like this:    {        "names": {             "en": "Germany",             "de": "Deutschland"        },        "cities": [ "Berlin", "Frankfurt" ]    }  You could look up just the English name by calling mmdblookup with a lookup path of:    mmdblookup --file ... --ip ... names en  Or you could look up the second city in the list with:    mmdblookup --file ... --ip ... cities 1  Array numbering begins with zero (0).  If you do not provide a path to lookup, all of the information for a given IP  will be shown.

Iv. Test

Test IP Address: 112.225.35.70 Shandong Qingdao Unicom

1. Get the country information. The country information is correct and displays Russian in Garbled text!

$/Usr/local/bin/mmdblookup -- file/mnt/data/geolite2/GeoLite2-Country.mmdb -- ip 112.225.35.70 {"continent": {"code ": "AS" <utf8_string> "geoname_id": 6255147 <uint32> "names": {"de": "Asien" <utf8_string> "en ": "Asia" <utf8_string> "es": "Asia" <utf8_string> "fr": "Asie" <utf8_string> "ja ": "commandid bandwidth" <utf8_string> "pt-BR": "ásia" <utf8_string> "ru": "commandid bandwidth" <utf8_string> "zh-CN ": "Asia" <utf8_string >}" country ": {" geoname_id ": 1814991 <uint32>" iso_code ":" CN "<utf8_string>" names ": {" de ": "China" <utf8_string> "en": "China" <utf8_string> "es": "China" <utf8_string> "fr": "Chine" <utf8_string> "ja ": "China" <utf8_string> "pt-BR": "China" <utf8_string> "ru": "Your region has been changed" <utf8_string> "zh-CN ": "China" <utf8_string >}} "registered_country": {"geoname_id": 1814991 <uint32> "iso_code": "CN" <utf8_string> "names": {"de ": "China" <utf8_string> "en": "China" <utf8_string> "es": "China" <utf8_string> "fr": "Chine" <utf8_string> "ja ": "China" <utf8_string> "pt-BR": "China" <utf8_string> "ru": "Your region has been changed" <utf8_string> "zh-CN ": "China" <utf8_string> }}}

2. The data obtained from the city information is a bit tangled. The Province has no problems and the city has problems! The official demo address is very accurate. Maybe this is the difference between free and paid :)

$/Usr/local/bin/mmdblookup -- file/mnt/data/geolite2/GeoLite2-City.mmdb -- ip 112.225.35.70 {"city": {"geoname_id": 1805753 <uint32> "names ": {"de": "Jinan" <utf8_string> "en": "Jinan" <utf8_string> "es": "Jinan" <utf8_string> "fr ": "Jinan" <utf8_string> "ja": "Tainan City" <utf8_string> "pt-BR": "Jinan" <utf8_string> "ru ": "Ц з и н" <utf8_string> "zh-CN": "Jinan" <utf8_string >}} "continent": {"code ": "AS" <utf8_string> "geoname_id": 6255147 <uint32> "names": {"de": "Asien" <utf8_string> "en ": "Asia" <utf8_string> "es": "Asia" <utf8_string> "fr": "Asie" <utf8_string> "ja ": "commandid bandwidth" <utf8_string> "pt-BR": "ásia" <utf8_string> "ru": "commandid bandwidth" <utf8_string> "zh-CN ": "Asia" <utf8_string >}" country ": {" geoname_id ": 1814991 <uint32>" iso_code ":" CN "<utf8_string>" names ": {" de ": "China" <utf8_string> "en": "China" <utf8_string> "es": "China" <utf8_string> "fr": "Chine" <utf8_string> "ja ": "China" <utf8_string> "pt-BR": "China" <utf8_string> "ru": "Your region has been changed" <utf8_string> "zh-CN ": "China" <utf8_string >}" location ": {" latitude ": 36.668300 <double>" longpolling ": 116.997200 <double>" time_zone ": "Asia/Shanghai" <utf8_string>} "registered_country": {"geoname_id": 1814991 <uint32> "iso_code": "CN" <utf8_string> "names ": {"de": "China" <utf8_string> "en": "China" <utf8_string> "es": "China" <utf8_string> "fr ": "Chine" <utf8_string> "ja": "China" <utf8_string> "pt-BR": "China" <utf8_string> "ru ": "commandid was too large" <utf8_string> "zh-CN": "China" <utf8_string >}} "subdivisions": [{"geoname_id ": 1796328 <uint32> "iso_code": "37" <utf8_string> "names": {"en": "Shandong Sheng" <utf8_string> "zh-CN ": "Shandong Province" <utf8_string >}}]}

Because the obtained data is in Json format, you can format and output the content as prompted in the help document, for example, output the city-> names-> zh-CN content in the city database.

$/Usr/local/bin/mmdblookup -- file/mnt/data/geolite2/GeoLite2-City.mmdb -- ip 112.225.35.70 city names zh-CN "Jinan" <utf8_string>

Although GeoIP2 is not ideal for city positioning analysis, it is acceptable for me!