Nginx Google Agent module (ngx_http_google_filter_module)

Two sentences.

Wen.lu all the way to the present, inseparable from the support of everyone!
Many friends have asked me through various ways: "You do not open source ah ..."
First of all to apologize to those friends ah, in fact, I do not want to open source, but the previous version of the configuration is too complex. Nginx three-party expansion with a lot of, plus Lua, and the breakthrough of thousands of lines of configuration engineering, such a clumsy feat, really embarrassed to share
Then decided to write an extension, so that Google's wen.lu configuration and the use of the same as simple.

Location/{
Google on;
}

You did not read the wrong, "one-line configuration, Google I have!" ”
Now G2.wen.lu is driven by this extension
Lua-nginx
Dependent libraries
Pcre Regular
Ngx_http_proxy_module Reverse Proxy
Ngx_http_substitutions_filter_module multiple Replacements

Installation

Take Ubuntu 14.04 as an example i386, x86_64 all apply

#
# Install GCC & git
#
Apt-get install build-essential git gcc g++ make

#
# Download the latest version of the source code
# Nginx Official website:
# http://nginx.org/en/download.html
#
wget "Http://nginx.org/download/nginx-1.7.8.tar.gz"

#
# Download the latest version of Pcre
# PCRE Official website:
# http://www.pcre.org/
#
wget "Ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz"

#
# Download the latest version of OpenSSL
# OPESSL Official website:
# https://www.openssl.org/
#
wget "Https://www.openssl.org/source/openssl-1.0.1j.tar.gz"

#
# Download the latest version of Zlib
# zlib Official website:
# http://www.zlib.net/
#
wget "Http://zlib.net/zlib-1.2.8.tar.gz"

#
# Download this extension
#
git clone https://github.com/cuber/ngx_http_google_filter_module

#
# download Substitutions Extension
#
git clone https://github.com/yaoweibin/ngx_http_substitutions_filter_module


#
# decompression
#
Tar xzvf nginx-1.7.8.tar.gz
Tar xzvf pcre-8.38.tar.gz
Tar xzvf openssl-1.0.1j.tar.gz
Tar xzvf zlib-1.2.8.tar.gz

#
# Enter the Nginx source directory
#
CD nginx-1.7.8

#
# Set compilation options
#
./configure \
--prefix=/opt/nginx-1.7.8 \
--with-pcre=.. /pcre-8.38 \
--with-openssl=.. /OPENSSL-1.0.1J \
--with-zlib=.. /zlib-1.2.8 \
--with-http_ssl_module \
--add-module=.. /ngx_http_google_filter_module \
--add-module=.. /ngx_http_substitutions_filter_module

#
# Compile, install
# If the extension has an error, please send issue to
# https://github.com/cuber/ngx_http_google_filter_module/issues
#
Make
sudo make install

#
# Start up, the installation process is over
#
Sudo/opt/nginx-1.7.8/sbin/nginx

#
# After configuration changes, you need to reload Nginx for the configuration to take effect.
#
Sudo/opt/nginx-1.7.8/sbin/nginx-s Reload
Migrating from release

#
# Install GCC & git
#
Apt-get install build-essential git gcc g++ make

#
# Install the release version
# (Please ignore the installed)
#
Apt-get Install Nginx

#
# View release compilation options and versions
#
Nginx-v
# Nginx version:nginx/1.4.7
# Built by GCC 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
# TLS SNI Support enabled
# Configure arguments:
#--with-cc-opt= '-g-o2-fstack-protector--param=ssp-buffer-size=4 \
-wformat-werror=format-security-d_fortify_source=2 ' \
#--with-ld-opt= '-wl,-bsymbolic-functions-wl,-z,relro '
#--prefix=/usr/share/nginx \
#--conf-path=/etc/nginx/nginx.conf \
#--http-log-path=/var/log/nginx/access.log \
#--error-log-path=/var/log/nginx/error.log \
#--lock-path=/var/lock/nginx.lock \
#--pid-path=/run/nginx.pid \
#--http-client-body-temp-path=/var/lib/nginx/body \
#--http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
#--http-proxy-temp-path=/var/lib/nginx/proxy \
#--http-scgi-temp-path=/var/lib/nginx/scgi \
#--HTTP-UWSGI-TEMP-PATH=/VAR/LIB/NGINX/UWSGI \
#--with-debug \
#--with-pcre-jit \
#--with-ipv6 \
#--with-http_ssl_module \
#--with-http_stub_status_module \
#--with-http_realip_module \
#--with-http_addition_module \
#--with-http_dav_module \
#--with-http_geoip_module \
#--with-http_gzip_static_module \
#--with-http_image_filter_module \
#--with-http_spdy_module \
#--with-http_sub_module \
#--with-http_xslt_module \
#--with-mail \
#--with-mail_ssl_module

#
# Download the corresponding Nginx large version
# Nginx Official website:
# http://nginx.org/en/download.html
#
wget "Http://nginx.org/download/nginx-1.4.7.tar.gz"

#
# Download this extension
#
git clone https://github.com/cuber/ngx_http_google_filter_module

#
# download Substitutions Extension
#
git clone https://github.com/yaoweibin/ngx_http_substitutions_filter_module

#
# Install the dev package for dependent libraries
#
Apt-get Install Libpcre3-dev libssl-dev zlib1g-dev libxslt1-dev libgd-dev Libgeoip-dev

#
# Please compare the Configure parameters of your own release to configure, do not copy the following configuration directly
#
./configure \
--with-cc-opt= '-g-o2-fstack-protector--param=ssp-buffer-size=4 \
-wformat-werror=format-security-d_fortify_source=2 ' \
--with-ld-opt= '-wl,-bsymbolic-functions-wl,-z,relro '
--prefix=/usr/share/nginx \
--conf-path=/etc/nginx/nginx.conf \
--http-log-path=/var/log/nginx/access.log \
--error-log-path=/var/log/nginx/error.log \
--lock-path=/var/lock/nginx.lock \
--pid-path=/run/nginx.pid \
--http-client-body-temp-path=/var/lib/nginx/body \
--HTTP-FASTCGI-TEMP-PATH=/VAR/LIB/NGINX/FASTCGI \
--http-proxy-temp-path=/var/lib/nginx/proxy \
--HTTP-SCGI-TEMP-PATH=/VAR/LIB/NGINX/SCGI \
--HTTP-UWSGI-TEMP-PATH=/VAR/LIB/NGINX/UWSGI \
--with-debug \
--with-pcre-jit \
--with-ipv6 \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_realip_module \
--with-http_addition_module \
--with-http_dav_module \
--with-http_geoip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_spdy_module \
--with-http_sub_module \
--with-http_xslt_module \
--with-mail \
--with-mail_ssl_module \
--add-module=.. /ngx_http_google_filter_module \
--add-module=.. /ngx_http_substitutions_filter_module

#
# Overwrite binary files
#
CP-RF Objs/nginx/usr/sbin/nginx

#
# Restart Nginx This is the end of the migration process
#
Service Nginx Stop
Service Nginx Start

#
# After configuration changes, you need to restart Nginx for the configuration to take effect
#
Service Nginx Restart

Basic Configuration methods

HTTP Configuration method

server {
server_name < your domain >;
Listen 80;

Resolver 8.8.8.8;
Location/{
Google on;
}
}
HTTPS configuration mode

server {
server_name < your domain >;
Listen 443;

SSL on;
Ssl_certificate < your certificate >;
Ssl_certificate_key < your private key >;

Resolver 8.8.8.8;
Location/{
Google on;
}
}

Advanced Configuration Method
Basic Search
Need to configure resolver for domain name resolution

server {
# ... Enumerate only partial configurations
Resolver 8.8.8.8;
Location/{
Google on;
}
# ...
}

Google academic

Google_scholar relies on Google, so google_scholar cannot be used independently.
Because of the recent upgrade of Google Academic, the use of the HTTPS protocol, and NCR has been supported, so no longer need to specify the Google academic TLD
Configure Nginx

Location/{
Google on;
Google_scholar on;
}

Default language preference

The default language preference can be set by google_language, and if not set, the default is to use ZH-CN (Chinese)

Location/{
Google on;
Google_scholar on;
# set into German
Google_language "de";
}

The supported languages are as follows.

AR-> Arab
BG-> Bulgaria
CA-> Catalan
ZH-CN-> China (Simplified)
ZH-TW-> China (trad.)
HR-> Croatia
CS-> Czech
Da-> Denmark
NL-> Holland
En-> English
TL-> Philippines
Fi-> Finland
FR-> France
De-> Germany
El-> Greece
IW-> Hebrew
Hi-> Hindi
Hu-> Hungary
ID-> Indonesia
It-> Italy
JA-> Japan
Ko-> North Korea
LV-> Latvia
Lt-> Lithuania
No-> Norway
FA-> Persia
PL-> Poland
PT-BR-> Portugal (Brazil)
PT-PT-> Portugal (Portugal)
RO-> Romania
RU-> Russia
SR-> Serbia
SK-> Slovakia
SL-> Slovenia
ES-> Spain
SV-> Sweden
TH-> Thailand
TR-> Turkey
UK-> Ukraine
VI-> Vietnam

Search Engine Crawler License
Any search engine crawler is not allowed to crawl Google mirrors
The default robots.txt as follows is already built in.

User-agent: *
Disallow:/
If you want to use Google's own robots.txt, set Google_robots_allow to On

#...
Location/{
Google on;
Google_robots_allow on;
}
#...

Upstream reduces the cost of a domain name resolution, and by configuring Google IP with multiple network segments, it can reduce the probability of being detected by Google robot recognition programs (bomb verification code) to some extent.

# Google IP can be obtained through the following methods

# Google IP can be obtained through the following methods
➜~ Dig www.google.com @8.8.8.8 +short
173.194.38.209
173.194.38.211
173.194.38.212
173.194.38.210
173.194.38.208

You can then configure the IP to be acquired as follows

Upstream www.google.com {
Server 173.194.38.209:443;
Server 173.194.38.211:443;
Server 173.194.38.212:443;
Server 173.194.38.210:443;
Server 173.194.38.208:443;
}

Proxy protocal
The default is to communicate with the back-end server using HTTPS.
You can use Google_ssl_off to force some domains down to the HTTP protocol.
This setting allows some domains that need to be forwarded two times to be forwarded through the HTTP protocol, thereby no longer relying on SSL certificates.

#
# For example ' www.google.com ' acting as follows
# VPS (HK)-> VPS (US)-> Google
#

#
# VPS (HK) configuration
#
server {
# ...
Location/{
Google on;
Google_ssl_off "www.google.com";
}
# ...
}

Upstream www.google.com {
IP >:80; for server < VPS (US)
}

#
# VPS (US) configuration
#
server {
Listen 80;
server_name www.google.com;
# ...
Location/{
Proxy_pass https://www.google.com;
}
# ...
}

Copyright & License
All code follows the same open source protocol as Nginx