What benefits can Elasticsearch:nginx bring to the ElasticSearch cluster?

Elasticsearch is an advanced, high-performance, extensible open-source search engine that provides both structured and unstructured data for full-text search and real-time analysis.

It features a RESTful API that can be easily integrated into existing Web architectures using HTTP. Therefore, in the case of high concurrency, we can use Nginx reverse proxy load balancer to multiple Elasticsearch servers.

Architecture diagram:

So what are the benefits of using Nginx?

1. Log logs for each API access request. (Elasticsearch itself does not support this feature, only Slowlog and service logs)

2. Support a large number of client connections. ES official blog In the recommended use of keep-alives, between Nginx and ES using a long connection. I understand that because, in general, ES is the bottom-level of the schema, access to it is generally a fixed top-level service, which is appropriate for use with keep-alive. (It doesn't really work, no keep-alive,nginx can be used to support a larger number of client connections)

3. Load-balanced requests Elasticsearch server.

4. Cache data, reduce the same content request Elasticsearch server again.

5. Provide active health detection (nginx plus only), constantly detect the back-end Elasticsearch server is normal, and actively switch. (When an ES hangs, Nginx does not distribute the request to this node, when the node is back to normal, auto-homing)

6. Report Rich monitoring metrics (nginx plus only), providing monitoring and management.

7. Security verification. Only clients with account name passwords are allowed access to the ES cluster.

8. Restrict access to special interfaces such as "_shutdown". (This function is quite practical)

9. Access control with roles (e.g. user role has data access, Admin role has cluster control permissions)

= = = I am the partition line of the configuration example = =

A simple nginx configuration is as follows:

Upstream elasticsearch_servers {zone elasticsearch_servers 64K;    Server 192.168.187.132:9200;    Server 192.168.187.133:9200; KeepAlive 40;}    Match Statusok {status 200;    Header content-type ~ "Application/json"; Body ~ ' status ': 200 ';}    server {Listen 9200;    Status_zone Elasticsearch;        Location/{Proxy_pass http://elasticsearch_servers;        Proxy_http_version 1.1;        Proxy_set_header Connection "";        Proxy_cache Elasticsearch;        Proxy_cache_valid 302 10m;        Proxy_cache_valid 404 1m;        Proxy_connect_timeout 5s;        Proxy_read_timeout 10s; Proxy_set_header Connection "keep-alive";      proxy_set_header proxy-connection " Keep-alive ";    Health_check interval=5s fails=1 passes=1 uri=/match=statusok; } # REDIRECT Server error pages to    The static page/50x.html Error_page 502 503 504/50x.html; Location =/50x.html {       root/usr/share/nginx/html;   }    access_log logs/es_access.log combined;} Server {    Listen 8080;    root/usr/share/nginx/html;    location/{& nbsp;       index status.html;   }    location =/ Status {        status;   }}

Long connections, load balancing, caching of valid requests for 10 minutes, proactive health monitoring, state collection.

I am the partition line for security authentication configuration = = =

A configuration with security authentication is as follows:

Events {  worker_connections  1024;} HTTP {  upstream Elasticsearch {    server 127.0.0.1:9200;  }  server {    listen 8080;    Auth_basic "Protected Elasticsearch";    Auth_basic_user_file passwords;    Location/{      Proxy_pass http://elasticsearch;      Proxy_redirect off;}}}  

Passwords files and nginx.conf are in the same directory, and the format is "User name: Crypt (3) encrypted password string" On the line:

$ printf "john:$ (OpenSSL passwd-crypt s3cr3t) n" > Passwords

After the above configuration is restarted, the direct Access service is disabled:

$ curl-i localhost:8080# http/1.1 401 unauthorized# ...

The correct user name password allows for smooth access:

$ curl-i john:s3cr3t@localhost:8080# http/1.1 ok# ...

I'm a split line for access restricted configuration = = =

Location/{  if ($request _filename ~ _shutdown) {    return 403;    break;  }  Proxy_pass Http://elasticsearch;  Proxy_redirect off;}

Once this configuration has been made, direct access to _shutdown is denied:

$ curl-i-X POST john:s3cr3t@localhost:8080/_cluster/nodes/_shutdown# http/1.1 403 forbidden# ....

For my current project, the upper-level app only needs access to the data in ES, so API interfaces such as cluster and node should deny access to the upper application. At the same time, the-delete of resources that should not be deleted should also be prohibited. The ES cluster is a security guarantee, or it can easily be modified to configure the cluster or delete large amounts of data.

I'm a multi-role configuration Split line = = =

Events {  worker_connections  1024;} HTTP {  upstream Elasticsearch {      server 127.0.0.1:9200;  }  # Allow access to/_search and/_analyze for authenticated "users"  #  Server {      listen 8081;      Auth_basic           "Elasticsearch Users";      Auth_basic_user_file users;      Location/{        return 403;      }      Location ~* ^ (/_search|/_analyze) {        proxy_pass http://elasticsearch;        Proxy_redirect off;      }  }  # Allow access to anything for authenticated "admins"  #  Server {      listen 8082;      Auth_basic           "Elasticsearch Admins";      Auth_basic_user_file admins;      Location/{        Proxy_pass http://elasticsearch;        Proxy_redirect off;}}}  

Differentiate between admins and users two permissions, admins can access all APIs, and users only allow access to _search and _analyze interfaces.

The cost of multi-role access restrictions is that each role accesses the cluster using a different port number, which is architecturally reasonable-a client needs only one role and one access port.

With Lua, more granular URL permission control is available, and Nginx's embedding of LUA is well supported and concise, so there is no more in-depth exploration. Interested to know.

Reference Documentation:

http://www.ttlsa.com/nginx/nginx-elasticsearch/

Https://www.elastic.co/blog/playing-http-tricks-nginx

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

The above describes what benefits Elasticsearch:nginx can bring to the Elasticsearch cluster. , including the aspects of the content, want to be interested in PHP tutorial friends helpful.

