Dynamic Upstreams of Nginx

Original address: Http://www.oschina.net/translate/nginx-with-dynamic-upstreams

I recently made a setup in my work and I have a user-oriented Nginx service that forwards the access to run on AWS Elastic Load Balancer (as you know. ELB) on a service. This does not seem to be a difficult task in itself, you just need to find the hostname of ELB and Ngin X point to it, so it's not done, right?

Location/{Proxy_pass http://service-1234567890.us-east-1.elb.amazonaws.com;}

Test no problem, and then set up the firewall/security group configuration correctly, it should be able to work well. After a few hours, you may find that the service is no longer working, although no changes have been made. Direct access to the ELB endpoint is working, but access to Nginx is always timed out.

Elb-End Enlightenment

To figure out why the service suddenly stopped, you need to know how ELB works:

When you create an elastic load balancer (Elastic load Balancer), you will get a return record of the DNS, and AWS will tell you all the access services that are in use. A DNS record is a polling DNS (round robin DNS) record that points to two or more IP addresses-depending on how many zones you have available. The DNS record is set to 60 seconds of lifetime (time to live), which means there is almost no record cache.

Short TTL allows AWS to quickly change the running load on the machine without any complex virtual IP problems without disrupting the service. That's why they specifically tell you not to look up the hostname and send traffic to one of the IP addresses, so that your service may stop working for load balancing at some undefined time in the future.

Back to Nginx

The problem is that, for Nginx, when it reads a configuration, it immediately requests the hostname to DNS and then uses its results until the next time the configuration is reloaded. Before this time comes, ELB may change the IP address and let your Nginx forward the request to some address that is not serving you.

Nginx Plus

The solution to this problem is to pay for Nginx Plus, which adds the resolve tag to indicate the server on the upstream packet. That's what makes Nginx proud of the DNS record of the TTL, occasionally re-processing the records sequentially, and getting a list of updates that the server uses.

For this feature cost per server $1.500 each year, which seems to cost a lot. Of course this is the other feature you want to get with Nginx Plus, and if you don't need them, this will be an expensive upgrade.

Free Choice

A more affordable option is to write such a configuration:

?

12345

resolver 172.16.0.23;set $upstream_endpoint http://service-1234567890.us-east-1.elb.amazonaws.com;location / {    proxy_pass $upstream_endpoint;}

It will take effect and Nginx will follow the TTL of logging DNS Records, and if a request comes in, it will be re-interpreted and the cached record will expire. Why is that?

The answer can be found at the end of the proxy_pass instruction document, which declares:

The server name, port, and the passed URI can also be specified using a variable:

?

1	proxy_pass http://$host$uri;

Even like this:

?

1	proxy_pass $request;

In this case, the server name is looked up in the server groups described, and if not found, it is determined using resolver.

When we provide Proxy_pass with a variable, we basically use it to change behavior, but it does require us to specify a DNS resolver in the configuration. The DNS resolver used in the example should be able to work on all servers running in the default VPC or EC2 on AWS (applicable). You can also check/etc/resolv.conf at any time to find out which DNS servers are available and used by AWS for your server.

Caveat about forwarding URIs (warning)

If the location you set in Nginx is not just/, then you need to notice that when given a variable as a parameter, Proxy_pass subtly alters the behavior.

First of all, it is important to quickly summarize how proxy_pass works in normal operation:

Normal performance behavior.

Imagine that we have an Nginx configuration that includes these:

location/foo/{Proxy_pass http://127.0.0.1:8080;}

When we send a/foo/bar/baz request to this site, Nginx forwards the request to Http://127.0.0.1:8000/foo/bar/baz.

location/foo/{# Note the trailing slash↓proxy_pass http://127.0.0.1:8080/;}

Nginx will remove the specified URI from the location record and pass the remainder to the upstream server. So the request/foo/bar/baz will be forwarded to Http://127.0.0.1:8080/bar/baz.

Change behavior

When we use a variable as the parameter of the Proxy_pass, the behavior of the trailing slash above will change. For example, we have this configuration.

?

12345

resolver 172.16.0.23;set$upstream_endpoint http://service-1234567890.us-east-1.elb.amazonaws.com/;location /foo/{    proxy_pass $upstream_endpoint;}

When we send a request/foo/bar/baz to that configuration, the forwarding request will not go to/and not the expected/bar/baz.

The solution is to remove the trailing slash from upstream's endpoint and then manually rewrite it like this:

?

123456

< Code class= "Shell Plain" >resolver 172.16.0.23; set   $upstream _endpoint http: Code class= "Shell plain" >//service-1234567890 .us-east-1.elb.amazonaws.com; location  /foo/  {      rewrite ^ /foo/ (. *)  /$1  break &NBSP;&NBSP;&NBSP;&NBSP; proxy_pass  $upstream _endpoint;

Then when you send the request/foo/bar/baz,upstream will accept the request we want/bar/baz.

Concluding remarks

To know that this is not only applicable to setting up a upstream server with ELB, it is suitable for configuring all upstream servers in Nginx to modify the DNS configuration situation.

Hope this works for you, if you have any suggestions or just want to contact me simply, use Twitter to contact Tenzer.

Dynamic Upstreams of Nginx