Tuesday, February 5, 2013

NGINX as sticky balancer for HA using cookies

I recently needed to find a solution to implement several http load balancers that balance among several backend servers so each balancer can proxyfy any of the backend servers. The abstract architecture of the deploy is represented in this image.


Yes, the backends have an NGINX, I'll explain that later...

Of course one of the requirements was that the session had to be kept but there was a problem: the backend server did not synchronized the session among themselves. This situation directly required the balancers to send each user to the same backend server each time the user makes a request. This way,  as the one user is using only one backend, there won't be any problem with the session persistence.

I know that the Upstream Module from NGINX provide the above feature through the ip_hash parameter as, if you have in all the balancer servers the same list of backends in the same order, the IP for a user will always match the same backend server. You can check this taking a look at the module source code at
nginx-X.X.XX/src/http/modules/ngx_http_upstream_ip_hash_module.c.
However, using this module has several cons:
  • No support for IPv6. In fact, all request from an IPv6 will be redirected to the first backend server (as I understood from the source code). NOTE: IPv6 addresses are supported starting from versions 1.3.2 and 1.2.2. The version included in last ubuntu (12.10) is 1.1.19 and it is the version I was testing so it lacks of ipv6 support.
  • Colissions as it only uses the 3 first numbers of the IP for the hash. That means that all the ips of the same C-class network range will go to the same backend server.
  • All users behind a NAT will access to the same backend server.
  • If you add new backends, all the hashes will change and sessions will be lost.
Because of these problems there are some situations where the balacers will not be fair at all, overloading some nodes while others are idle. For this reason I wondered if it could be possible to balance among servers using other solution than ip_hash.

I found the project nginx-sticky-module which uses cookies for balance among backneds, but it is not integrated into nginx so I had to recompile it which I don't really like for production environments as I prefer to use official packages from the GNU/Linux distribution. So I wondered... would be NGINX so versatile that would let me implement something like this module only using configuration parameters? Guess the answer! There exist a way!

So, now that we are in context, let's start with a little test. Our objective in this example is to use nginx to balance between two backend nodes using cookies. The solution we will implement will work the way it is represented in the following diagram.






In the diagram you can see that the user USER0 when accessing to our application service is directed to balancer1 probably through a DNS round robin resolution. In step 1 the user accesses for the first time to the service so the balancer1 will choose one backend by round robin and, in this example, backend1 was chosen (for the luck of our draftsman). This backend will set the cookie backend=backend1 so the client have "registered" in the backend.

At step 2 the user will access the platform for the second time. This time, the DNS round robin will send our user to the balancer balancer3 and, thanks to the cookie named backend, the balancer will be able to know which backned should attend the request: backend1.

Now that we know how it works, how do we make it work with NGINX ?


Well, lets talk first about how the cookie is set in the backend server. If you can change some code on the web application you can force the application to set the cookie for you. If not, you can proxify the application with NGINX in the same server and you could even use that NGINX to implement the cache of the platform. I will choose to proxyfy the backend application with NGINX as it is a generic solution to our problem.

So a very simple configuration of the NGINX in the backend server backend1 would look like this:


server {
   listen 80;

   location / {
      add_header Set-Cookie backend=backend1;
      proxy_pass http://localhost:8000/;
      proxy_set_header X-Real-IP $remote_addr;
   }
}

Each backend server will have to have an specific configuration with its backend identifier. I recommend you for testing to set up two backends like this one with ids backend1 and backend2. Of course in production environments we could use md5 hashes or uuids to avoid predictable identifiers.

We can now configure a balancer to balance between these two backends. The balancer configuration would look like this:

upstream rrbackend  {
   server 192.168.122.201;
   server 192.168.122.202;
}

map $cookie_backend $sticky_backend {
   default bad_gateway;
   backend1 192.168.122.201;
   backend2 192.168.122.202;
}

server {
    listen 80;


    location / {
       error_page 502 @rrfallback;
       proxy_pass http://$sticky_backend$uri;
    }
  
   location @rrfallback {
      proxy_pass  http://rrbackend;
   }
}

The good part is that the configuration will be the same for all balancers if you have more than one. But lets explain how this configuration works.

In first place we declare the backends for the round robin access.

Then we map the cookie backend variable to the $sticky_backend NGINX variable. If $cookie_backend is backend1, then $sticky_backend value will be 192.168.122.201 and so on. If no match is found, the default value bad_gateway will be used. This part will be used to choos the sticky backend.

The next thing is to declare the server part, with its port (server_name and whatever) and its locations. Note that we have a @rrfallback location that resolves to a backend using round robin. When a user access the location /, two things can happen:
  1. The user accesses for the first time so no backend cookie is provided and $sticky_backend NGINX variable will have the value bad_gateway. In this case, the proxy_pass will try to access a server called bad_gateway which will return an error 502 (Bad Gateway) and, as declared in the line above the proxy_pass, this error will fallback to @rrfallback so a backend will be choosed by round robin.
  2. The users passes the backend cookie with a valid value so the $sticky_backend NGINX variable will have a valid IP from the map and redirected to the backend indicated in the cookie through the proxy_pass.
  3. The same than 2. but the backend is down. In this case, the proxy_pass will return a 502 error and the process will start from 1: new backend by round-robin.

And this is all! You can test it both with a browser to check that you'll stay on the same server or make requests with curl/wget without sending the cookie to check the initial round-robin.

As always, comments are more than welcome!








21 comments:

  1. This set up seemed to work for us initially, but we found that when using a php redirect on our application server, it breaks. For some reason nginx doesn't handle the redirect correctly when a redirect request comes in and nginx responds with the 502 bad gateway. Any thoughts appreciated. Thanks.

    ReplyDelete
    Replies
    1. How are you setting the cookies? In this example I configured an nginx to set the cookies, but some applications delete that cookie from the session and cause missbehaviours.

      I would need some more information on the nginx configuration and how the cookies are being sent. You can contact me at g+.

      Regards.

      Delete
    2. Tried sending you a couple posts on g+.

      Delete
    3. Not sure I have a handle on how g+ works, sorry.

      Here's the configs,

      nginx config : http://pastebin.com/QDpjDSBY
      site.conf config : http://pastebin.com/SyQTmq8v

      The
      rewrite /(.*) /$1 break;
      proxy_set_header Host $host;
      proxy_redirect default;
      proxy_pass_request_headers on;

      lines in the site1.conf were me playing with it trying to get it to work, those lines may not be needed.

      Delete
    4. same files in gist,

      https://gist.github.com/christisking/456584c73eeba526d2dc
      https://gist.github.com/christisking/2eca0a49a36bd6b5f390

      Delete
    5. headers are getting set in php like this :

      $insertGoTo = "https://www.site1.com/dir/client/index.php";
      header(sprintf("Location: %s", $insertGoTo));

      Delete
    6. I've received your messages at g+, but anyway I'll post here :).

      I meant the cookie used by the balancer: in your case 'cachestat'. Anyway you are inserting a Location header to implement a redirect but are you returning a 301 or 302 HTTP code?

      Delete
  2. Yes, cachestat is being set in our php app code on the apache backend webs. In our header function above, we aren't explicitly setting 301 or 302 ... I assume the default is a 302 based on the php docs.

    ReplyDelete
    Replies
    1. Well, I don't know how does php work but you should to ensure that your application is sending the HTTP redirection codes.

      Delete
    2. http://us2.php.net/manual/en/function.header.php

      Does a php header redirect work with your setup?

      Delete
    3. It should work but I haven't tested it.

      Delete
  3. The 502 didn't work for us with php based redirects , had to end up using nginx split_client to randomize ...

    ReplyDelete
  4. do you know any way to implement HA with this approach? because if one instance fall down, the client does not be routed to another.
    tks.

    ReplyDelete
    Replies
    1. Take a look at @rrfallback. If one upstream server falls nginx, instead of return a 502 error to the client, will choose another upstream using round robin, so the client will be routed to another application server.

      Delete
  5. Hi

    have you ever tried sticky module with https connections? Is possible to to that?

    Thanks!

    PD:awesome post by the way :)

    ReplyDelete
    Replies
    1. Hi,

      I hadn't tried with https connections but I don't see why it shouldn't work. Only to point out that you'll have to use the backend certificates on the balancers so the users negotiate the SSL connection with the balancers properly.

      Thanks for the comment and the compliment.

      Delete
  6. hi diego, nice receipt! we're setting up a page dedicated to nginx and some modules we are building (see http://nginx-goodies.com/ ) and i'd like to ask if you could contact via email that is displayed there. i'd like to include your post into our lb-receipt-collection as a nice way to create stickyness with out-of-the-box-modules

    ReplyDelete
  7. Hi Harry, I answered you weeks ago in a direct message to your g+ account, just in case you didn't noticed.
    BTW, nice idea with the page, it will be very useful for sure!

    ReplyDelete
  8. One very, very important fact that needs to be mentioned for those who are coming here for Google, looking for a way to do sticky sessions w/ Nginx:

    If your upstream servers use hostnames instead of IP addresses, you must use the `resolver` directive in your server block. Otherwise, Nginx does not know how to turn the hostname string contained in the variable into an address.

    A coworker and I spent a good part of a day trying to figure this one out! Please read this blog post for more information: http://www.nginx-discovery.com/2011/05/day-51-proxypass-and-resolver.html

    ReplyDelete
    Replies
    1. Yes, that is right. Thanks a lot for taking the time to post this in a comment.

      Delete
  9. Hello. I think, you can use "rrbackend" as default, so you don't need trick with 502.

    upstream rrbackend {
    server 192.168.122.201;
    server 192.168.122.202;
    }

    map $cookie_backend $sticky_backend {
    default rrbackend;
    backend1 192.168.122.201;
    backend2 192.168.122.202;
    }

    server {
    listen 80;

    location / {
    proxy_pass http://$sticky_backend$uri;
    }
    }

    ReplyDelete