top

Tuesday, 30 October 2012

Tomcat Clustering Series Part 2 : Session Affinity Load Balancer

Hi this second part of the Tomcat Clustering Series. In my first part we discuss about how to setup simple load balancer. and we seen how load balancer distribute the request to tomcat instance in round robin fashion. [Check the video below for better understanding]

In this post we discuss about what is the problem is occur in simple load balancer when we introduce sessions in our web application. and we will see how to resolve this issue.

Its continuation of my first part of this series. so kindly go read my first part then continue here.

How Session works in Servlet/Tomcat?
       Before going into problem, let see the session management in Tomcat.
If any if the page/servlet create the session then Tomcat create the Session Object and attached into group of session (HashMap kind structure) and that session can identify using session-id, its just random number generated through any one of the hash algorithm. then respond to client with cookie header field. That cookie header field are key value pair. so tomcat create jsessioid is the key and random session-id is the value.

Once response reached to client (Web Browser) its update the cookie value. If already exist, then its overrides the cookie value. Then all further communication browser send the cookie attached with request to that server.

HTTP is stateless protocol. so server can't find the client session usual way. so server reads the header of the request and extract the cookie value and server got the Random session-id. then it search through group of session maintained by the tomcat. (It usually hash-map). then tomcat got perfect Session of that particular client (Web Browser).

If client cookie value doesn't match with group of sessions then tomcat create the completely new session and send the new cookie to browser. then browser update it.


this index.jsp code to deploy all tomcat instances
<%@page import="java.util.ArrayList"%>
<%@page import="java.util.Date"%>

<%@page import="java.util.List"%>
<%@page contentType="text/html" pageEncoding="UTF-8"%>


    
        
        JSP Page
    
    

        Instance 1 


        
       
        
Session Id : <%=request.getSession().getId()%> Is it New Session : <%=request.getSession().isNew()%> Session Creation Date : <%=new Date(request.getSession().getCreationTime())%> Session Access Date : <%=new Date(request.getSession().getLastAccessedTime())%> Cart List
    <% String bookName = request.getParameter("bookName"); List listOfBooks = (List) request.getSession().getAttribute("Books"); if (listOfBooks == null) { listOfBooks = new ArrayList(); request.getSession().setAttribute("Books", listOfBooks); } if (bookName != null) { listOfBooks.add(bookName); request.getSession().setAttribute("Books", listOfBooks); } for (String book : listOfBooks) { out.println("
  • "+book + "
  • "); } %>

Book Name



What is the Problem In Simple Load Balancer?
     If we deploy the web application, in which its used the session then actual problem is occurred.

I describe with sequence

1. User request one web page, in that web page its used sessions (like shopping cart).
2. Load balancer intercept the request and use the round robin fashion its send to one of the tomcat. suppose this time its send to tomcat1.
3. tomcat1 create the session and respond with cookie header to client.
4. load balancer just act as relay. its send back to client.


5. next time user request again the shopping cart to server. this time user send the cookie header also
6.Load balancer intercept the request and use the round robin fashion its send to one of the tomcat. this time its send to tomcat2.
7. Tomcat 2 receive the request and extract the session-id. and this session id is doesn't match with their managed session. because this session is available only in tomcat1. so tomcat 2 is create the new session  and send new cookie to client
8. Client receive the response and update the cookie(Its overwrite the old cookie).


9. Client send one more time to request that page and send the cookie to server.
10. Load balancer intercept the request and use the round robin fashion its send to one of the tomcat. this time its send to tomcat3.
11. Tomcat 3 receive the request and extract the session-id. and this session id is doesn't match with their managed session. because this session is available only in tomcat2. so tomcat3 is create the new session  and send new cookie to client
12. Client receive the response and update the cookie.(Its overwrite the old cookie).


13. Client send one more time to request that page and send the cookie to server.
14. Load balancer intercept the request and use the round robin fashion its send to one of the tomcat. this time its send to tomcat1.
15. Tomcat 1 receive the request and extract the session-id. and this session id is doesn't match with their managed session. because client session id is updated by tomcat 3 last time. so even though tomcat 1 have one session object created by this client. but client session id is wrong.so tomcat1 is create the new session  and send new cookie to client (for more info watch the video below)
16. Client receive the response and update the cookie.


this sequence is continue ...

as the result every request one session is created. instead of continue with old one.


Here root cause is Load balancer. If load balancer redirect the request correctly then this problem is fixed. but how load balancer know in advance about this client before is processed by particular tomcat.
HTTP is stateless protocol. so HTTP doesn't help this situation. and other information is jsessionid cookie. its good but its just random value. so we can't take decision based on this random value.
ex:
Cookie: JSESSIONID=40025608F7B50E42DFA2785329079227


Session affinity/Sticky Session

Session affinity overrides the load-balancing algorithm by directing all requests in a session to a specific tomcat server. so when we setup the session affinity our problem is solved. but how to setup because session values are random value. so we need to generate the session value some how identify the which tomcat generate response.

jvmRoute

Tomcat configuration file (server.xml) caontain  <Engine> tag have jvmRoute property for this purpose. so edit the config file and update the <Engine > tag like this

<Engine name="Catalina" defaultHost="localhost“ jvmRoute=“tomcat1” >

here we mention jvmRoute="tomcat1" here tomcat1 is worker name of this tomcat. check the workers.properties file in last post

Add this line to all tomcat instances conf/server.xml file and change the jvmRoute value according to workers name and restart the tomcat instances.

Now all tomcat generate the session-id pattern like this
<Random Value like before>.<jvmRoute value>

ex: tomcat1 generate the session id  like
Cookie:JSESSIONID=40025608F7B50E42DFA2785329079227.tomcat1


here tail have which tomcat generate the session. so load balancer easily find out to where we need to delegate the request. in this case its tomcat1.

so update all tomcat instances conf/server.xml file to add the jvmRoute property to appropriate worker name values. and restart the instances. all problem is fixed and entire load balance works fine even session based application.

but there is still one drawback

if 5 user acessing the website. In session affinity is setup. here

tomcat 1 serves 2 user,
tomcat 2 serves 2 user, 
tomcat 3 serves 1 user,  then suddenly one of instance is failed, then what happen?

suppose instance 1 (tomcat1) is failed, then those 2 users lost their session. but their request are redirect to one of the remaining tomcat instances (tomcat2,tomcat3). so they still access the web page. but they lost previous sessions. this is one of the draw back. but its compare to last post load balancer. its works in session based web application also.

next post we will see how to set up the session replication in load balancer.


please share ur thoughts through comments.

Update : Check the Session Replication Process in Third Part of this Series


Screen Cast:

Wednesday, 10 October 2012

Tomcat Clustering Series Part 1 : Simple Load Balancer

I am going to start new series of posts about Tomcat clustering. In this first post we will see what is problem in normal deployment in only single machine, what is clustering and why is necessary and how to setup the simple load balancer with Apache httpd web server + Tomcat server cluster.[Check the video for better understanding]

Why need Clustering? (Tomcat Clustering) 
          In normal production servers are running in single machine. If that's machine may be failed due to crashed or hardware defects or OutOfMemory exception then  our site can't access by anybody.
so how to solve this problem?. to add more tomcat machine to collectively (group/cluster) run as a production server. (oppose of single machine). Each tomcat has deployed the same web application. so any tomcat can process the client request. If one tomcat is failed, then other tomcat in the cluster to proceeds the request.

Here one big problem is arrive. each tomcat instances are running in dedicated physical machine or many tomcat instances are running in single machine(Check my post about Running multiple tomcat instances in single machine). so each tomcat running on different port and may be in  different IP.

the problem is in client perspective, to which tomcat we need to make the request? because there are lots of tomcat part of clustering is running. each tomcat we need to make IP and Port combination. like
http://192.168.56.190:8080/ or http://192.168.56.191:8181/
so how to solve this problem?
To add one server in-front of all tomcat clusters. to accept all the request and distribute to the cluster. so this server acts as a load balancer. There is lots of server is available with load balancing capability. here we are going to use Apache httpd web server as a load balancer. with mod_jk module.
so now all clients to access the load balancer (Apache httpd web server) and don't bother about tomcat instances. so now ur URL is http://ramkitech.com/ (Apache runs on port 80).

Apache httpd Web Server
           Here we are going to use Apache httpd web server as a Load Balancer. To provide the load balancing capability to Apache httpd server we need to include the either mod_proxy module or mod_jk module. here we are using mod_jk module.

Before continuing this post check my old post (Virtual Host Apache httpd server) about How to install the Apache httpd server and mod_jk module. and how to configure the mod_jk.

How to setup the Simple Load Balancer
         For simplicity purpose i going to run 3 tomcat instances in single machine(we can run on dedicated machine also) with Apache httpd web server. and single web application is deployed in all tomcat instances.
here we use mod_jk module as the load balancer. by default its use the round robin algorithm to distribute the requests. now we need to configure the workers.properties file like virtual host concept in Apache httpd server.

worker.list=tomcat1,tomcat2,tomcat3

worker.tomcat1.type=ajp13
worker.tomcat1.port=8009
worker.tomcat1.host=localhost

worker.tomcat2.type=ajp13
worker.tomcat2.port=8010
worker.tomcat2.host=localhost

worker.tomcat3.type=ajp13
worker.tomcat3.port=8011
worker.tomcat3.host=localhost

here i configure the 3 tomcat instances in workers.properties file. here type is ajp13 and port is ajp port (not http connector port) and host is IP address of tomcat instance machine.

there are couple of special workers we need add into workers.properties file.

First one is add load balancer worker, here the name is  balancer (u can put any name).


worker.balancer.type=lb
worker.balancer.balance_workers=tomcat1,tomcat2,tomcat3

here this worker type is lb, ie load balancer. its special type provide by load balancer. and another property is balance_workers to specify all tomcat instances like tomcat1,tomcat2,tomcat3 (comma separated)

Second one, add the status worker, Its optional. but from this worker we can get statistical of load balancer.

worker.stat.type=status

here we use special type status.

now we modify the worker.list property.

worker.list=balancer,stat

so from outside there are 2 workers are visible (balancer and stat). so all request comes to balancer. then balancer worker manage all tomcat instances.

complete workers.properties file

worker.list=balancer,stat

worker.tomcat1.type=ajp13
worker.tomcat1.port=8009
worker.tomcat1.host=localhost

worker.tomcat2.type=ajp13
worker.tomcat2.port=8010
worker.tomcat2.host=localhost

worker.tomcat3.type=ajp13
worker.tomcat3.port=8011
worker.tomcat3.host=localhost


worker.balancer.type=lb
worker.balancer.balance_workers=tomcat1,tomcat2,tomcat3

worker.stat.type=status


Now workers.properties confiuration is finished. now we need to send the all request to balancer worker.
so modify the httpd.conf file of Apache httpd server

LoadModule    jk_module  modules/mod_jk.so

JkWorkersFile conf/workers.properties

JkLogFile     logs/mod_jk.log
JkLogLevel    emerg
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkOptions     +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat     "%w %V %T"

JkMount  /status  stat
JkMount  /*  balancer

the above code is just boiler plate code. 1st line load the mod_jk module, 2nd line to specified the worker file (workers.properties file). all others are just logging purpose.

The last 2 lines are important.
JkMount  /status  stat   means any request to match the /status then that request forward to stat worker. Its status type worker. so its shows status of load balancer.

JkMount  /*  balancer this line matches all the request, so all request is forward to balancer worker. In balancer worker is uses the round robin algorithm to distribute the request to other tomcat instances.

That's it.
now access the load balancer from the browser. each and every request is distribute to 3 tomcat instances. If one of the tomcat instances are failed then load balancer dynamically understand and stop to forward the request to that failed tomcat instances. Other tomcat instances are continue to work. If that failed tomcat is recovered from failed state to normal state then load balancer add to cluster and forward the request to that tomcat. (check the video)

Here big question is How Load balancer knows when one tomcat instance is failed or tomcat is just recovered from failed state?
Ans : when one tomcat instance is failed, load balancer don't know about that instances is failed. so its try to forward the request to all tomcat instances. If load balancer try to forward the request to failed tomcat instance, its will not respond. so load balancer understand and marked the state as a failed and forward the same request to another tomcat instances. so client perspective we not feel one tomcat instances are failed.

when tomcat instances recovered from failed state. that time also load balancer don't know that tomcat is ready for processing. Its still marked the state is failed. In periodic interval load balancer checks the health status of all tomcat instances. (by default 60 sec). after checking health status then only load balancer came to know that tomcat instance is ready. and its update the status is OK.


please share ur thoughts through comments.

Video :
Related Posts Plugin for WordPress, Blogger...