On the memcached of Key/value King: three, memcached the application of distributed storage scene to solve session

A high-availability session Server scenario Introduction 1.1 stateless features of the application server

The application-tier server (which is generally referred to as the Web server) handles the business logic of the Web application, one of the most notable features of the application: the stateless nature of the application.

PS: mention of stateless features, have to say the HTTP protocol . We often hear that HTTP is a stateless protocol, with two consecutive requests for the same session not knowing each other, they are parsed by the newly instantiated environment, and the environment does not save any information about the session, except that the application itself may have been stored in the global object. The reason we don't feel the stateless nature of HTTP when we use ASP. WebForm is because Microsoft has implemented ViewStatefor us, which is the basic unit for saving page information in ASP. NET WebForm. The essence is a hidden field in HTML that will submit data from this hidden field to the server side when the callback occurs.

In many scenarios, users need to interact with our site systems multiple times, and a solution is needed to overcome the dilemma of stateless features. Fortunately, on the shoulders of giants, we have a good solution, that is browser-side cookie and server-side session. In the general single-machine development (this is generally referred to as the case of only one Web server), the server side we usually use session to store the user login status (typically a custom object instance), in most of the Management Information system development (after all, the internal system user volume is not much, It is common for a Web server to provide both Web services and memory for session objects.

However, under the large user volume, the single version of the session will appear inefficient, and even drag the performance of the Web server. This is because each user's HTTP requests are sent to the server side, and the server software for each Web server (for example, IIS, Tomcat, and so on) creates a thread for the request to process and respond. However, the number of requests that a server can receive at the same time is limited after all (depending on the configuration of the server, such as the number of threads that can be created for i3, i5, and i7 types in the CPU), When the number of high concurrent requests in a time period (for example: the network purchase seconds to kill the system often at the same time there will be a large number of concurrent), then this application server will receive an unprecedented load of requests, and eventually can not withstand the high load and lead to downtime, the site has to stop service.

At this time, again think of that sentence: when a bull pull not a train, do not look for a stronger cow, but with two cows to the cart . Therefore, we can use the server cluster technology to improve the Web server, increase n Web server deployment of the same Web application to form a Web server cluster to provide services to the outside, through the load Balancer device or software to distribute the number of concurrent requests to each Web server evenly, For example, if a system is flooding 100,000 requests at the same time during a promotion, while 5 Web servers in the server cluster are serving simultaneously, the load Balancer device allocates the 100,000 request to the Web server in a more balanced way by an algorithm. On average, only 20,000 requests per server are assumed.

Through the server cluster, the request load problem has been better solved, then the new problem comes again: because the session default is in-process (INPROC), that is, it is stored in the Web server memory inside. When a cluster is built, the user's session is built into one of the Web servers allocated by the load balancer. But when the user next access or access to other subsystems in the system (such as: I first in the Baidu Encyclopedia to login, and then visit Baidu Bar), because the session is also stored in the last service to provide the Web server inside, System Check rule (now this Web server inside detects that the user does not have a session) will cause the user's repeated login (such as: are in Baidu's Web page, it lets you login several times, you cool?) Obviously, it's uncomfortable. At this point, we need to solve the Web server cluster session management, let's look at how to do the Web server cluster session management.

1.2 Session Management of Application server cluster

Let's take a look at some common ways of session management in a clustered environment:

①session replication: This scheme is simple, the cluster of several servers synchronization between the session object, any one server downtime will not result in the loss of Session objects, the server only need to get from the computer. However, this scenario is only suitable for small clusters. When the scale is large, a large number of session copy operations will occupy the server and network resources, the system overwhelmed .

②session binding: Use the load-balanced source address hash algorithm to always distribute requests originating from the same IP address to the same server. In this way, during the entire session, all user requests are processed on the same server, that is, the session is bound to a particular server, ensuring that the session is always available on this server. (This scenario is also called session stickiness .)

However, this scenario does not meet high-availability requirements . Because once a server goes down, the session on the machine is no longer there, and the user requests to switch to another machine because there is no session to complete the business process. As a result, few websites use this scenario for session management.

③cookie record session: Using browser-supported cookies to record the session is simple, highly available, and supports linear scaling of the server, so many websites use cookies more or less to record the session. However, Cookie recording session has disadvantages: such as limited by cookie size, each request response to transfer cookie impact performance, the user closed the cookie will cause access is not normal, etc.

④Session Server : Using the session server (cluster) to manage the session, the application server accesses the session server every time the session is read and written. This scheme actually separates the state of the application server into a stateless application server and stateful session server .

　　　　

From the above several ways, each has pros and cons, but session server is the most consistent with the high-availability requirements of the program, but also often used in enterprises. So, for a stateful session server, a simpler approach is to take advantage of distributed caches (such as memcached, Redis, etc.) for a brief introduction to Redis that reads my blog: A nosql approach to everyone's love of Redis), Databases , etc., are encapsulated on the basis of these products to meet the storage and access requirements of the session. Based on the above introduction, we use memcached today to build our session server to solve the shared access of the session of the Web server cluster.

PS: Why use a distributed caching scheme instead of a database to store the session? This will have to analyze the performance bottleneck of data access, in general, disk IO Read and write is the slowest, because the database data is actually stored in the file, although most of the current database has a B + tree structure, Reading a single piece of data is still 4 times the data read and write (three disk access to the data index and row ID, a data file read operation, finally know the database operation more trouble). While the distributed cache such as memcached is stored in the server's memory in the simple form of key/value, the random read and write speed of memory is the burst disk IO, so the Intranet + memory dual internal mode is the perfect solution.

There are two types of disks:

① mechanical HDD: Drive the head arm through the motor, drive the head to the specified disk location to access the data . It enables fast sequential read and write, slow random read and write .

② Solid State Drive (also known as SSD): no mechanical device, data stored on a long-lasting memory of the silicon Crystal , so can be as fast as memory random access .

In the current website application, most of the application access data is random, in this case the SSD has better performance, but the cost-effective needs to be improved (pretty expensive, click).

Second, memcached implementation of the session Distributed Storage 2.0 case Overview

(1) Simulated login case scenario

Suppose we have an ASP. NET-based information system, which uses a unified system login page for users to log in, after landing the default jump to a User Center home page, and display: Welcome, {User account name}.

① System Login Page effect:

② User Homepage Effect:

(2) Technical system selection of simulation

ASP. Mvc+ef Code first+mysql+memcached

2.1 Initial preparatory work

(1) Create an empty project of ASP. MVC4, the view engine is selected as "Razor";

(2) In the project to create a new folder, named "Lib", the main storage of some necessary DLL files;

(3) Add a reference to these DLLs in the project, note that EntityFramework.dll is introduced here to support the latter Codefirst development method, the EF version must be 4.1 and above. PS: You can also install EntityFramework through the Package manager. This is where our preparations are made, and then we can begin our formal work.

2.2 Generating MySQL database with EF codefirst

First, EF is an ORM(object-relational mapping) framework that allows us to use objects to map to the underlying database structure as we program. The ORM Framework is responsible for converting the recordset from the database to an object, or generating the appropriate SQL commands to the database based on the current state of the object, and completing the data access work (common data access operations can be referred to as crud:create, Read, Update, Delete).

EF brings more efficiency to database application development, and it makes it easier to write easy-to-maintain, easy-to-scale systems, and is good enough to meet the needs of most development scenarios, although not as much as ADO. Unlike ADO, EF hasa higher level of abstraction : It maps the database to DbContext, maps the data accessed in the database directly to the entity object , and shields the underlying database from the internal structure. You do not need to complete the crud directly using the underlying objects provided by the underlying data access engine, such as the dbconnection,dbcommands provided by ADO.

EF supports three development models: Code First, Database first, and model first. Here we use Code First mode, which can help us achieve the goal of rapid development iterations. Finally, EF is not the focus of this article, and if you don't know EF or code first, you can refer to Jin Xu-liang's "EF Quick" series of articles, which are not mentioned here.

(1) Create a new class in the Models folder named "UserInfo". It is mapped to the UserInfo table in the MySQL database as our entity class (this data table is not yet created in the MySQL database)

View Code

(2) Then create a new class in the Models folder named "Mydbcontext", which is inherited from DbContext and used as the database context for EF operations. It is important to note that the Name=mysqldemo,mysqldemo here is the name of the database. And here this. The Database.createifnotexists () method is used to determine if the Mysqldemo database already exists? If it does not exist, create one.

View Code

(3) A new database connection string is added to Web. config, which is set to "Mysqldemo" and is consistent with the name in the constructor in Mydbcontext above.

View Code

At this point, since we only use the UserInfo table, we now have the end of the code for the database and EF.

2.3 Self-encapsulated memcached help class to provide service interfaces externally

(1) Add a new appsetting in Web. config to save the address list of the memcached server:

    <!--set memcached server cluster List--    <add key= "memcachedservers" value= " 192.168.80.10:11211,192.168.80.11:11211 "/>

(2) A new class in models, named "Memcachedhelper", we set it as a static class, so the method is all static, we do not need to instantiate to call directly. As you can see, we used a static constructor to initialize the global static object, which does not belong to any one instance, so this constructor is only executed once and is automatically called by. NET before the first instance of this class is created or any static members are referenced.

View Code

Here we do not have a list of configurations for Sockiopool, with its default configuration. However, it is important to note that the poolname of Sockiopool poolname and memcacheclient must be aligned.

2.4 Call Memcached Help class interface to store user logon status when user logs on

(1) Create a new controller with the name "Logoncontroller". Mainly used to display the system landing page and user authentication of the AJAX operation (the user session into memcached also in this operation). As for the login page of the HTML and JS script will not be mentioned here, please download the demo file to view.

View Code

Now look at the core code of this controller:

①index This action is mainly used to display the login view, the page code is no longer posted, just look at the following Ajax request code: with jquery Ajax to validateuserinfo this action to submit a user name and password, Returns a JSON object if the server-side check succeeds, and the browser-side determines if the success property is true and jumps to the system home page if True.

View Code

②validateuserinfo This action is based on the AJAX request submitted by the browser to determine whether the user name and password is correct. Here we use the mydbcontext we just wrote, first to instantiate it. (This step is very important, then our MySQL database has mysqldemo this database, when the first instantiation of Mydbcontext, EF will help us in MySQL to create Mysqldemo this database, Its essence is to help us generate a string of SQL statements such as crate database Mysqldemo, and then use the graceful syntax of a lambda expression to validate. If you are unfamiliar with lambda expressions, you can learn from the article "Lambda Expressions (C # Programming Guide)" in MSDN.

③ is the most central part, we use the GUID (guaranteed as the uniqueness of the Key) as SessionId write to the browser side of the cookie, and as a Key into the memcached distributed cache, It also sets the default expiration time (here is 20 minutes). After that, each time the browser submits a request to the server, the cookie is attached to the HTTP message, and the server can use the cookie as a key to go to the memcached server to find the session object.

PS: in general, Memcached's data key name is fastidious, here the wisdom of the old Horse recommended a naming convention: {namespace}-{Department name}-{project name}. Let's make it simple here, using a GUID as the key, because the GUID is unique each time it is generated.

2.5 Package Basecontroller Fix the check rule before action triggers

(1) In the past development of information system projects, we will make a global validator in the system, to determine whether the user each operation request has the corresponding permissions, here we mainly verify whether the user is logged in, so that we in the specific module to obtain the user's session object. In the case of a single machine, we generally deposit the session within the local process, so we determine whether there is a login status in the session. Here we use memcached to store the session object, then we add a rule before each action execution: determine if there is a login state for the current user in memcached, and if so, continue with the action. If the wood has, then sorry, please login .

(2) in WebForm we can write a basepage, make it inherit from page, rewrite the onload event, and then let other pages inherit from BasePage, it can be applied to the check whether the user is logged in the page class that inherits BasePage. So, in the MVC pattern, the request object is no longer the Xxx.aspx page type, but the/controllername/actionname route, so we need to look for a global filtering method for action. Well, here we can not help but think of a tall English:AOP (Aspect oriented programming), which is a hot topic in software development, using AOP to isolate parts of the business logic , Thus, the coupling degree between the parts of business logic is reduced, the reusability of the program is improved, and the efficiency of the development is improved. Therefore, we add the Global check rule to action is to separate the action's check method from the action's business process, to reduce the coupling degree, and to improve the reuse of the action checksum method, which conforms to the idea of AOP. So, for the most of the day, how exactly is it implemented in ASP. Don't worry, we can write a basecontroller like BasePage, make it inherit from the controller, and then inherit the other controllers that need to check the user's login status to Basecontroller.

View Code

(3) Now look at the above code:

① gets a cookie (i.e. SessionID) from the HTTP request message submitted by the browser, and if there is no cookie then it is not logged in, and redirects to the landing page;

② If there is a cookie then go to the memcached server query whether there is the data content of this key, if there is no content, then redirect to the landing page, if there is content, return this object, because the return is the type of object, So we need to convert the user object instance assigned to the global.

③ get to the UserInfo object, re-save it to memcached, here is actually the session extension failure time, to achieve a session of the effect of the sliding time mechanism .

PS: Because we give this cache setting expiration time in the action of login verification is an absolute time, not a sliding expiration time. The so-called absolute time is defined to expire after the specified time , and the sliding time means that no access requests are invalidated within the specified time .

2.6 Test Run user login with session storage

(1) To this end, most of the code is complete. Here we need to do a small test, a new controller, named "HomeController", the index page as the landing page, display the Session object in the Username property. The code is simple, with only two lines:

        Public ActionResult Index ()        {            //gets session from Memcached and passes to view            viewbag.username = currentuser.username;            return View ();        }    

As for view, please refer to the demo file, here is not to repeat, just a bunch of HTML.

(2) ① open two Windows Server servers in the virtual machine to ensure that the memcached service is successfully turned on. (If you are installed on this machine, you can open the service directly)

② open MySQL database service, here my MySQL is installed in this machine, so only need to open the service.

(3) Click Debug, start running the IIS Express server, test it. SAO years (here is to say: you need to enter a random account and password to create a database, because we are using code first way, and then into MySQL add a few lines of test user data, Here I have added a few lines to start the test. I added a line in the database {account: Edisonchou, Password: 123456}, so I entered the account and password, click Login, there will be a friendly prompt: "In the verification, please wait ..." (if you are running the first time, then you need to create a database, this time will be longer, Don't worry).

(4) Wait for the system verification to complete, will jump to the system home page, display the Welcome screen. As you can see, the two places are displayed according to the username of the session.

(5) Now we manually modify the URL, enter/logon/index, jump to the landing page. Then, modify the URL to/home/index, see if we need to login again, because we have put the session into the memcached, so again into the/home/index this action will be judged, Pass the past cookie through the browser to go to the memcached session, if any, continue execution. Here, we have logged in before, so we no longer need to log in again. Here is the HTTP message to jump/home/index again, you can see that after the login has been written to the cookie, after the request will be brought with a cookie to the server side, the server can also be a cookie as a key to get the session object.

(6) Open a new browser, and then use another user data to log in:

(7) After logging in, check the HTTP message to get the SessionID in the cookie:

(8) According to two users SessionID as key, in the memcached server to view the storage situation: After viewing, two sessions are stored in the 192.168.80.11 this server inside. Why there is no average distribution, this is because the algorithm is calculated based on the hash value, just the two SessionID calculated hash value is to be allocated to the 192.168.80.11 this server inside, so it is stored in this inside. (Note that this is only the test results that run on my machine, the test results of netizens may be different from mine, which is normal)

At this point, our quiz is over and our case ends here.

Iii. Summary of Learning

In this article, I first took the effort to introduce the session server scenario to the feasibility of building the session server, and then use the ASP. Mvc+ef Code first+mysql+memcached to simulate an information system login scenario, The memcached is used to store the session object, and the action filter is used to verify the user's login status. Finally, a small test is done to verify that the Memcached store our session object.

Of course, this case is just a toy-level demo, and it takes many performance tests and optimizations to apply to real-world development. However, we can learn some useful ideas, and understanding these ideas is beneficial to our practical development ability. Finally, I hope that my article can give more like me the rookie developer to bring a little light, if you feel useful, then trouble point "recommended" it, thank you!

Reference documents

(1) Hae, "large Web site technology architecture-core principles and case studies", http://item.jd.com/11322972.html

(2) Mullen, "Memcached Open Class", http://bbs.itcast.cn/thread-14836-1-1.html

(3) Cyclone, "memcached distributed cache Substitution Session Solution", http://www.cnblogs.com/xuanfeng/archive/2009/06/04/1494735.html

(4) Jin Xu-liang, "The Entity framework is a quick walkthrough", http://blog.csdn.net/bitfan/article/details/12779517

(5) MSDN, static classes and static members (C # Programming Guide), http://msdn.microsoft.com/zh-cn/library/79b3xss3.aspx

(6) Ruffian, "Memcached+cookie Replacement Session solution (MVC Edition)", http://www.cnblogs.com/piziyimao/archive/2013/01/29/2882236.html

(7) Sleepy Dragon Fly, ". NET cache usage ", http://blog.csdn.net/ttotcs/article/details/7476234

(8) Lulu Studio, "ASP. Mvc:action filter (Filtering)", http://www.cnblogs.com/QLeelulu/archive/2008/03/21/1117092.html

Accessories download

(1) memcachedmvcdemo:http://pan.baidu.com/s/1gd3rvkb

Zhou Xurong

Source: http://www.cnblogs.com/edisonchou/

The copyright of this article is owned by the author and the blog Park, welcome reprint, but without the consent of the author must retain this paragraph, and in the article page obvious location to give the original link.

Key/value King memcached: Three, memcached to solve the session of the application of distributed storage scenarios