Use Tokyo tyrant to build a Distributed Key-value persistent storage system compatible with the memcached protocol and supports failover and high concurrency.

Tokyo Cabinet
It is a dBm database developed by the Japanese pinglin yixiong. It reads and writes 1 million data records in hash mode in 0.643 seconds and reads 1 million data records in 0.773 seconds, it is several times that of DBM such as Berkeley dB.

　　

Tokyo tyrant
Is the network interface of the Tokyo cabinet database developed by the same author. It is compatible with memcached and can exchange data over HTTP.

Tokyo tyrant and Tokyo cabinet constitute a distributed persistent storage system that supports high concurrency. For any original memcached client, you can regard Tokyo tyrant as a memcached. However, its data can be stored permanently. This is similar to the memcachedb of Sina.
Same nature.

　　Compared with memcachedb, Tokyo tyrant has the following advantages:

1. failover: Tokyo tyrant supports dual-host and dual-host mode, and both master and slave databases can be read and written. memcachedb currently supports read and write separation in a way similar to that of MySQL master and slave databases, supports the "primary server read/write, secondary server read-only" mode.

　　

　
Use $ memcache-> addserver instead of $ memcache-> connect to Tokyo.
Tyrant server because when the memcache client uses the addserver server pool
Current_server_num "hash algorithm hash keys to different servers, PHP, C, and Python
The client is such an algorithm. The addserver of the memcache client has a Failover mechanism. When the addserver has two memcached instances
If one of the servers is down, current_server_num will change from 2 to 1.

REFERENCE The memcached official website and the PHP manual in two paragraphs:

Reference http://www.danga.com/memcached/
If a host goes down, the API re-maps that dead host's requests onto the servers that are available.

Http://cn.php.net/manual/zh/function.Memcache-addServer.php

Failover
May occur at any stage in any of the methods, as long as other servers
Are available the request the user won't notice. Any kind of socket or
Memcached server level errors (memory T out-of-memory) may trigger
Failover. Normal client errors such as adding an existing key will not
Trigger a failover.

 

2. Small log file size: the log file used by Tokyo tyrant for Master/Slave synchronization is relatively small, which is about 1.3 times the size of the database file, while the synchronization log file of memcachedb is very large. If it is not cleaned regularly, it is easy to fill up the disk.

3. Outstanding Performance in massive data volumes:


　　

However, Tokyo tyrant also has its disadvantages: In a 32-bit operating system, a single file in the Tokyo cabinet database, which is used as the back-end storage of Tokyo tyrant, cannot exceed 2 GB, the 64-bit operating system is not subject to this restriction.
Therefore, if you use Tokyo tyrant, we recommend that you install and run it on a 64-bit CPU and operating system.

　　I. Installation

1. Compile and install the tokyocabinet database.

Wget http://tokyocabinet.sourceforge.net/tokyocabinet-1.4.28.tar.gz

Tar zxvf tokyocabinet-1.4.28.tar.gz
CD tokyocabinet-1.4.28/
./Configure
Make
Make install
CD ../

2. Compile and install tokyotyrant.

Wget http://tokyocabinet.sourceforge.net/tyrantpkg/tokyotyrant-1.1.29.tar.gz

Tar zxvf tokyotyrant-1.1.29.tar.gz
CD tokyotyrant-1.1.29/
./Configure
Make
Make install
CD ../

 

　　Ii. Configuration

　　1. Create a tokyotyrant data file storage directory

Mkdir-P/ttserver/

　　2. Start the main process (ttserver) of tokyotyrant)

　　(1) standalone Mode

Ulimit-shn 51200
Ttserver
-Host 127.0.0.1-port 11211-thnum 8-DMN-PID/ttserver. PID
-Log/ttserver. Log-le-ulog/ttserver/-ulim 128 M-Sid 1-RTS
/Ttserver. RTS/ttserver/database. Tch

　　(2) Dual-Host Mode

Server 192.168.1.91:

Ulimit-shn 51200
Ttserver
-Host 192.168.1.91-port 11211-thnum 8-DMN-PID
/Ttserver. PID-log/ttserver. Log-le-ulog/ttserver/
-Ulim 128 M-Sid 91-mhost 192.168.1.92-mport 11211-RTS
/Ttserver. RTS/ttserver/database. Tch

Server 192.168.1.92:

Ulimit-shn 51200
Ttserver
-Host 192.168.1.92-port 11211-thnum 8-DMN-PID
/Ttserver. PID-log/ttserver. Log-le-ulog/ttserver/
-Ulim 128 M-Sid 92-mhost 192.168.1.91-mport 11211-RTS
/Ttserver. RTS/ttserver/database. Tch

　　(3) parameter description

　
Ttserver [-host name] [-port num] [-thnum num] [-Tout num] [-DMN]
[-Pid path] [-Log Path] [-LD |-le] [-ulog path] [-ulim num] [-UAS] [-Sid
Num] [-mhost name] [-mport num] [-RTS path] [dbname]

-Host Name: Specifies the domain name or IP address of the server to be bound. All IP addresses on this server are bound by default.
-Port num: Specifies the port number to be bound. The default port is 1978.
-Thnum num: specifies the number of threads. The default value is 8 threads.
-Tout num: Specifies the timeout time (in seconds) for each session ). It never times out by default.
-DMN: run as a daemon.
-Pid path: output process ID to the specified file (the file name is specified here ).
-Log Path: Output log information to the specified file (the file name is specified here ).
-Ld: the debug debugging information is also recorded in the log file.
-Le: only error information is recorded in the log file.
-Ulog path: Specifies the path for storing synchronization log files (the directory name is specified here ).
-Ulim num: Specifies the size of each synchronization log file (for example, 128 MB ).
-UAS: Update logs using asynchronous Io records (this option reduces disk Io consumption, but the data is first stored in the memory and will not be written to the disk immediately, if the restart server or ttserver process is killed, some data will be lost. It is generally not recommended ).
-Sid num: Specifies the server ID number (each ttserver needs a different ID number when active/standby mode is used)
-Mhost name: Specifies the domain name or IP address of the master server in master-slave synchronization mode.
-Mport num: Specifies the port number of the master server in master-slave synchronization mode.
-RTS path: Specifies the file name used to store the synchronization timestamp.

If a hash database is used, you can specify the parameter "# bnum = xxx" to improve performance. It can specify the number of buckets. For example, if "# bnum = 1000000" is specified, the latest and most popular 1 million records can be cached in the memory:

Ttserver
-Host 127.0.0.1-port 11211-thnum 8-DMN-PID/ttserver. PID
-Log/ttserver. Log-le-ulog/ttserver/-ulim 128 M-Sid 1-RTS
/Ttserver. RTS/ttserver/database. Tch # bnum = 1000000

If a large number of clients access ttserver, ensure that the file descriptor is sufficient. The default file descriptor of many servers is 1024. You can use the ulimit command to increase this value before starting ttserver. For example:

Ulimit-shn 51200

　　3. Stop tokyotyrant (ttserver)

PS-Ef | grep ttserver

Locate the process ID of the ttserver and kill it. For example:

Kill-term 2159

 

　　Iii. Call

　　1. Any memcached client can directly call tokyotyrant.

　　2. You can also use HTTP. The following uses the Linux curl command as an example to describe how to operate tokyotyrant:

(1) write data and write the data "value" into the "key:

Curl-x put http: /127.0.0.1: 11211/Key
-D "value"

(2) read data from the "key:

Curl http: // 127.0.0.1: 11211/Key

(3) Delete data and delete "key ":

Curl-x Delete http: /127.0.0.1: 11211/Key