1. Why use Redis? 2016-05-0815:54:43
- In the DWCS project, it is necessary to use the Redis database as a key implementation of data exchange between multiple worker nodes and the master node, because Redis's <key,value> storage model can be well implemented with the MapReduce model- Mincemeat.py is a good match for data transmission and communication of nodes.
- and Redis data is written in memory to read, fast, convenient is to save data read, especially in the case of large amounts of data, a good way.
- Redis master and slave as a master-slave configuration, master-slave replication mechanism can make master as the data "publisher", Multiple Slave (data "subscribers") to update data, as long as the slave node Request for Data Sync Sync requests from master, the server on themaster node opens a daemon that backs up the master Redis database as a file (backups), and if the server backup is not complete, Client clients with Redis on the master node connect their server and read and write operations, at which point Master does not block client reads and writes, and two processes (client and backup processes) run synchronously. <master non-blocking > during backup
- Master can connect multiple slave, but read and write data is detached, master copies the database "write", Slave is only "read" permission. (The read-write operation object here is the Redis database on the master node), if the data is written on the slave, then the data exists only in the slave local database, i.e. < Set data on master, get to data on slave, set data on slave, and get less than > on master. It can be said that this type of Master--slave deployment is a one-way tree-like transmission diagram of data . But since the DWCS project data is neededMany-to-oneData transfer-all worker nodes collect the parsed URLs for processing by the master node, and Redis isOne -to-many "master--slaves" mode, and <One slave cannot connect multiple master>. So use the "deploy a Redis server on the master node, deploy the Redis client on the worker node, and connect to the master Redis server forMulti-Client-->masterclient-side Farm connection update database. Do this:
- Advantage: It is quick and easy to update the client side data of each worker node on the same Redis server.
- Cons: Unsafe, because Redis is in memory, fast operation, easy access to Redis Server password brute force, multi-client saving data is prone to misoperation or the data is overwritten by other clients writing the same key value.
- FIX: Set a password? Encrypt data before data is stored? To determine if the key value repeats before the data is stored, set the auto-grow key value to avoid duplication?
Configuration on the 2.Redis Server "Master Node"
- Currently, Windows-enabled Redis is an unofficial version of Microsoft Redis (msopentech / redis), and after the download is unpacked, the following files are available:
- The redis.windows.conf is the configuration file that needs to be changed. After opening:
- Port 6379 indicates that its default port is 6379
- Several commonly used configuration changes are: port, logfile, slaveof (configured for master-slave replication), save, Slave-read-only, Requirepass (configured for master-slave replication), Masterauth (configured for master-slave replication), ...
- After modifying the configuration file, save, DOS switch to Redis directory, start Redis Server:
- In Dcws, you only need to use the master node's Redis Server, so the configuration I only changed (varies from person to person), the other is the default value:
Requirepass *******
On the "Worker node" of the 3.Redis client configuration:
- Enter DOS on the worker node's machine, switch to Redis's directory to run the client, and connect to the server on the remote Master node server:
- -H followed by the master ip,-p after the port,-a followed by the connection password (that is, Requirepass set password)
4. After successful connection:
Server side:
Client side:
Redis 3.0.501 Learning Use (1)
