Sort bigcouch data

Bigcouch Architecture

Chttpd

Encapsulate fabic interfaces and couchdb cluster operations on the HTTP Layer

 

Fabric 

The operation proxy of the couchdb cluster.

It is mainly used to control operations on the couchdb cluster and Erlang layer.

 

Rexi 

Rexi is a special RPC server application in the Operation node cluster of couchdb.

 

Mem3

The couchdb cluster node addition program is used in bigcouch to track two important information of the cluster:

Member node information;

Ing relationship between each database node (partition;

The node information and partition information are tracked in the local node database.

Splitting and merging shards are not yet mature.

 

Bigcouch database parameter Q Parameter

Specify the number of database partitions (Consistent hash partitions) when creating a database ).

Multiple shards may exist on the same node, allowing you to increase the number of nodes in the cluster without the need for re-shard operations.

 

The default value is 8, which can be changed in the configuration file:

The Q field of the Cluster Area in the default. ini file;

 

You can also specify this parameter when creating a database, for example:

Curl-x put http: // 172.16.10.2: 5984/test_db? Q = 4 & n = 3

 

Partition:

When q = 4, the database can be divided into four partitions.

 

 

If there are five nodes, these four partitions will be distributed on five nodes, for example:

 

N Parameter

The number of redundant copies for each shard.

 

The default value is 3, which can be changed in the configuration file:

The N field of the Cluster Area in the default. ini file;

You can also specify this parameter when creating a database, for example:

Curl-x put http: // 172.16.10.2: 5984/test_db? Q = 4 & n = 3

 

Suppose n = 3, and there are five nodes. Each partition will copy three copies and segment them to five nodes according to a certain algorithm, the document in the partition copies three copies and synchronizes them to each backup partition. The example is as follows:

 

 

Delete bigcouch Node

 

Node A to be deleted, file backup Node B

 

Data Migration

 

Copy the. Couch file in the var/lib/shards/directory of node A to the same directory of Node B through SCP and other methods;

 

Data Synchronization

 

After the replication, if the data of node A is updated, perform the synchronization operation on port 5986;

 

Notification bigcouchNew cluster file address

 

Update the database configuration document in the DBS database. Content example:

 

{

"_ Id": "test_db ",

"_ Rev": "2-12788d2d1bb389c3fced1a952c2ea124 ",

 

"By_node ":{

"[Email protected]": [

"Too many -3fffffff ",

"80000000-bfffffff"

],

"[Email protected]": [

"40000000-7fffffff ",

C0000000-ffffffff"

],

"[Email protected]": [

"40000000-7fffffff ",

"C0000000-ffffffff ",

"Too many -3fffffff ",

"80000000-bfffffff"

],

"[Email protected]": [

"40000000-7fffffff ",

C0000000-ffffffff"

],

"[Email protected]": [

"Too many -3fffffff ",

"80000000-bfffffff"

]

},

"By_range ":{

"Commandid -3fffffff ":[

"[Email protected]",

"[Email protected]",

"[Email protected]"

],

"40000000-7fffffff ":[

"[Email protected]",

"[Email protected]",

"[Email protected]"

],

"80000000-bfffffff ":[

"[Email protected]",

"[Email protected]",

"[Email protected]"

],

"C0000000-ffffffff ":[

"[Email protected]",

"[Email protected]",

"[Email protected]"

]

}

}

 

Modify the by_node and by_range fields, and change the information of the parameter to B.

 

DisconnectedAConnection between servers and clusters 

Modify the set-cookie value and restart the bigcouch server.

 

Remove ClusterANode

Access the nodes database through port 5986 of the cluster and delete server a documents

 

DeleteAOn the serverNodesInformation andDBSInformation

 

Clear the files in the shards directory and restart bigcouch of server.

5. the node is unavailable when it is disconnected from the cluster in the node environment.

 

Example of conflict management

Environment Introduction

Three nodes in the Cluster:

A: 172.16.10.2

B: 172.16.10.3

C: 172.16.10.5

 

Database: test_db

Q = 1, n = 3

The database is not partitioned and three copies are backed up.

 

Add data:

{

   "_id": "2809580fa3dc3d8d719c02c229000518",

   "_rev": "1-1bd878534def1c2c9224e7398ab6a987",

   "name": "beforeChange"

}

 

Manufacturing conflict

 

1. Disable server B and server C;

2. Modify the data of server

{

   "_id": "2809580fa3dc3d8d719c02c229000518",

   "_rev": "2-6de54d127839d323b743435f447cb29f",

   "name": "mike"

}

 

 

3. Disable server;

4. Start server B after server a is disabled;

5. Modify server B's data

{

   "_id": "2809580fa3dc3d8d719c02c229000518",

   "_rev": "2-19f995ede86cd2c367b2d90568a6f8c6",

   "name": "MIKE"

} 

 

6. Start server;

Verify data

Data synchronized to server B as

 

2. Data of server C is also synchronized to data of server.

 

Conflict Detection

Conflict check function:

 

Function (DOC ){

If (Doc. _ conflicts ){

Emit (Doc. _ conflicts, null );

}

}

The doc. _ conflicts attribute in the document is an array listing all conflicting revisions.

 

Handle conflicts

Couchdb uses an algorithm to select a successful revision and resolve the conflict. Applications should not trust this algorithm, but should first resolve conflicts.

Automatic conflict handling

Each revision contains a list of previous revisions, and the revision with the longest revision history will eventually become a winning revision; if the two revisions have exactly the same length, then their _ rev values will be compared to the ASCII sequence, and those with larger ASCII values will eventually win. In the above example, 2-6de54d127839d323b743435f447cb29f won, and 2-19f995ede86cd2c107b2d90568a6f8c6 was selected.

Manual conflict handling

First, rewrite the desired value in the target document, and then delete the unwanted revision.

The details are as follows:

1. Read the current document;

2. Read the old (conflicting) version;

3. Merge logic specific to the application domain;

4. Update the document to the new (merged) version;

5. Remove the conflicting document version.

Sort bigcouch data