(Elastic ricds Quick Start Guide) Chapter 1. Quick Start Guide, elastic ricdschapter

(Elastic ricds Quick Start Guide) Chapter 1. Quick Start Guide, elastic ricdschapter

This document is the translation of Quick Start Guide in chapter 1 of tricricds3.6.14. It aims to help readers quickly build a tricricds cluster and popularize some basic concepts and terminologies.

 

This document describes how to synchronize two databases with the same schema between two mongoricds nodes. The following example builds a distribution business model with a central database (we call it root or corp node) and databases of multiple retail stores (we call it client or store node ). For this tutorial, we only have one store node, for example. If you want to, you can expand the example and configure the second store after the tutorial is complete.

In this example, we will install two independent metrics ricds to represent two different servers. One stands for the store server and the other stands for the corp server. Each replica installed on a secondary ricds instance is responsible for a database, that is, a secondary ricds instance is a node in the technical ricds terminology. A replica is used to install a database and represents a node. This configuration is the most common (we can choose to configure an installation replica to take charge of two nodes, which is called multi-homing, will be discussed at the end of the tutorial ). You are likely to run two replica sets on one machine. We will run two replica Sets Installed on two different ports. We will use 8080 to run the corp server, and 9090 to run the store server, as shown in.

The Corp tricricds installation copy will capture data changes in item (item table and item_selling_price table) on the store side, such as itemnumber, description, and store price. Store-side volume ricds captures changes in the sales transaction (sale_transaction table and sale_return_line_item table) data, such as the sales time and product. The price information is only sent to the specific store related to the price, thus reducing the number of pricing information data sent to each store. For example, in other words, the pricing information of store 001 is only sent to the database of store 001 and not to the database of store 002.

This simple configuration always enables the client to initiate communication with the root node, which is a very common configuration. In this configuration, the client regularly pulls data from the database server and attaches it to the root node. The client also pushes the captured changes to the root node.

The summary is enough. Let's start and take the following steps:

1. install and configure two metrics ricds applications

2. Create the metrics ricds configuration file and the database tables used to store corp data and store data.

3. Create retail data in the corp Database

4. Start the mongoricds server and register the store node to the corp node.

5. Send the initial load of data to the store node.

6. Execute a data push and pull operation.

7. Verify the sent and received batch information

Installing metrics ricds

First, we will install two copies of the secondary ricds, and then configure their connection information to the database separately:

1. Download The javasricds Installation File

2. Create two folders to represent two machines. One stores the installation copy of The Corp node and the other stores the installation copy of the store node. For example, you can name these two folders sym-corp and sym-store001, and the following will assume that you are named like this. Decompress the downloaded file to two folders.

3. the attribute file is used to store the minimum configuration information required to start metrics ricds. Copy the corp sample property file to the corp engine folder, and then copy the store property file to the engine folder of the store. If you are using the recommended folder name above, you will make the following copy:

samples/corp-000.properties to sym-corp/symmetric-ds-3.x.x/engines/

And

samples/store-001.properties to sym-store001/symmetric-ds-3.x.x/engines/

4. Browse two property files to view different configurations. For example, a root node is given a group ID with a value of corp, a store node is given a group ID with a value of store, and a root node is given an external ID with a value of 000, the store node is given an external node with the value of 001.

The following are the configuration properties of both configuration files under the engine Directory of the two nodes, specifying how to connect to a specific database server (the following value is just an example ):

# The class name for the JDBC Driver
         db.driver=com.mysql.jdbc.Driver
        
         # The JDBC URL used toconnect to the database
         db.url=jdbc:mysql://localhost/sample
        
         # The user to login as whocan create and update tables
         db.user=symmetric
        
         # The password for the userto login as
         db.password=secret

5. Next, set the following properties in the store-001.properties file to specify the location of the root node:

# The HTTP URL of the root node to contact for registration         registration.url=http://localhost:8080/sync/corp-000

6. Tip. the URL of an engine is in the following general format:

http://{hostname}:{port}/sync/{engine.name}

The engine. name section in the URL comes from the node configuration file.

1.2. Creating and Populating Your Databases

Important

You must first create a database for your root node and client node. The name of the created database must be consistent with the configuration in the preceding configuration file.

 

First, create a database table for the example in the database of the root node, load the sample data, and load the sample configuration:

1. Open a command line prompt and go to the samples folder of the corp installation copy.

2. on the root node, run the following command to create a sample table for item, price, and sale:

../bin/dbimport --engine corp-000 --format XMLcreate_sample.xml

Warning Messages from command lines are safe and can be ignored.

Another brief description of the property file. During the startup phase, metrics ricds looks for one or more attribute files in the engines folder. Because we have already specified a-engine parameter in the command line, mongoricds will look for a specific named file, corp-000.properties. In this example, because there is only one attribute file in the engines folder, metrics ricds loads the configuration in this attribute file by default. Therefore, in this example, the-engine parameter is not specified. With this parameter added, running this example can reduce errors caused by wrong installation of copies. Your ricds instance will complain about the loss of the attribute configuration file you specified.

3. Next, create a database table specified by metrics ricds in the database of the corp node. These tables will contain the configuration information for the synchronization operation. The following command uses the automatically created feature to create all necessary metrics ricds system tables:

../bin/symadmin --engine corp-000create-sym-tables

4. In the last step, perform the following operations to load the sample product and transaction data and metrics ricds configuration information to the database on the root node:

../bin/dbimport --engine corp-000insert_sample.sql

Important

For MySQL, run the insert_sample_mysql. SQL file. For case-sensitive tables and columns, MySQL uses quotation marks instead of double quotation marks.

We have now mounted the corp Database Table and then filled in our metrics ricds configuration information and sample data. Next, we will create a sample database table in the database of the store node to prepare to receive data.

5. Open a command prompt and enter the samples folder of the replica installation on the storage node.

6. Execute the following command to create an empty sample table in the database of the store node:

../bin/dbimport --engine store-001 --format XMLcreate_sample.xml

Warning Messages from command lines are safe and can be ignored. You can view the create_sample.xml file to see what it has.

Log on to the database and check the tables in the two databases:

1. Find the item table to be synchronized from the corp node to the store node: item and item_selling_price.

2. Find the sale table from the store node to the corp node: sale_transaction and sale_return_line_item.

3. Find the system table of metrics ricds. These tables are prefixed with sym _, such as sym_channel, sym_trigger, sym_router, and sym_trigger_router.

4. Verify that the item table of the corp node has sample data

1.3. Starting metrics ricds

The database creation and configuration in this tutorial are now complete. It's time to start using metrics ricds. Now we start two metrics ricds nodes and observe their log output.

1. Open two command line prompts and go to the samples directory under the root directory of the two installation copies.

2. In the samples directory of corp, execute the following command to start corp startup ricds:

../bin/sym --engine corp-000 --port 8080

At the first startup, the corp node will create all the triggers in the sample configuration file. Then listen for synchronization and registration requests for the corp-000 engine at port 8080.

3. In the samples directory of store001, run the following command to start metrics ricds:

../bin/sym --engine store-001 --port 9090

When this command is used to start the server of the store node for the first time, the system table of your ricds instance is created automatically. Then try to register with the corp node continuously (the address of the corp node can be known through the registration URL configured earlier ). Because the registration has not been enabled, the store node will receive an authentication failure (403 HTTP response ). We will discuss it in the Registration Section below.

1.4. Registering a Node

When an unregistered node starts, it will try to register it to the node specified by the registration URL (in almost all cases, it is the root node ). By allowing Registration and returning configuration information for registered nodes, the Registration node centrally manages other nodes on the network. In this tutorial, the Registration node is the root node and corp node. This node is also involved in synchronization with other nodes.

Therefore, next, we need to enable registration for the store node so that the store node can receive data from the initial load and then receive data from corp, you can also send data to the corp node. There are several ways to complete this task. We will use the metrics ricds management function to execute a command on the corp node (because this node is responsible for registration management ).

1. The mongoricds application of the corp node and store node has been started. Open a new command line prompt to go to the samples directory under the root directory of the installation copy of The corp node.

Run the following command to enable the registration function of the store node:

../bin/symadmin--engine corp-000 open-registration store 001

Now, the registration function of corp has been enabled for nodes with group as store and external id as 001. The Group and external id information corresponds to the configuration in the store-001.properties profile in the store node. In metrics ricds, each Node is assigned to a Node Group, and an external ID is assigned to the application. In this tutorial, we name the group representing the store as the store, and then we use the number marker starting with 001 as the external ID (000 representing the corp node ). More information about node groups is described in the next section.

2. Check the log output of the store node to see if it has been successfully registered with the corp node. Store is configured to try to register once at random intervals, up to 1 minute. Once the registration is successful, the corp and store nodes start synchronization.

1.5. Sending an Initial Load

Next, we will send an initial load of data to the store node, and then use the node management function on the corp node.

1. Open a command line prompt and go to the samples folder in the root directory of the corp installation copy. (Note: Most system commands are usually directly issued on the corp server. For example, all configuration files are input in corp and then synchronized to any client node .)

2. Run the following command to send an initial load of data to the store node:

../bin/symadmin --engine corp-000 reload-node 001

After using this command, the corp node queues data requests from each store node. The initial load of data includes = data in each table configured for synchronization.

3. Observe the log output of the two nodes and view the data transmission. Store is configured to pull data from the corp node every minute.

1.6 Pulling Data

Next, we will make some changes to the item data of the central database (we will add a new item), and then observe that the data is pulled to the store node.

1. Open an interactive SQL Session with the Corp Database

2. Add a new product in store001 and store002 (in this example, only store001 is used:

insert into "item"("item_id", "name") values (110000055, 'Soft Drink');
insert into "item_selling_price"("item_id", "store_id", "price") values(110000055, '001', 0.65); insert into "item_selling_price" ("item_id","store_id", "price") values (110000055, '002', 1.00);

Once the statement is submitted, the data changes will be captured by mongoricds, and data will be pulled by store nodes waiting in queue.

4. Observe the log output of the two nodes and view the data transmission status. Store is configured to pull data from the corp node every minute.

5. In this tutorial, item_selling_price is configured as a column-matching Router. Changes to specific pricing data will only be sent to nodes with store_id and external ID matching. Execute SQL statements in the database of the store node to verify whether new data has arrived. In this example, the first record is only sent to store 001.

1.7. Pushing Data

Now, we will counterfeit a transaction and then observe how the transaction information is pushed to the central node. (Here we can find that data pulling and pushing are all tasks of slave nodes, and the central node is only responsible for management)

1. Open an SQL session for the database on the store Node

2. Add a new transaction information to the database of the store node.

insert into "sale_transaction"("tran_id", "store_id", "workstation","day", "seq") values (1000, '001', '3', '2007-11-01', 100);
insert into "sale_return_line_item"("tran_id", "item_id", "price","quantity") values (1000, 110000055, 0.65, 1);

Once the statement is submitted, data modifications are captured and queued for the store node to be pushed to the central node.

3. Observe the log output of the two nodes and view the data transmission status. The Store node is configured to push data to the corp node once every minute.

1.8 Verifying Outgoing Batches

We have already demonstrated how to push and pull data. Next we will describe how to obtain the data's batch and sent information. A batch is used to track and send one or more data changes to a given node. Create a batch for the node where the data is located. The node that receives the data and then responds.

In addition, tables are grouped into different data channels in analyticdb. To allow different data types to be synchronized when some data types are incorrect. For example, if an error occurs in the batch of the given channel, the batch will be tried in every synchronization operation of the channel until the batch does not have an error. Only when this batch does not have an error will other batchs sent to this channel be sent. In this way, data changes in the data source can be sent to the destination in the same order, and these changes are applied in the same order in the destination. However, no wrong batch on one channel will not be blocked by another channel with the wrong batch. In this way, data changes on one channel are not blocked by errors on the other channel.

You can view the sent batch information by following these steps:

1. Open a session connected to the database server of the corp or store Node

2. Confirm the captured data changes:

select * from sym_data order by data_id desc;

Each row of Data in this table represents a row of changed data. Data_id increases sequentially, so the latest data id should be associated with your data insert SQL. Event_type indicates I, U indicates update, and D indicates delete. For insert and update, the captured data is in the row_data column. For update and deletion, the value of the primary key is in the pk_data column.

3. Use the data_id obtained in the previous step to confirm that the data changes in a batch:

select * from sym_data_event where data_id = ?;

Batch is created based on necessary target node information and is called a Route Job. As part of the Route Job, the data change is assigned to a batch using the batch_id, which is used for data tracing and synchronization. The connection information between Batch and data is connected through the sym_data_event table.

4. Use the batch_id obtained in the previous step to send the data change to the destination and receive a response:

select * from sym_outgoing_batch where batch_id =?;

The initial Batch has a NE (new) status when it is just created and not yet sent to a node. Once a node response is received, the batch status changes to OK when the node is sent successfully. If the node fails to be sent, the status changes to ER. If the batch fails, the error_flag of the batch will be set to 1, and the failed batch may succeed when it tries again. In this case, the status needs to be changed.

Understanding the three tables and the fourth table discussed in the next section are the key to diagnosing possible synchronization problems. When using metrics ricds, whether in an experimental or production environment, you can spend some time monitoring these tables to better understand how metrics ricds works.

1.9. Verifying Incoming Batches

The node receiving data maintains the batch information returned by the node and the statistics related to the loaded data. Duplicate batchs are ignored by default, but can be modifiedincoming.batches.skip.duplicatesChange this behavior.

Observe the received batch information through the following steps:

1. Open an SQL session that interacts with the database of the corp or store Node

2. Use the batch_id obtained from the previous part to confirm the batch_id information for receiving and responding:

select * from sym_incoming_batch where batch_id =?;

A batch represents a set of data changes loaded by nodes. The table records the nodes that created and sent the batch. The status of the batch is OK, and the status of the batch is ER.

1.10. Multi-Homing

This is the end of our Quick Start Guide. We have successfully created two synchronization nodes and executed the synchronization operation in the two databases. Now let's go back and discuss an issue discussed in the first step of the Guide. That is, in the Guide, let you install the two replica copies.

In the preceding example, an attribute file is replaced in the engines directory of each javasricds installation copy. When metrics ricds is started, metrics ricds is initialized, and a metrics ricds engine is created based on the provided configuration file (again, an engine is a metrics ricds node responsible for a specific database ).

In fact, a javasricds application can start multiple engines at the same time. When javasricds is started, it searches for a file ending with '. properties' in the engines directory. It will start a metrics ricds engine for every configuration file found. The-engine parameter in the command line prompt will overwrite the above operation, so that javasricds will start only one engine specified in the command line. In the case that multiple engines are started when a replica is installed in one javasricds, it is called 'multi-homing '.

For the above example, if we want to change to the multi-homing architecture, we only need to do the following steps:

1. Install a secondary ricds copy instead of the two above. There is no need for a directory to represent two machines.

2. Copy two configuration files from the samples directory of the installation copy to the engines directory.

3. All the commands run above are run in the samples directory of the installation copy.

4. When you start javasricds, you do not need to specify a specific engine. This will start two engines. This command is still run from the sampls directory, as follows:

../bin/sym --port 8080

Note that we do not use port 9090 when using the above command. Metrics ricds now monitors data exchange between corp and store engine on port 8080.

5. In addition to the server startup commands, you must use the-engine parameter to specify the engine for execution of all other commands. This is because you need to process commands for specific nodes (engines), such as opening registration and initiating initial loads from corp to store.