Solrcloud 4.9 distributed cluster deployment and precautions

Environment Construction

I. zookeeper

Reference: http://blog.chinaunix.net/uid-25135004-id-4214399.html

There are 4 machines 10.14.2.201 10.14.2.202 10.14.2.203 10.14.2.204 installing zookeeper cluster on all machines 1. Download installation package unzip tar xvf zookeeper-3.4.5.tar.gz-C/export/CD/export/ln-s zookeeper-3.4.5 zookeeper2, Set environment Variable Vim/etc/profile # Set zookeeper enviromentexport zookeeper_home =/export/zookeeperexport Path = $ path: $ zookeeper_home/bin3. Load the environment variable source/etc/profile4. Configure zookeepercd/export/zookeeper/confcp zoo_sample.cfg Zoo. using Vim Zoo. CFG ticktime = 2000 initlimit = 10 synclimit = 5 datadir =/export/zookeeper/datadatalogdir =/export/zookeeper/logclientport = 2181server. 1 = 10.14.2.201: 2888: 3888server. 2 = 10.14.2.202: 2888: 3888server. 3 = 10.14.2.203: 2888: 3888server. 4 = 10.14.2.204: 2888: 38885. datadir and log directory datalogdirmkdir-P/export/zookeeper/datamkdir-P/export/zookeeper/log6. On 10.14.2.201, create the myid file Echo 1>/export/ zookeeper/data/myid the numbers in myid and zoo. "server. number = IP or domain name: port1: port2 "corresponds to the server in the configuration file 10.14.2.201. the number is 1, so the number of myid should be 17. On 10.14.2.202, create the myid file Echo 2>/export/zookeeper/data/myid 8 in the datadir directory, and on 10.14.2.203 in the datadir directory create the myid file Echo 3>/export/zookeeper/data/myid 9. Create the myid file Echo 4>/export/zookeeper/data/myid in the datadir directory on 10.14.2.204. execute 10. Modify the permission chown hadoop. hadoop-r/export/zookeeper/chown hadoop. hadoop-r/export/zookeeper11, start zookeeper/export/zookeeper/bin/zkserver. sh start12. Check the zookeeper status. 1> 10.14.2.201 #/export/zookeeper/bin/zkserver. sh status JMX enabled by default using config:/export/zookeeper/bin /.. /CONF/Zoo. CFG mode: follower 2> 10.14.2.202 #/export/zookeeper/bin/zkserver. sh status JMX enabled by default using config:/export/zookeeper/bin /.. /CONF/Zoo. CFG mode: follower 3> 10.14.2.203 #/export/zookeeper/bin/zkserver. sh status JMX enabled by default using config:/export/zookeeper/bin /.. /CONF/Zoo. CFG mode: Leader 4> 10.14.2.204 #/export/zookeeper/bin/zkserver. sh status JMX enabled by default using config:/export/zookeeper/bin /.. /CONF/Zoo. CFG mode: follower 13. zookeeper has been installed and started.

　　

Ii. solrcloud

Reference: http://lucien-zzy.iteye.com/blog/2002463

1. install Tomcat and configure the port. Deploy SOLR. War and copy the jar under SOLR \ example \ Lib \ ext to SOLR \ WEB-INF \ Lib.

2. Create

/Home/SOLR/solrcloud

/Home/SOLR/solrcloud/config-files (place all files under SOLR \ example \ SOLR \ collection1 \ conf)

/Home/SOLR/solrcloud/SOLR-lib (place all jar packages under SOLR \ WEB-INF \ Lib)

3. Configure the Chinese word Divider

4. Create a SOLR data directory

/Home/SOLR-cores

Generate SOLR. XML as follows:

<?xml version="1.0" encoding="UTF-8" ?><solr persistent="true"><logging enabled="true"><watcher size="100" threshold="INFO" /></logging><cores defaultCoreName="collection1" adminPath="/admin/cores" host="${host:}" hostPort="8080" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}"></cores></solr>

5. Create

Tomcat/CONF/Catalina

Tomcat/CONF/Catalina/localhost

And create SOLR. xml

<?xml version="1.0" encoding="UTF-8"?><Context docBase="/soft/apache-tomcat-7.0.37/webapps/solr" debug="0" crossContext="true"><Environment name="solr/home" type="java.lang.String" value="/home/solr/solr-cores" override="true"/></Context>

Docbase configuration based on actual conditions;

SOLR/home is the directory created earlier:/home/SOLR-cores

6. modify tomcat/bin/cataina. sh file, add java_opts = "-dzkhost = Master: 2181, slave1: 2181, slave2: 2181" at the top, or directly use an IP address.

Publish the preceding configurations to other machines in the cluster.

7. solrcloud uses the zookeeper cluster to ensure that configuration file changes are synchronized to each node in a timely manner. Therefore, you need to upload the configuration file to the zookeeper cluster. Perform the following operations:

java -classpath .:/home/solr/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.91.128:2181,192.168.91.129:2181,192.168.91.130:2181 -confdir /home/solr/solrcloud/config-files/ -confname myconf  

-Cmd upconfig: Upload the configuration file

-Confdir: directory of the configuration file

-Confname: Specifies the corresponding name.

java -classpath .:/home/solr/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd linkconfig -collection collection1 -confname myconf -zkhost 192.168.91.128:2181,192.168.91.129:2181,192.168.91.130:2181

-Cmd linkconfig: bind the configuration file to the specified collection.

-Collection

-Confname

8. Check the configuration information of zookeeper.

Zkcli. Sh-Server IP: Port

Ls/configs/myconf

Ls/live_nodes

9. start Tomcat and create collection

curl ‘http://192.168.91.128:8080/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=1‘

Action: Create

Name: Collection name

Numshards: Number of Parts

Replicationfactor: Number of replica sets

You can also use ZK to query collection conditions: ls/collections/mycollection

 

-------------------------------

1. ZK CLI

Zookeeper's command line utility (CLI ).

zkcliParameters

Short	Parameter usage	Meaning
	-cmd <arg>	CLI command to be executed:bootstrap,upconfig,downconfig,linkconfig,makepath,get,getfile,put,putfile,listOrclear. This parameter isMandatory
-z	-zkhost <locations>	Zookeeper host address. This parameter isMandatoryFor all CLI commands.
-c	-collection <name>	Forlinkconfig: Name of the collection.
-d	-confdir <path>	Forupconfig: A Directory of configuration files.
-h	-help	Display help text.
-n	-confname <arg>	Forupconfig,linkconfig: Name of the configuration set.
-r	-runzk <port>	Run zookeeper internally by passing the SOLR run port; only for clusters on one machine.
-s	-solrhome <path>	ForbootstrapOr when using-runzk:MandatorySolrhome location.

Zookeeper CLI examples

Below are some examples of using the zkcli CLI:
Uploading a configuration directory

java -classpath example/solr-webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983-confdir example/solr/collection1/conf -confname conf1 -solrhome example/solr

Put arbitrary data into a new ZK File

java -classpath example/solr-webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -zkhost 127.0.0.1:9983 -put /data.txt ‘some data‘

Put a local file into a new ZK File

java -classpath example/solr-webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -zkhost 127.0.0.1:9983 -putfile /data.txt /some/local/file.txt

Linking a collection to a configuration set

java -classpath example/solr-webapp/webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -cmd linkconfig -zkhost 127.0.0.1:9983-collection collection1 -confname conf1 -solrhome example/solr

Bootstrapping all the configuration directories in SOLR. xml

java -classpath example/solr-webapp/webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -cmd bootstrap -zkhost 127.0.0.1:9983-solrhome example/solr

Scripts

There are scripts in example/cloud-scripts that handle the classpath and class name for you if you are using SOLR out of the box with jetty. commands then become:

sh zkcli.sh -cmd linkconfig -zkhost 127.0.0.1:9983-collection collection1 -confname conf1 -solrhome example/solr

 

 

2. managing collections via the collections API

Create ---

http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=4

Name: Collection name

Numshards: specify the number of slices)

Replicationfactor: Number of copies

Maxshardspernode: The default value is 1. Note the following three values: numshards, replicationfactor, and livesolrnode. A normal solrcloud cluster does not allow deploying multiple replic of the same shard on the same livesolrnode, therefore, when maxshardspernode is set to 1, numshards * replicationfactor> livesolrnode reports an error. Therefore, the following conditions are met:
Numshards * replicationfactor <livesolrnode * maxshardspernode

Createnodeset:

Collection. configname: Specify the config used by the collection. This config must exist in zk.

Delete ---

http://localhost:8983/solr/admin/collections?action=DELETE&name=mycollection

Reload ---

http://localhost:8983/solr/admin/collections?action=RELOAD&name=mycollection

Split shard ---

http://localhost:8983/solr/admin/collections?action=SPLITSHARD&collection=<collection_name>&shard=shardId

 

3. Creating cores via coreadmin

curl ‘http://localhost:8983/solr/admin/cores?action=CREATE&name=mycore&collection=collection1&shard=shard2‘

 

 

-------------------------------

Q1: Multiple schemas in the same solrcloud?

java -classpath .:/home/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 10.200.51.190:2224,10.200.51.192:2224,10.200.51.194:2224 -confdir /home/solrcloud/fox-config-files/ -confname foxconf

Specify the configuration file and upload it to ZK.

curl ‘http://10.200.51.194:2223/solr/admin/collections?action=CREATE&name=fox4&numShards=1&replicationFactor=5&collection.configName=foxconf‘

Specify collection. configname for the created collection.