Elasticsearch Distributed Services with Java

Because of the previous blog elasticsearch5.x on Linux distributed Installation (multi-node) installed Elasticsearch service is relatively new, so the elasticsearch will be made into a single service operation, need to use when routing forwarding request this service.

Because the Elasticsearch version is newer, the mileage version of spring data is used, and the specific pom.xml configuration and current project can be downloaded to my GitHub home page (https://github.com/lh2420124680/ Elasticserver).

The bottom of the Elasticsearch is Lucene, and its search speed is very fast.

In the actual application we have two kinds of deployment scenarios, respectively, dealing with different business scenarios:

① Real-time Save data into index library

When the user saves the data, save a copy to the database, when the database is saved successfully, the saved data is sent to the message team (the message middleware in this project is ACTIVEMQ), and then when the message Queue supervisor hears that a message has been sent over, it receives the message, which is to accept the saved data. The message listener then executes the Elasticsearch method to save the data that is saved in the database to the corresponding index library. This approach is less intrusive to the original code because the Elasticsearch service is a separate project.

Process: The data to be inserted at the front end is sent to the background-> insert the database successfully-> the front end to send the message queue-> send the data into the message Queue-> message listener receives the message and inserts the data into the corresponding index library.

This method is suitable for real-time query and simple structure data.

② synchronizes data from the database to the index library on a daily basis.

This method needs to use some of elastic's plug-ins, you can refer to the Elasticsearch in this blog and Oracle database data synchronization

This method is suitable for querying the data which is not real-time but complex in structure.

This blog is primarily based on the first method, which reconstructs the elasticsearchtemplate in spring data because elasticsearchtemplate relies on entities for the return value type. and the entity needs to use the @document annotation, so the elasticsearchtemplate source code is reconstructed in this project, the sending parameter is changed to the string of the JSON object, and the result is the type of list<map>. This will not be used every time new business to use the Elasticsearch to add the current business entity to modify the code, refactoring only once deployed, can be followed by multiple business scenarios.

Here are a few points to watch out for, and a friend who needs in-depth research can go to my githib to download the project (https://github.com/lh2420124680/ElasticServer).

Elasticsearch Main configuration file: Com-zlb-elastic.xml

Instead of using elasticsearchtemplate, I use the custom elasticobjecttemplate class, let this class inherit the Elasticsearchtemplate class, and then rewrite the parent class.

The queryForList method in the Elasticsearchtemplate class is then modified. (only two parameters in the parent class are query objects and entity objects)

This project v1.0 only a single method of inserting into the index library and then slowly refactoring all the main methods of Elasticsearchtemplate.

Here we need to convert the returned JSON object to an object array because the parent class will use the

In addition, we need to refactor a class resultsmapper for the return result object of the index.

Resultsmapper is the processing class used to return the result object in the Elasticsearchtemplate class, and his return list is the transmitted entity generic, for example, when I query the transmission later for the man class, then he returns is the list<man> Because this refactoring removes the entity and uses a map instead of the entity model, so we may need to rewrite the Resultsmapper class, but Resultsmapper is an interface, We find that the Defaultresultmapper class implements this Resultsmapper interface, and all of us rewrite a defaultresultobjectmapper class to inherit Defaultresultmapper this class, The Mapentity method is then mainly rewritten.

The above is the source of the rewrite some of the main points, although not very clear, but read several times Elasticsearchtemplate's source will understand.

The message listener, when the data is heard, invokes the custom Elasticobjecttemplate method to insert into the index library.

Package COM.ZJY.JMS;
Import Com.alibaba.fastjson.JSON;
Import Com.alibaba.fastjson.JSONObject;
Import Com.zjy.biz.ElasticBiz;
Import Com.zjy.helper.ElasticObjectTemplate;
Import Com.zjy.helper.EscapeHelper;

Import Com.zjy.helper.StringUtils;
Import java.io.UnsupportedEncodingException;

Import Java.util.Map;
Import javax.jms.JMSException;
Import Javax.jms.Message;
Import Javax.jms.MessageListener;

Import Javax.jms.TextMessage;
Import org.springframework.beans.factory.annotation.Autowired;
Import Org.springframework.data.elasticsearch.core.query.IndexQuery;
Import Org.springframework.data.elasticsearch.core.query.IndexQueryBuilder;
Import Org.springframework.jms.annotation.JmsListener;

Import org.springframework.stereotype.Component; @Component public class Elasticreceiver implements MessageListener {private Elasticobjecttemplate Elasticobjecttemplat

	E
	Public Elasticobjecttemplate Getelasticobjecttemplate () {return elasticobjecttemplate; } public void SetelasticobjecttempLate (Elasticobjecttemplate elasticobjecttemplate) {this.elasticobjecttemplate = elasticobjecttemplate;
			public void OnMessage (message message) {try {String msg = (textmessage) message). GetText ();
			
			String data = Java.net.URLDecoder.decode (msg, "UTF-8");
			
			String unescape = escapehelper.unescape (data);
			Jsonobject parseobject = Json.parseobject (unescape);
			String dataid = Stringutils.getguid ();
			String IndexName = parseobject.getstring ("IndexName");
			String typeName = parseobject.getstring ("TypeName");
			
			String entity = parseobject.getstring ("entity"); Indexquery indexquery = new Indexquerybuilder (). Withid (Dataid) withindexname (indexname). Withtype (TypeName).  
			Withobject (Entity). build ();
			String index = Elasticobjecttemplate.index (indexquery);
		System.out.println ("IndexName:" +indexname + index);
		catch (JMSException e) {e.printstacktrace ();
		catch (Unsupportedencodingexception e) {e.printstacktrace (); }
	}

}

Elasticsearch query, removed service interface.

Package com.zjy.service;
Import java.util.List;

Import Java.util.Map;

Import Javax.servlet.http.HttpServletRequest;
Import Org.elasticsearch.index.query.BoolQueryBuilder;
Import Org.elasticsearch.index.query.QueryBuilders;
Import Org.elasticsearch.search.sort.SortBuilder;
Import Org.elasticsearch.search.sort.SortBuilders;
Import Org.elasticsearch.search.sort.SortOrder;
Import org.springframework.beans.factory.annotation.Autowired;
Import Org.springframework.data.domain.PageRequest;
Import Org.springframework.data.elasticsearch.core.query.IndexQuery;
Import Org.springframework.data.elasticsearch.core.query.IndexQueryBuilder;
Import Org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
Import Org.springframework.data.elasticsearch.core.query.SearchQuery;
Import org.springframework.web.bind.annotation.RequestMapping;
Import Org.springframework.web.bind.annotation.RequestMethod;

Import Org.springframework.web.bind.annotation.RestController; Import Com.alibaba.fastJson.
JSON;
Import Com.zjy.helper.DataConvertHelper;
Import Com.zjy.helper.ElasticObjectTemplate;
Import Com.zjy.helper.EscapeHelper;
Import Com.zjy.helper.ListResult;

Import Com.zjy.helper.StringUtils; @RestController @RequestMapping (value = "/elasticservice") public class Elasticservice {@Autowired private elasticobje
	
	Cttemplate elasticobjecttemplate; /** * @param request * @return/@RequestMapping (value = "/queryresource.ashx", method = Requestmethod.get) Publi C listresult<map<string, object>> Queryresource (HttpServletRequest request) {map<string, Object>
		where = Dataconverthelper.getrequestparams (request);
		String IndexName = Where.get ("IndexName"). ToString ();
		
		String typeName = Where.get ("TypeName"). ToString ();
		Create search criteria object (conditional parameter name + "_JZ" for exact match, parameter name + "_MH" for Fuzzy match) Boolquerybuilder boolquery = Querybuilders.boolquery ();
		String key = "";
		String value = "";
		String mode = "";
		String Reallyparam = ""; For (map.entry<string, OBJEct> Map:where.entrySet ()) {key = Map.getkey ();
			Value = Map.getvalue (). toString ();
			mode = Key.substring (Key.length ()-3);
			Reallyparam = key.substring (0, Key.length ()-3);
			if ("_jz". Equals (Mode)) {//The condition is an accurate query Boolquery.must (querybuilders.termquery (Reallyparam, value));
			else if ("_mh". Equals (Mode)) {//The condition is a fuzzy query Boolquery.must (querybuilders.matchquery (Reallyparam, value));
		}//Create Sort objects (0 is flashback, 1 is positive) object sort = Where.get ("sort");
		Object SortType = Where.get ("SortType");
		Sortbuilder sortbuilder = null; if (! Stringutils.isemptyornull (sort) &&! Stringutils.isemptyornull (SortType)) {if ("0". Equals (SortType)) {//Flashback Sortbuilder = Sortbuilders.fieldsort (sort.
			ToString ()). order (SORTORDER.DESC);
			else {//positive sequence Sortbuilder = Sortbuilders.fieldsort (sort.tostring ()). order (SORTORDER.ASC); ///Paging data int pageIndex = Stringutils.isemptyornull (Where.get ("PageIndex"))? 1:integer.valueof (Where.get ("PageIndex"). ToSTring ()); int pageSize = Stringutils.isemptyornull (Where.get ("PageSize"))?
		
		10:integer.valueof (Where.get ("PageSize"). toString ());
		Creates a SearchQuery query object that determines whether the sorted object is empty, is empty and does not create a sort object searchquery searchquery = null; if (Sortbuilder = = null) {searchquery = new Nativesearchquerybuilder (). Withquery (Boolquery). Withpageable (P
		Agerequest.of ((pageIndex-1) *pagesize, PageSize)). Build (); else {searchquery = new Nativesearchquerybuilder (). Withquery (Boolquery). Withsort (Sortbuilder). With
		Pageable (Pagerequest.of (PageIndex, PageSize)). Build ();
		
		} list<map<string, object>> List = Elasticobjecttemplate.queryforlist (SearchQuery, IndexName, typeName); SearchQuery = new Nativesearchquerybuilder (). Withquery (Boolquery). Withindices (IndexName). Withtypes (TypeN
		AME). Build ();
		
		Long Count = Elasticobjecttemplate.count (searchquery); listresult<map<string, object>> result = new listresult<map<string,Object>> ();
		Result.setrows (list);
		Result.settotal ((int) count);
		Result.setpageindex (PageIndex);
		Result.setpages (pageSize);
	return result;  @RequestMapping (value = "/delete.ashx", method = Requestmethod.post) Public boolean insert (HttpServletRequest request)
		{map<string, object> where = dataconverthelper.getrequestparams (request);
		String IndexName = Where.get ("IndexName"). ToString ();
		String typeName = Where.get ("TypeName"). ToString ();
		String gid = where.get ("gid"). toString ();
		Elasticobjecttemplate.delete (IndexName, TypeName, GID);
	return false; }
	
}

The interface rules for the parameters of the front-end transmission can refer to the Elasticsearch Service API interface document I wrote. (The document address is shown in the link below)

Elasticsearch Service API Interface document download address: Link: https://pan.baidu.com/s/1bap8LsjH-GkoT2xJuWZcsw Password: 6tn3

Code Address: Https://github.com/lh2420124680/ElasticServer