Elasticsearch Learning Primer

In the past few years, the development threshold of the search is getting lower, each language has the open Source Retrieval toolkit, and the function is more and more full, complete solution also more and more, more and more use, for example Lucene on has SOLR, Elasticsearch, Sensei and so on. They cover most of the needs, freeing up the development effort directly on the Search toolkit, allowing people to focus more on business development. Personal comparison optimistic Elasticsearch (es), ES is very simple to use, let people feel more in use a nosql, and allow a lot of plug-in features can be developed by themselves. We can easily test es through the rest client, so it's easy to learn.

ES's official website has a more comprehensive API, but I have seen the level of the API is still a bit messy, at least without MongoDB documents so easy to read. Get to know es from a simple application. For example, to build a Chinese news search engine, the news has "title", "Content", "Author", "Type", "Release Time" of the five fields; we want to provide "title and content Search", "sorting", "highlighting", "statistics", "filtering" and other basic functions. ES provides SMARTCN's Chinese word-breaker, which is recommended for use with IK word breakers, or for examples given by plugin authors.

Download and install the plugin, start es, then you can start the ES experience.

1. Create an index named Test

PUT Http://localhost:9200/test

2. Create mapping

POST http://localhost:9200/test/news/_mapping

The contents are:

2. {
4. "News": {
8. "Properties": {
10. "Content": {
12. ' Type ': ' String ',
14. "Store": "No",
16. "Term_vector": "With_positions_offsets",
18. "Index_analyzer": "Ik",
20. "Search_analyzer": "IK"
22. }
24. ,
26. "title": {
28. ' Type ': ' String ',
30. "Store": "No",
32. "Term_vector": "With_positions_offsets",
34. "Index_analyzer": "Ik",
36. "Search_analyzer": "Ik",
38. "Boost": 5
40. }
42. ,
44. "Author": {
46. "Type": "string", "index": "Not_analyzed"
48. }
50. ,
52. "Publish_date": {
54. "Type": "Date", "format": "Yyyy/mm/dd", "index": "Not_analyzed"
56. }
58. ,
60. "category": {
62. "Type": "string", "index": "Not_analyzed"
64. }
66. }
68. }
70. }

Copy Code

The URL of the test, news in ES is the index/type, the feeling is the corresponding database like library name/table name of the relationship, post content in the properties corresponding to the contents of mapping, inside 5 fields. Type indicates the field type, content, Title field to be participle and highlight so set the word breaker and turn on Term_vector. Specific types of APIs can be seen here.

3. Manufacture and submit some data:

POST http://localhost:9200/test/news/

The content is made up of several articles:

1. {
3. "Content": "China's premier visits Europe",
5. "title": "The U.S. economic situation is good this year",
7. "Publish_date": "2010/07/01",
9. "Author": "Zhang San",
11. "Category": "Finance"
13. }

Copy Code

4. Search

POST Http://localhost:9200/test/news/_search

The content includes several parts:

Page: from/size, Field: Fields, sort, query: queries, filtering: Filter, highlighting: Highlight, Statistics: facets

2. {
4. "From": 0,
6. "Size": 10,
8. "Fields": ["title", "Content", "Publish_date", "category", "author"],
10. "Sort": [
12. {"Publish_date": {"Order": "ASC"}},
14. "_score"
16. ],
18. "Query": {
20. "BOOL": {
22. "Should": [
24. {
26. "term": {"title": "China"}
28. },
30. {
32. "term": {"Content": "China"}
34. }
36. ]}
38. },
40. "Filter": {
42. "Range": {
44. "Publish_date": {
46. "From": "2010/07/01",
48. "To": "2010/07/21",
50. "Include_lower": true,
52. "Include_upper": false
54. }
56. }
58. },
60. "Highlight": {
62. "Pre_tags": ["<tag1>", "<tag2>"],
64. "Post_tags": ["</tag1>", "</tag2>"],
66. "Fields": {
68. "title": {},
70. "Content": {}
72. }
74. },
76. "Facets": {
78. "Cate": {"Terms": {"field": "Category"}}
80. }
82. }
86. Here query I chose to construct a title or content containing "China",
90. The results are as follows:
94. {
96. "Took": 18,
98. "Timed_out": false,
100. "_shards": {
102. "Total": 5,
104. "Successful": 5,
106. "Failed": 0
108. },
110. "Hits": {
112. "Total": 6,
114. "Max_score": null,
116. "Hits": [{
118. "_index": "Test",
120. "_type": "News",
122. "_id": "_fm13zccsnwatpouziok_a",
124. "_score": 0.024621923,
126. "Fields": {
128. "CONTENT": "Reuters reported how the United States in Iraq yesterday, how China will face the situation." ",
130. "Author": "Zhang San",
132. "title": "The United States left Iraq is a mess",
134. "Category": "Politics",
136. "Publish_date": "2010/07/10"
138. },
140. "Highlight": {
142. "Content": ["Reuters reported how the United States in Iraq yesterday how to,<tag2> China </tag2> will face the situation." " ]
144. },
146. "Sort": [1278720000000, 0.024621923]
148. }, {
150. "_index": "Test",
152. "_type": "News",
154. "_id": "4fey1t6-rmomojyts4foaq",
156. "_score": 0.024621923,
158. "Fields": {
160. "Content": "China will visit the northern part of Russia Today," the Associated Press reported. ",
162. "Author": "Zhang San",
164. "title": "American Visit to Russia",
166. "Category": "Politics",
168. "Publish_date": "2010/07/11"
170. },
172. "Highlight": {
174. "Content": ["AP reports,<tag2> China </tag2> will visit Russia's northern region today. " ]
176. },
178. "Sort": [1278806400000, 0.024621923]
180. }, {
182. "_index": "Test",
184. "_type": "News",
186. "_id": "Ll-8bzcntt2yekqs1owc_a",
188. "_score": 0.61871845,
190. "Fields": {
192. "Content": "U.S. Secretary of State Hillary Clinton told reporters that most of South Korea's Seoul Jiangnan District are willing to move to North Korea." ",
194. "Author": "John Doe",
196. "title": "China's economy will face a downside risk",
198. "category": "Economy",
200. "Publish_date": "2010/07/12"
202. },
204. "Highlight": {
206. "title": ["<tag2> China </tag2> economy will face downside risk"]
208. },
210. "Sort": [1278892800000, 0.61871845]
212. }, {
214. "_index": "Test",
216. "_type": "News",
218. "_id": "dnb6gtpsraoexc1axcpz0q",
220. "_score": 0.048311904,
222. "Fields": {
224. "Content": "China's premier talks about domestic sports, the gold medal is undoubtedly the most important, the United States",
226. "Author": "Zhang San",
228. "title": "The National System of American Sports",
230. "Category": "Sports",
232. "Publish_date": "2010/07/14"
234. },
236. "Highlight": {
238. "Content": ["<tag2> China </tag2> Premier talks about domestic sports, gold medal is undoubtedly the most important, the United States"]
240. },
242. "Sort": [1279065600000, 0.048311904]
244. }, {
246. "_index": "Test",
248. "_type": "News",
250. "_id": "4lh55yoaqve7cariyvfynq",
252. "_score": 0.048311904,
254. "Fields": {
256. "Content": "China's military threat to South-East Asia will continue, Russia is closely concerned about developments",
258. "Author": "Zhang San",
260. "title": "National efforts to develop education",
262. "Category": "Politics",
264. "Publish_date": "2010/07/15"
266. },
268. "Highlight": {
270. "Content": ["<tag2> China </tag2> military threat to South-East Asia will continue, Russia closely monitor developments"]
272. },
274. "Sort": [1279152000000, 0.048311904]
276. }, {
278. "_index": "Test",
280. "_type": "News",
282. "_id": "HFOVOBDCTI-2ERYAQV-AKG",
284. "_score": 0.12422675,
286. "Fields": {
288. "Content": "China's economic rebound is weak, the U.S. economy continues to slump, other economies are unreliable",
290. "Author": "John Doe",
292. "title": "Europe's debt crisis sweeping the world",
294. "category": "Economy",
296. "Publish_date": "2010/07/19"
298. },
300. "Highlight": {
302. "Content": ["<tag2> China </tag2> economic rebound weak, the U.S. economy continues to slump, other economies are unreliable"]
304. },
306. "Sort": [1279497600000, 0.12422675]
308. } ]
310. },
312. "Facets": {
314. "Cate": {
316. "_type": "Terms",
318. "Missing": 0,
320. "Total": 10,
322. "Other": 0,
324. "Terms": [{
326. "Term": "Politics",
328. "Count": 4
330. }, {
332. "term": "Economy",
334. "Count": 3
336. }, {
338. "term": "Sport",
340. "Count": 3
342. } ]
344. }
346. }
348. }

Copy Code

The results contain several parts that are needed. It should be noted that facet statistics is the result of the hit statistics, filter is the result of filtering, filter does not affect the facet, if you want to count the filter out of the use of the filter facet.

ES provides a lot of functionality, and the combination of JSON is messy than the usual syntax for SQL, but with a combination that can satisfy a variety of complex applications, many of the settings that affect performance are worth looking at. For developers, it is also necessary to use the built-in API interface for two development, such as developing your own participle, synchronization with other libraries, and so on. ES is still in 0.19.9 but there have been many official trainings, so the future should be limitless, and it may be the first choice for the search system solution.

Elasticsearch Learning Primer