Elasticsearch October 2014 briefing, elasticsearch

Elasticsearch October 2014 briefing, elasticsearch
1. Elasticsearch Updates

1.1 released Kibana 4 Beta 1 and Beta 1.1

Kibana 4 is different from Kibana in layout, configuration, and bottom-layer Chart Drawing. After learning the functional requirements of many communities based on Kibana 3, Kibana's self-Kibana 2 major change resulted in the second major change made by Kibana 3. Kibana has always been committed to helping users solve their difficult problems in data visualization.

The interface is changed from the original 'dashboard' to the current three tabs: 'discover', 'visualize', and 'dashboard', and a total of four different tabs are added with 'settings.

Discover: Is this Tab interface similar to the previous Kibana 2? Index-oriented query: searches for index content. You can enter 'lucene Query String syntax 'or es's 'json-format query' in the search box. query does not belong to a dashboard only, but is shared globally, it can also be passed in through a URL to link to a query.

Visualize: You can use els' aggregations to create charts with complex logic. (The elasticsearch facet and aggregations used by kibana3 are more complex than other functions.) Click the gray icon at the bottom of visualize to view the original data of the chart, response and request processing time.

Dashboard: Dashboard is used to create a chart set to meet specific visualization requirements, but it is easier to configure and maintain than before. Obviously, Kibana 4 greatly increases the reusability of charts and queries. Charts created in Visualize can be reused multiple times on the Dashboard. Now, a Dashboard can also display data of Multiple indexes.

Settings: some global Settings of kibana

Note: Kibana 3 is not compatible with the schema of Kibana 4. The configuration of Kibana 3 cannot be imported into Kibana 4. Kibana 4 requires Elasticsearch version> = 1.4.0

1.2 released Elasticsearch 1.4.0.Beta1

This version mainly improves stability and reliability.

1.2.1 Better node stability through CED memory usage.

The recently introduced doc value mechanism has been greatly improved to replace the original in memory fielddata. Doc value is used for aggregation in the original memory. for statistics, the queried field value is stored on the disk and the filesystem cache of the system kernel is used to accelerate access to doc value, make the performance close to the original fielddata.

Added a limit on the memory used to process a single request (request circuit breaker ).

1.2.2 Better cluster stability through improved discovery algorithms.

Fixed the cluster instability problems encountered by many users in the production environment. The resiliency status details the issues submitted by the user, their repair progress, and measures to protect data in the production environment.

1.2.3 Better detection of specified upted data through checksums.

The Checksums verification function is added to shard recovery, merging, and transaction log to verify whether the data is damaged.

1.2.4 Other major updates

Groovy replaces mvel with ES's default scripting language to improve security and operational efficiency.
For security reasons, CORS is disabled by default.

Three types of aggregations are added: filters, children, and scripted_metric.

As the integrated Lucene version in elasticsearch is updated, it will not support Lucene 3 in future elasticsearch versions. therefore, ES adds the your_index/_ upgrade rest api to convert old indexes into indexes compatible with the latest Lucene.

2. Elasticsearch Ecosystem Updates

2.1 released Elasticsearch Hadoop 2.0.2 and 2.1.Beta2

3. Amazing Slides & tutorials & videos

Playing http tricks with nginx

Use nginx as the es proxy and provide the persistent http link, load balance, security control (basic http auth, role based auth, oauth) functions.

Link: http://www.elasticsearch.org/blog/playing-http-tricks-nginx/

Deploying the ELK stack using Docker

Popular app container docker deployment ELK

Link: https://clusterhq.com/blog/deploying-multi-node-elasticsearch-logstash-kibana-cluster-using-docker/

Elasticsearch from the Top Down Tracing a Request Down to the Bits

The underlying implementation of Elasticsearch discusses how Elasticsearch clusters process index and query requests.

Link: https://found.no/foundation/elasticsearch-top-down/

Building Scalable Search from Scratch with ElasticSearch

A basic detailed tutorial

Link: http://www.airpair.com/elasticsearch/posts/elasticsearch-robust-search-functionality

4. Meetups in China

The third elasticsearch domestic developer Exchange conference was held in October 25. For details, visit http://www.meetup.com/elasticsearch-china-users/events/210253352 /.
Conference PPT download: http://pan.baidu.com/s/1i3qsoBF

References:

[1]: http://www.elasticsearch.org/blog/kibana-4-beta-1-released/ "Kibana 4 Beta 1 released"
[2]: http://www.elasticsearch.org/blog/kibana-4-beta-1-1-pointy-needles-blunted/ "kibana 4 beta 1.1: pointy needles blunted"
[3]: http://www.elasticsearch.org/blog/elasticsearch-1-4-0-beta-released/ "Elasticsearch 1.4.0.Beta1 released"
[4]: http://www.elasticsearch.org/guide/en/elasticsearch/resiliency/current/index.html "resiliency status"
[5]: http://www.elasticsearch.org/blog/scripting/ "all about scripting"
[6]: http://www.elasticsearch.org/blog/elasticsearch-hadoop-2-0-2-and-2-1-beta2/ "Elasticsearch Hadoop 2.0.2 and 2.1.Beta2 released"
[7]: http://www.elasticsearch.org/blog/2014-10-08-this-week-in-elasticsearch/ "This week in Elasticsearch October 8, 2014"
[8]: http://www.elasticsearch.org/blog/2014-10-15-this-week-in-elasticsearch/ "This week in Elasticsearch October 15,201 4"
[9]: http://www.elasticsearch.org/blog/2014-10-22-this-week-in-elasticsearch/ "This Week in ElasticsearchOctober 22,201 4"
[10]: http://www.elasticsearch.org/blog/2014-10-29-this-week-in-elasticsearch/ "This week in ElasticsearchOctober 29,201 4"
[11]: http://www.elasticsearch.org/blog/playing-http-tricks-nginx/ "playing http tricks with nginx"
[12]: https://clusterhq.com/blog/deploying-multi-node-elasticsearch-logstash-kibana-cluster-using-docker/ "deploying the ELK stack using Docker"
[13]: https://found.no/foundation/elasticsearch-top-down/ "Elasticsearch from the Top Down Tracing a Request Down to the Bits"
[14]: http://www.airpair.com/elasticsearch/posts/elasticsearch-robust-search-functionality "Building Scalable Search from Scratch with ElasticSearch"

Repost this article please indicate the author and the source [Gary's influence] http://garyelephant.me, do not for any commercial purposes!
Author: Gary Gao (garygaowork [at] gmail.com) focuses on the internet, distributed, high-performance, NoSQL, automation, and software teams