Elasticsearch "Index aliases alias"

Source: Internet
Author: User
Tags apache log kibana logstash

Https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-aliases.html

One

The Elasticsearch alias, which is similar to a database view.

To create an alias:

We create an alias My_index_alias for the index My_index, so that we operate on the My_index_alias as if it were a my_index operation

POST/_aliases{"Actions": [{"Add": {"index": "My_index", "Alias": "My_index_alias"}} ]}

Aliases can not only associate an index, it can aggregate multiple indexes

We create an alias My_index_alias for the index my_index_1 and my_index_2, so that the operation of the My_index_alias (read only) will operate my_index_1 and my_index_2, similar to the aggregation of My_ Index_1 and my_index_2. We are not able to write to the My_index_alias, when there are multiple indexes alias, can not distinguish between the end of the operation which

POST/_aliases{"Actions": [{"Add": {"index": "My_index_1", "Alias": "My_index_alias"} }, {"Add": {"index": "My_index_2", "Alias": "My_index_alias"}}]}get/my_index_alias/_s earch{}

To create an alias for filtered:

For example, for the same index, we see different data for different people,

If My_index has a field that is a team, theteam field records that the data is the team. The data between the team is not visible.

post /_aliases{   "Actions": [    {       "Add": {         "index":  "My_index",          "Alias":  "My_index__teama_alias",         "Filter":{             "term":{                  "Team": "TeamA"              }        }       }    },    {        "Add": {         "index":  "My_index",          "Alias":  "My_index__teamb_alias",          "filter ":{            " term ":{                  "Team": "Teamb"              }        }       }    },    {        "Add": {         "index":  "My_index",          "Alias":  "My_index__team_alias"       }     }  ]}GET /my_index__teamA_alias/_search  can only see the TeamA data get /my_index__ teamb_alias/_search  can only see Teamb data get /my_index__team_alias/_search  can see TeamA, but also can see TeamB data


Second, the situation

Scenario 1: Use Logstash to collect all the current Nginx logs, put es, index named NGINX-YYYY.MM.DD

Later added Apache log, hoping to put in the same index, unified called WEB-YYYY.MM.DD

We just have to change the configuration of Logstash and then reboot so that the data will be written to the new index, but the same day's index will be written to 2 indexes, Kibana is not good configuration.


Solution:

1. Today is 2015-11-13. We created an alias called web-2015.07.28 for nginx-2015.11.13, and all of the Nginx logs were the same.

Change the dashboard's configuration index name to WEB-YYYY.MM.DD in 2.kibana

3. Change the configuration of the elasticsearch inside the Logstash to web-yyyy.mm.dd and restart


Scenario 2: use Logstash to collect all the current Nginx logs, put es, index named NGINX-YYYY.MM.DD

One day 2015-11-13 want to index by month, index name to NGINX-YYYY.MM

Note: As in Scenario 1, it is not possible for us to create a nginx-2015.11 alias to perform other indexes this month. Since an alias is pointing to multiple indexes, it is not possible for ES to know to write to that real index when writing this alias.


Solution 1:

1. New index nginx-2015.11 and its nginx-2015.11.01, nginx-2015.11.02, ...,nginx- 2015.11.30, etc.

2. Wait until the next day to change the Logstash configuration to nginx-yyyy.mm and restart

3. If the index is reserved for 10 days only, on the day of the 10 day, change the Kibana configuration to nginx-yyyy.mm

Cons: The second and third steps require manual, can write crontab scheduled tasks


We do not want users (Kibaba) to feel any change, and not to let the user feel the loss of data.

Elasticsearch "Index aliases alias"

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.