Python Elasticsearch api, elasticsearchpython

Source: Internet
Author: User

Python Elasticsearch api, elasticsearchpython
Description: ElasticSearch is a Lucene-based search server. It provides a distributed full-text search engine with multi-user capabilities, based on RESTful web interfaces. The following describes how to use the Python API to query data to facilitate calls by other systems. Install API

pip install elasticsearch
Create an es connection
from elasticsearch import Elasticsearches = Elasticsearch([{'host':'10.10.13.12','port':9200}])

  

Data Retrieval
es.search(index='logstash-2015.08.20', q='http_status_code:5* AND server_name:"web1"', from_='124119')

 

Common Parameters
  • Index-index name
  • Q-use Lucene query syntax to query specified matches
  • From _-default query start point 0
  • Doc_type-Document Type
  • Size-the default value of the specified number of queries is 10.
  • Field-specified fields are separated by commas (,).
  • Sort-sorting field: asc/desc
  • Body-use Query DSL
  • Scroll-rolling Query
  Statistical Query# The syntax is roughly the same as that of search, but only the statistical value is output.
In[52]: es.count(index='logstash-2015.08.21', q='http_status_code:500')Out[52]:{u'_shards':{u'failed':0, u'successful':5, u'total':5}, u'count':17042}

  

  Knowledge Extension
  • Rolling demo
# Initialize the scrollpage = es.search(    index ='yourIndex',    doc_type ='yourType',    scroll ='2m',    search_type ='scan',    size =1000,    body ={    # Your query's body})sid = page['_scroll_id']scroll_size = page['hits']['total']# Start scrollingwhile(scroll_size >0):    print "Scrolling..."    page = es.scroll(scroll_id = sid, scroll ='2m')    # Update the scroll ID    sid = page['_scroll_id']    # Get the number of results that we returned in the last scroll    scroll_size = len(page['hits']['hits'])    print "scroll size: "+ str(scroll_size)    # Do something with the obtained page
In the preceding demo, data is retrieved at a time. After the data is obtained, the latest data is not obtained. What should we do if we want to get the latest data after rolling? A statistical value, such as total: 5, is generated during scrolling. After jumping out of the loop, we can use the _ from parameter to locate the data after 5 starts to scroll.
  • Query DSL
  Range filter query rangeGt:> greater than lt: <less than gte: >=greater than or equal to lte: <= less than or equal
"range":{    "money":{        "gt":20,        "lt":40    }}      
  1.   

Bool combination filterMust: All clauses must match AND be the same as AND. Must_not: All clauses must NOT match and NOT. Shocould: at least one clause matches with OR.
{    "bool":{      "must":[],      "should":[],      "must_not":[],    }}
Term Filter
  • Term Single Filter
{    "terms":{      "money":20    }}

  

  • Multiple matching conditions are allowed for the complex terms version.
{    "terms":{      "money": [20,30]    }}
Regular Expression Query
{    "regexp": {        "http_status_code": "5.*"    }}
Match Query
  • Exact match
{    "match":{      "email":"123456@qq.com"    }}
 
  • Multi_match multi-field search
{    "multi_match":{      "query":"11",      "fields":["Tr","Tq"]    }}
Demo
  • Obtain data from the last hour
{'query':    {'filtered':        {'filter':            {'range':                {'@timestamp':{'gt':'now-1h'}}            }        }    }}
  1.   

  • Conditional filter Query
{    "query":{        "filtered":{            "query":{"match":{"http_status_code":500}},            "filter":{"term":{"server_name":"vip03"}}        }    }}
 
  • Terms Facet single field statistics
{'facets':    {'stat':        {'terms':            {'field':'http_status_code',              'order':'count',        'size':50}        }    },    'size':0}        

   

  • Count multiple fields at a time
{'facets':    {'cip':        {'terms':            {'fields':['client_ip']}},              'status_facets':{'terms':{'fields':['http_status_code'],              'order':'term',              'size':50}}},        'query':{'query_string':{'query':'*'}},    'size':0}

  

  • Count multiple fields together
{'facets':    {'tag':        {'terms':            {'fields':['http_status_code','client_ip'],              'size':10           }        }    },    'query':        {'match_all':{}},    'size':0} 

  

  Data assemblyThe following is a demo of the kibana homepage, which is used to count the number of logs in a period of time.
{"Facets": {"0": {"date_histogram": {"field": "@ timestamp", "interval": "5 m"}, "facet_filter ": {"fquery": {"query": {"filtered": {"query": {"query_string": {"query": "*" }}, "filter ": {"bool": {"must": [{"range": {"@ timestamp": {'gt ': 'Now-1h '}}}, {"exists": {"field": "http_status_code.raw" }}, # ------------- ------- # matching conditions] }}}}, "size": 0}

  

If you want to add matching conditions, add filtering conditions in the above code identification section, follow the code format below
{"query": {    "query_string": {"query": "backend_name:baidu.com"}    }},

  

Here, we will introduce the Query dsl api later.

 

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.