Elasticsearch Bulk API Bulk Index

This blog provides an easy way for Elasticsearch to index multiple documents. The support of Bulk API can implement batch add, delete, update and so on once request. The bulk operation uses the UDP protocol, and UDP cannot ensure that data is not lost when communicating with the Elasticsearch server.

First, Bulk API

With the bulk command, the REST API _bulk ends with a bulk operation written in the JSON file, with the syntax format given by the official website:

action_and_meta_data\noptional_source\naction_and_meta_data\noptional_source\n....action_and_meta_data\noptional_source\n

In other words, each operation has 2 rows of data, the end of which is to return the line. The first line describes the action command and the original data, and the second line is a custom option. For example, execute inserting 2 data, delete a piece of data, create a new Bulkdata.json, write the following:

{ "Create" :{"_index": "blog", "_type": "article", "_id":  "3" }}{ "title":"Title1","Posttime":"2016-07-02","content": "Content one" }{ "Create" :{"_index": "blog", "_type": "article", "_id":  "4" }}{ "title":"Title2","Posttime":"2016-07-03","content": "Content 2" }{ "Delete":{"_index": "blog", "_type": "article", "_id":  "1" 
      }}

Perform:

$ curl-xpost "Http://localhost:9200/_bulk?pretty" --Data-binary@bulkAdd.json{"took": One,"Errors":false,"Items":[{"Create": {"_index":"Blog","_type":"article","_id":"very","_version":1,"_shards": {"Total":1,"Successful":1,"Failed":0},"Status":201}  }]}

return value:

Note: At the end of the line to enter a newline, or it will be because the command is not recognized and an error occurs.

$ Curl-Xpost "Http://localhost:9200/_bulk?pretty"--data-binary@bulkAdd. JSON {"Error" :{"Root_cause" :[ {"Type" : "Action_request_validation_exception","Reason" : "Validation Failed:1: No requests added;"} ],"Type" : "Action_request_validation_exception","Reason" : "Validation Failed:1: No requests added;"},"Status" :  -}

Second, Batch Export

The following example is to batch export the document in the index library to a file in JSON format, where the cluster name is "Bropen", the index library is named "blog", the type is "article", the project root directory under the new Files/bulk.txt, The index content is written to Bulk.txt:

Import Java. IO. BufferedWriter;Import Java. IO. File;Import Java. IO. FileWriter;Import Java. IO. IOException;Import Java. NET. InetAddress;Import Java. NET. Unknownhostexception;import org. Elasticsearch. Action. Search. SearchResponse;import org. Elasticsearch. Client. Client;import org. Elasticsearch. Client. Transport. Transportclient;import org. Elasticsearch. Common. Settings. Settings;import org. Elasticsearch. Common. Transport. Inetsockettransportaddress;import org. Elasticsearch. Index. Query. QueryBuilder;import org. Elasticsearch. Index. Query. Querybuilders;import org. Elasticsearch. Search. Searchhits;public class Elasticsearchbulkout {public static void main (string[] args) {try {Settings Settings = Settings. Settingsbuilder(). Put("Cluster.name","Bropen"). Build();//Cluster.name in Elasticsearch.ymlClient client = Transportclient. Builder(). Settings(settings). Build(). Addtransportaddress(New Inetsockettransportaddress (inetaddress. Getbyname("127.0.0.1"),9300));QueryBuilder QB = Querybuilders. Matchallquery();SearchResponse response = Client. Preparesearch("Blog"). Settypes("article"). Setquery(querybuilders. Matchallquery()). Execute(). Actionget();Searchhits resulthits = response. Gethits();File Article = new file ("Files/bulk.txt");FileWriter FW = new FileWriter (article);BufferedWriter BFW = new BufferedWriter (FW);if (resulthits. Gethits(). Length==0) {System. out. println("0 data found!");} else {for (int i =0; i < resulthits.gethits (). length; i++) {String jsonstr = resulthits. Gethits() [i]. getsourceasstring();System. out. println(JSONSTR);BfW. Write(JSONSTR);BfW. Write("\ n");}} BFW. Close();Fw. Close();} catch (Unknownhostexception e) {E. Printstacktrace();} catch (IOException e) {E. Printstacktrace();}    }}

third, batch import

Read by line from the Bulk.txt file that you just exported, and then bulk import. First, the client.prepareBulk() Bulkrequestbuilder object is instantiated by calling, and the Add method of the Bulkrequestbuilder object is called for adding data. Implementation code:

Import Java. IO. BufferedReader;Import Java. IO. File;Import Java. IO. FileNotFoundException;Import Java. IO. FileReader;Import Java. IO. IOException;Import Java. NET. InetAddress;Import Java. NET. Unknownhostexception;import org. Elasticsearch. Action. Bulk. Bulkrequestbuilder;import org. Elasticsearch. Client. Client;import org. Elasticsearch. Client. Transport. Transportclient;import org. Elasticsearch. Common. Settings. Settings;import org. Elasticsearch. Common. Transport. Inetsockettransportaddress;public class Elasticsearchbulkin {public static void main (string[] args) {try {Settings Settings = Settings. Settingsbuilder(). Put("Cluster.name","Bropen"). Build();//Cluster.name configured in Elasticsearch.ymlClient client = Transportclient. Builder(). Settings(settings). Build(). Addtransportaddress(New Inetsockettransportaddress (inetaddress. Getbyname("127.0.0.1"),9300));File Article = new file ("Files/bulk.txt");FileReader fr=new FileReader (article);BufferedReader bfr=new BufferedReader (FR);String Line=null;Bulkrequestbuilder bulkrequest=client. Preparebulk();int count=0;while ((LINE=BFR. ReadLine())!=null) {Bulkrequest. Add(Client. Prepareindex("Test","article"). SetSource(line));if (count%Ten==0) {Bulkrequest. Execute(). Actionget();} count++;System. out. println(line);} bulkrequest. Execute(). Actionget();BfR. Close();Fr. Close();} catch (Unknownhostexception e) {E. Printstacktrace();} catch (FileNotFoundException e) {E. Printstacktrace();} catch (IOException e) {E. Printstacktrace();}    }}

Reference Documentation:

1. Elasticsearch Reference [2.3]? Document APIs? Bulk API

Elasticsearch Bulk API Bulk Index