Storm topology submitted to the cluster

Source: Internet
Author: User

Question: How will topology be submitted to the cluster after the definition of each component (written **spout.java and **bolt.java) is completed topology?

Reference: http://www.cnblogs.com/fxjwind/archive/2013/06/05/3119056.html

1, the Topology object is created by Topologybuilder.createtopology () after the main method Setspout, Setbolt in **topology.java, and in the * * The main method of topology is submitted by using the following code:

Stormsubmitter.submittopology (Args[0], conf, builder.createtopology ());

2, in the Submittopology method of Stormsubmitter.java, Sir into a Nimbusclient object, and then execute the Submitjar (,,) method to commit

Nimbusclient client = nimbusclient.getconfiguredclient (conf); Submitjar (conf, progresslistener);

3, in the Submitjar method (only two parameters), first obtain the address of the jar file to be committed (represented by a string object), and then call Submitjar (,,,) to commit. Here we first determine whether the topology has been submitted

private static void Submitjar (Map conf, Progresslistener Listener) {
if (submittedjar==null) {
Log.info ("Jar not uploaded to master yet. Submitting jar ... ");
String Localjar = System.getproperty ("Storm.jar");
Submittedjar = Submitjar (conf, Localjar, listener);
} else {
Log.info ("Jar already uploaded to master. Not submitting jar. ");
}
}

4, in Submitjar (,,,), first generate a nimbusclient to obtain the destination address of the commit JAR file. Stormsubmitter is a thrift Client,nimbus is a thrift Server that sends a JAR file via RPC to nimubs through the following three steps

Returns a string submittedjar after execution of the method, marking whether the topology has been committed. This parameter is used as the submittopologywithopts () parameter in the 5th step

String uploadlocation = Client.getclient (). Beginfileupload ();
Client.getclient (). Uploadchunk (Uploadlocation, Bytebuffer.wrap (Tosubmit));
Client.getclient (). Finishfileupload (uploadlocation);

The classes in the Backtype.storm.utils Toolkit are used Bufferfileinputstream.java

5, this returns to the Submittopology method of Stormsubmitter.java, which executes

if (opts!=null) {       
Else {         // This was for backwards compatibility         client.getclient (). Submittopology ( Name, Submittedjar, serconf, topology);

Take a look at the method called in the 5th step, the public class Nimbusclient extends thriftclient the fact that the submission process is encapsulated by the thrift.

Reference: http://www.cnblogs.com/fxjwind/archive/2013/06/04/3117385.html

 Public void throws alreadyaliveexception, invalidtopologyexception, Org.apache.thrift.TException    {      send_submittopologywithopts (name, Uploadedjarlocation, jsonconf, topology, options);      Recv_submittopologywithopts ();    }
 Public void throws org.apache.thrift.TException    {      new  Submittopologywithopts_args ();      Args.set_name (name);      Args.set_uploadedjarlocation (uploadedjarlocation);      Args.set_jsonconf (jsonconf);      Args.set_topology (topology);      Args.set_options (options);      Sendbase ("submittopologywithopts", args);    }

Storm topology submitted to the cluster

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.