Strom concurrency Model:How does the topology work? (Can be compared with mapreduce)First Floor: ClusterSecond Layer: Supervisor (host, node, machine)Third Floor: worker (process) Fourth Floor: Executor (thread) Layer Fifth: Task (objects of threads, such as spout and blot)//topology:n workers== per ==>1 excutor=>1 task//At this point, it's the same as the MapReduce one.
|
|
| Topology Code |
|
| conf.setnumworkers ( 2 )//Processtopologybuilder.setspout ("Blue-spout", New Blue (), 2 )//threadTopologybuilder.setbolt ("Green-spot", New Green (), 2 ). Setnumtasks (4). Shufflegrouping ("blue-sqout")//Thread, ObjectTopologybuilder.serbolt ("Yellow-sqout", New Yellow (), 6 ). Shufflegrouping ("Green-bolt") |
|
Summary:The worker first set how many is topology inside how many, executor degree of parallelism there is how many is set how many, task if not set that is a executor corresponds to a task, If there is a write that is the task count is the number of tasks set divided by executor number to decide. "Note" Hadoop's mapreduce concurrency model, a job with a lot of tasks, then there are 1000 map, there are 100 reduce, a cluster has a lot of nodes, for example, there are 500 nodes, then this time, Is that I have some map task on each node, some reduce task, this time my 1000 map to each machine up and running, each maptask is a process, each reduce task is also a process, Hadoop its concurrency model is very simple, two levels, it has a lot of machines, each machine has a lot of task processes, the task process inside the run is our MapReduce function
From for notes (Wiz)
Class Note--strom concurrency model
