Dcframework dynamic distributed computing framework (00) -- order

One of the most basic purposes of distributed computing is to make full use of the computing power of idle computing nodes. Normally, weFirstWill be distributed computingProgramOn each computing nodeDeploymentWell, then you can call distributed computing. Such "deployment" is static. Static is called because there is a more dynamic and flexible way-we canAnytime, dynamicAttach usNewCompleted ProgramAutomatically [Submit]DeployedThe distributed framework dynamically runs. For example, assume that I have just written a new sort algorithm program (assuming that the set is X. DLL), and then I want to sort the existing 10000000 data, but my machine configuration is too bad and cannot run such a large computing task, then I can set X. DLL and the data to be sorted are transmitted to the dynamic distributed computing framework instance deployed elsewhere. The dynamic distributed computing framework calls X. DLL to sort data and return results. This is"Dynamic. The main goal of my dcframework (dynamic distributed computing framework) is to achieve this effect. I wonder if this distributed computing architecture has something to do with cloud computing?

If C or C ++ is used to achieve such dynamic results, it should not be a simple task, but it is powerful. NET platform makes it possible to achieve this goal, and it is not that difficult.

I select the platform in. Net, which makes many functions easier to implement and simplifies many problems. The main features of dcframework are as follows:

(1) Based on the. NET platform. This has two meanings: dcframework is implemented using. Net (2.0), and only. Net-based programs can use the dynamic distributed computing architecture supported by dcframework.

(2) dcframework uses dctask to represent a dynamic distributed computing task.

(3) distributed computing nodes can be dynamically added or deleted. The so-called computing node is the node used to truly execute distributed computing tasks. In dcframework, it is called dc worker node.

(4) In the dcframework architecture, a special node is called the DC master node. Its function is to manage all the DC worker nodes in a unified manner, the new distributed computing task is assigned to the DC worker node with the lowest load.

(5) dcframework uses remoting technology to resolve communication between DC worker nodes and DC master nodes, as well as between DC master nodes and clients (clients requiring Dynamic Distributed Computing.

(6) dcframework uses reflection technology to dynamically load the Assembly and call the calculation method.

(7) dcframework uses dcappdomain (encapsulation of. Net appdomain) to build an independent environment for executing distributed tasks. That is, each distributed computing task is finally executed in an independent dcappdomain.

(8) dcframework uses serialization technology to pass called objects.

A simple illustration of node deployment in dcframework is as follows:

 

Now, let's assume that a client A has a computing task to be submitted to the dcframework for execution. Let's see how this process works.

Client A encapsulates the computing logic in the myclass class, which is located in X. DLL assembly, and X. DLL depends on Y. DLL, Y. DLL depends on Z again. DLL, and X. DLL, Y. DLL, Z. DLL is a user-defined assembly. If a computing task is executed locally on Client A, this is the case:

Myclass = new myclass ();
// Assign values to the myclass attribute and initialize it. Some code is omitted here.
Object result = myclass. dosomething ();

The following code runs the myclass. dosomething () method on a DC worker node in the dcframework architecture.

Myclass = new myclass ();
// Assign values to the myclass attribute and initialize it. Some code is omitted here.

Dcjobdispatcher dispatcher = ......;
Dispatcher. Dispatch (myclass, "dosomething ");

The execution process of distributed computing is described as follows:

(1) dcjobdispatcher automatically remoting the connection to the DC master node.

(2) dcjobdispatcher encapsulates a distributed computing task as a dctask. The dctask instance contains file data corresponding to the X. dll, Y. dll, and Z. dll assembly.

(3) dcjobdispatcher submits the dctask to the DC master. The DC master submits it to the node with the smallest load in the DC worker node (for example, worker a) for execution.

(4) worker a creates a separate dcappdomain for the new task, and then loads X. DLL, Y. DLL, Z. DLL, then the myclass object can be deserialized, and its dosomething method can be called through reflection to start computing tasks.

(5) After the calculation is complete, worker a uninstalls the corresponding dcappdomain and returns the calculation result.

The above is a brief introduction to the execution of dynamic distributed tasks by dcframework. More details will be described in subsequent articles.

Welcome to the discussion. Thank you.

 

Directory:

Dcframework dynamic distributed computing framework (01) -- infrastructure

 

Download:

1. Complete dcframework demo source code: dcframeworkdemo