Spark API programming Hands-on-04-to implement operations on Union, Groupbykey, join, reduce, lookup, etc. in the Spark 1.2 release

Below is a look at the use of Union:

Use the collect operation to see the results of the execution:

Then look at the use of Groupbykey:

Execution Result:

The join operation is the process of a Cartesian product operation, as shown in the following example:

To perform a join operation on RDD3 and RDD4:

Use collect to view execution results:

It can be seen that the join operation is exactly a Cartesian product operation;

The reduce itself, which is an action-type operation in an RDD operation, causes the job to be committed and executed:

Let's look at the use of the lookup:

Execution Result:

Spark API programming Hands-on-04-to implement operations on Union, Groupbykey, join, reduce, lookup, etc. in the Spark 1.2 release