Using Mrunit to test mapper and reducer

Mrunit can take less time and can test mapper and reducer separately

Steps:

1, the use of Mrunit test mapper and Reducer

2, the implementation of the MapReduce code localization test

3. Using Hadoop logs

4. Track execution metrics by counter

The process of testing mapper

1, instantiate the Mapdirver class, as the test mapper is parameterized

2. Call the Withmapper method to add the mapper instance you want to test

3, according to the situation select Withconfiguration Method Set Mapper configuration file

4. Call the Withinput method to pass the required key and value

5. Set the desired output by calling the Withoutput method

6, call the function Rantest method to pass the input value of the set to Mapper

The process of testing reducer

1. First create a list of Intwritable objects to be used as reducer input

2. Instantiating Reducedriver objects

3. Call the Withreduce method to set the reducer instance to be tested

4, through the withconfiguration to pass the desired configuration

5. Call the Withinput method to pass the input value to reducer

6. Call the Withoutput method to set the desired reducer output

7. Finally call the Runtest method and pass the input of the setting to the reducer

Using Mrunit to test mapper and reducer