HadoopHerriot test framework _ Overview

Herriot testing framework is a new testing framework for Hadoop-0.21.0 and later versions. It is mainly used to simulate a real large-scale distributed system as much as possible, and to implement automated testing for the system. Similar to the previous Hadoop testing framework MiniDFSCluster, Herriot also adopted the JUnit framework. In addition, Herriot introduced the cross-section Herriot testing framework, which is a new testing framework for Hadoop-0.21.0 and later versions, it is mainly used to simulate a real large-scale distributed system as much as possible and implement automated testing for the system. Similar to the previous Hadoop testing framework MiniDFSCluster, Herriot also adopted the JUnit framework. In addition, Herriot also introduced the Aspect-oriented programming (AOP) technology-AspectJ to implement code injection and error injection, enrich the test content.

Uses the Herriot framework for Hadoop ClusterIn typical scenarios, we can see that, unlike earlier MiniDFSCluster testing frameworks, each node (NameNode, JobTracker, DataNode, and TaskTracker) in the Herriot framework is in a separate JVM process, this is consistent with the real Hadoop cluster. the test machine uses the local Herriot package to remotely call and inject Herriot code into the target node.

To test the Java system, we need to inject some test-related code into the tested Java system. There are usually three methods: 1. directly add the test code to the code, compile the code into a class bytecode. This method is relatively simple, but the test code is contaminated with the original code and may pose a security risk. 2. After the code is compiled into a class bytecode, add a test method by modifying the class bytecode, that is, generate a new class file (usually called ins ).TrUmentEdClass), this implementation is more complex than the first method, but does not pollute the original code, but the bytecode used for testing is not the same as the class bytecode generated by the original code; 3. Add the test function by modifying the JVM. This method is the most complex, but does not affect the code and bytecode. The AOP technology used by Herriot is the second type, namely, testing function injection by modifying bytecode.

[Content navigation]
Page 1: Overview	Page 1: Source Code Analysis
Page 1: Demo instance