Kafka.utils, as the name implies, is a toolkit package, inside the class encapsulates a lot of common function implementation--here, I have a feeling: Originally to read Kafka source code and learn the Scala language, this thought Kafka implementation will use a lot of function programming ( Functional programming, FP), as a result, most of the time, most of it is very simple to implement in an object-oriented way, and only a very small subset of the collections are handled using FP methods such as Map,reduce. I cannot but say a little regret. --Of course, you might see more FP figures in the core code behind Kafka.
is all the code for the Kafka.utils package:because it's hard to have a logical relationship with other package code, let's say one:
First, Annotations.scala3 annotation classes are defined in this source code file: Threadsafe, Nonthreadsafe, and immutable. They all inherit the staticannotation provided by Staticannotation--scala similar to @target (Elementtype.type) in Java, so the primary scope is classes and interfaces. Specific to these three meta-annotations (meta-annotation), it's easy to know what they mean: thread-safe, non-thread-safe, and immutability are labeled separately. The Simpleconsumer class commonly used in Kafka development is marked as @threadsafe.
two. Commandlineutils.scalaThis file uses jopt Simple Library is responsible for parsing command line parameters, specific use See official website:http://pholser.github.io/jopt-simple/ Kafka provides a object:commandlineutils in this file. The specific methods include:1. Printusageanddie: Print command use method and terminate program2. Checkrequiredargs: Use jopts Simple API (same as below) to check if necessary parameters are missing3. Checkinvalidargs: Checks whether the specified parameter is incompatible, that is, which parameters cannot be used simultaneously4. Parsekeyvalueargs: Parses the parameter pair in the Key=value format and returns a Properties object
Third, Crc32.scalaThis class is the implementation class for the CRC32 checksum, derived from the pure Java implementation version of the PUREJAVACRC32 class--CRC32 checksum code provided by Hadoop. This class is very long, there are many bit operations, because CRC32 calculation is not in the scope of this study, so you know this.
iv. Delayeditem.scala
This class is a generic class that implements the Java.util.Delayed interface. Used to mark objects that are executed after a given delay time. This class receives a generic T, a delay time, and a unit of delay time. In addition, a CompareTo and Getdelay method must be implemented to implement this interface. 1. Getdelay: Calculate how long the distance trigger time remains2. CompareTo: Compare delay Trigger time for 2 delayed objects
Wu, Filelock.scalaas the name implies, Filelock is a file lock, its constructor receives a file object, and always tries to create the file (if it does not exist), and then creates a FileChannel object to read and write the file randomly. Also create a Java.nio.channel.FlieLock file lock object to implement the following method:1. Lock: Lock the file if a lock on the file has thrown an exception2. Trylock: Attempt to lock the file and return False if successful returns True3. Unlock: If holding the lock use the Filelock.release method to release the lock4. Destroy: Release the lock and call FileChannel's Close method to destroy the channel
Liu, Iteratortemplate.scalaThis file view defines an iterator template that is used primarily to traverse the message collection. The iterator template has a status field, so before defining an iterator template abstract class, you first define a state status object, as well as a set of specific state object: Finish (done), readiness (Ready), Not_ready (not prepared), and failed (failed )。 This is followed by the definition of the Iteratortemplate abstract class, which simultaneously implements the trait iterator and Java iterator interfaces-a synthesizer of the iterator domain:) as mentioned earlier, the class has a field that indicates the status of the iterator: state, and a nextitem field that executes the next object in the traversal, of course, initialized to null--to say null and think of an off-topic. I doubt that the developer of Kafka is a deep Java programmer or a strong object developer, Scala recommends that option be used instead of NULL, and that Kafka code can be null or ubiquitous and, of course, to be better and more natural to integrate with Java. this abstract class provides many methods, but there seems to be only one abstract method: Makenext, the others are all concrete methods:1. Next: Throws an exception directly if the iterator has traversed and cannot find the next item or the next item is empty, otherwise the state is set to Not_ready and the next item is returned2. Peek: Just probe to see if the iterator is finished, if it throws an exception, it returns the next item directly, does not make a non-null judgment, and does not make a state setting3. Hasnext: If the status is failed directly throws an exception, if done returns false, if it is ready returns True, the Maybecomputenext method is called4. Makenext: Return to the next item, which is the only abstract method you need to implement. You also need to update the Status field in this method5. Maybecomputenext: Call Makenext gets to the next item if the status is done returns FALSE, otherwise returns true and the state is set to ready6. Alldone: Set the status to done and return null7. Resetstatus: As the name implies, the Reset Status field is Not_ready
Seven, Json.scalaa wrapper class for JSON that is used to convert JSON to string, which is not thread-safe. The JSON provided by Scala converts a numeric string into a double, but the class creates a simple function for converting a numeric string to an integer and specifying it as Json.globalnumberparser. There are only 2 methods of this class:1. Parsefull: Call Scala JSON's Parsefull method to convert a JSON string to an object and throw an exception if an error occurs2. Encode: Tells an object to encode into a JSON string. This object can only be one of null,boolean,string,number,map[string, t],array[t] or iterable[t], or it will be an error .
"Original" KAKFA Utils source Code Analysis (i)
