Spark Rdd class Source Learning (not finished)

Make a little progress every day ~ open it up ~

Abstract classRdd[t:classtag] (//@transient annotations indicate that a field is marked as transient.@transient Privatevar _sc:sparkcontext,//seq is a sequence in which elements have the order of insertion and can have duplicate elements. @transient Privatevar deps:seq[dependency[_]])extendsSerializable with Logging {if(Classof[rdd[_]].isassignablefrom (elementclasstag.runtimeclass)) {User programs that}//This should be a call to declare the Sparkcontext object before using the Rdd  Privatedef sc:sparkcontext = {    if(_SC = =NULL) {      Throw NewSparkexception ("RDD transformations and actions can only is invoked by the driver, not inside of" + "transformations; For example, rdd1.map (x = Rdd2.values.count () * x) is invalid because "+" the values transformation and count Action cannot be performed inside of the Rdd1.map "+" transformation. For more information, see SPARK-5063. ")} _SC}//building an RDD should be a one-to-ones relationship, such as a child rdd corresponding to the only parent RddDef This(@transientOneparent:rdd[_]) = This(Oneparent.context, List (Newonetoonedependency (oneparent))) Private[Spark] def conf:sparkconf =_conf//Settings for sparkconfdef getconf:sparkconf =Conf.clone ()//get the appropriate configuration informationdef jars:seq[string] =_jarsdef Files:seq[string]=_filesdef master:string= _conf.get ("Spark.master") def appname:string= _conf.get ("Spark.app.name")Private[Spark] def Iseventlogenabled:boolean = _conf.getboolean ("spark.eventLog.enabled",false)  Private[Spark] def eventlogdir:option[uri] =_eventlogdirPrivate[Spark] def eventlogcodec:option[string] =_eventlogcodec//the name of the Temp folder is spark+ random timestampVal externalblockstorefoldername = "spark-" +randomuuid.tostring ()//determine if the local modedef Islocal:boolean = (Master = = "Local" | | master.startswith ("local["))

Spark Rdd class Source Learning (not finished)