"Spark mllib crash book" model 02 Logistic regression "Logistic regression" (Python version)

Catalog Logistic regression principle Logistic regression code (Spark Python)

Logistic regression principle

See blog: http://www.cnblogs.com/itmorn/p/7890468.html

Back to Catalog

Logistic regression code (Spark Python)

code data:https://pan.baidu.com/s/1jHWKG4I Password: acq1

#-*-coding=utf-8-*- fromPysparkImportsparkconf, SPARKCONTEXTSC= Sparkcontext ('Local') fromPyspark.mllib.classificationImportLogisticregressionwithlbfgs, Logisticregressionmodel fromPyspark.mllib.regressionImportLabeledpoint#Load and parse the data to load and parse it, converting each number to a floating point. The first number of each line as a marker, followed by a featuredefParsepoint (line): Values= [Float (x) forXinchLine.split (' ')]    returnLabeledpoint (Values[0], values[1:]) Data= Sc.textfile ("Data/mllib/sample_svm_data.txt")PrintData.collect () [0]#1 0 2.52078447201548 0 0 0 2.004684436494304 2.00034729926846 .....Parseddata =Data.map (parsepoint)PrintParseddata.collect () [0]#(1.0,[0.0,2.52078447202,0.0,0.0,0.0,2.00468 ....#Build ModelModel =Logisticregressionwithlbfgs.train (parseddata)#evaluating the model on training data evaluates the error on the training setLabelsandpreds = Parseddata.map (LambdaP: (P.label, Model.predict (p.features))) Trainerr= Labelsandpreds.filter (LambdaLP:LP[0]! = lp[1]). COUNT ()/Float (parseddata.count ())Print("Training Error ="+ str (TRAINERR))#Training Error = 0.366459627329#Save and load model saving models and loading modelsModel.save (SC,"Pythonlogisticregressionwithlbfgsmodel") Samemodel= Logisticregressionmodel.load (SC,"Pythonlogisticregressionwithlbfgsmodel")PrintSamemodel.predict (Parseddata.collect () [0].features)#1

Back to Catalog

"Spark mllib crash book" model 02 Logistic regression "Logistic regression" (Python version)