Python Spark Kmeans Demo

The official Demo

 fromNumPyImportArray fromMathImportsqrt fromPysparkImportSparkcontext fromPyspark.mllib.clusteringImportKmeans, KMEANSMODELSC= Sparkcontext (appname="Clusteringexample")#Load and parse the datadata = Sc.textfile ("/root/spark-2.1.1-bin-hadoop2.6/data/mllib/kmeans_data.txt") Parseddata= Data.map (LambdaLine:array ([Float (x) forXinchLine.split (' ')]))#Build the model (cluster the data)clusters = Kmeans.train (Parseddata, 2, maxiterations=10, initializationmode="Random")#Evaluate clustering by computing within Set Sum of squared Errorsdeferror (Point): Center=clusters.centers[clusters.predict (point)]returnsqrt (SUM ([x**2 forXinch(Point-Center)]) Wssse= Parseddata.map (LambdaPoint:error (point)). Reduce (LambdaX, Y:x +y)Print("within Set Sum of squared Error ="+str (wssse))#Save and load Model#Clusters.save (SC, "Target/org/apache/spark/pythonkmeansexample/kmeansmodel")#Samemodel = Kmeansmodel.load (SC, "Target/org/apache/spark/pythonkmeansexample/kmeansmodel")

Python Spark Kmeans Demo