Spark MLlib Deep Learning convolution neural network (depth learning-convolutional neural network) 3.3

3. Spark MLlib Deep Learning convolution neural network (depth learning-convolutional neural network) 3.3

Http://blog.csdn.net/sunbow0

Chapter III Convolution neural Network (convolutional neural Networks)3 Example3.1 test Data

Follow the above example data, or create a new image recognition data.

3.2 CNN Example

??? //2 test Data

??? Logger.getRootLogger.setLevel (level. WARN)

??? Val Data_path="/user/tmp/deeplearn/train_d.txt"

??? Val examples=sc. Textfile (data_path). Cache ()

??? Val train_d1=examples. map{line =

????? Val f1 = Line.split ("\ t")

????? Val f =F1. map (f =f. ToDouble)

????? Val y =f. Slice (0,ten)

????? Val x =f. Slice (f. Length)

????? (newBDM (1,y. Length,y), (new BDM (1,x.length,x ). Reshape (+)/255.0)

???}

??? Val Train_d=train_d1. Map (F=> (f._1, F._2))

?

??? //3 set the training parameters. Building a model

??? //opts: iteration Length, iteration count, cross-validation scale

??? Val opts= Array (100.0,1.0,0.0)

??? Train_d. Cache

??? Val numexamples=train_d. Count ()

??? println (S"numexamples = $numexamples.")

??? Val Cnnmodel=newCNN ().

????? Setmapsize (new BDM (1,2, Array (28.0,28.0))).

????? Settypes (Array ("i", "C","s","C","S")).

????? Setlayer (5).

????? Setonum (ten).

????? Setoutputmaps (Array (0.0, 6.0,0.0,12.0,0.0)).

????? Setkernelsize (Array (0.0, 5.0,0.0,5.0,0.0)).

????? Setscale (Array (0.0, 0.0,2.0,0.0,2.0)).

????? Setalpha (1.0).

????? Setbatchsize (50.0).

????? Setnumepochs (1.0).

????? Cnntrain (train_d,opts)

?

??? //4 Model Test

??? Val cnnforecast=Cnnmodel. Predict (train_d)

??? Val cnnerror=Cnnmodel. Loss (cnnforecast)

??? println (S"Nnerror = $cnnerror.")

??? Val printf1=cnnforecast. Map (F=> (F.label ). Data(0), F.Predict_label. Data (0)). Take ($)

??? println (" predicted result - actual value: Pre-measured value: Error ")

??? for (i <- 0 untilPRINTF1 . Length)

????? println (printf1 ( _1 + \t " + I ). _2 + \t + (printf1 (). _2 -printf1 ( i ). _1 ))??? val numexamples = train_d . Count ()

??? println (S"numexamples = $numexamples.")

??? println (Mynn. _2)

??? for (i <-0 toMynn. _1. Length-1) {

????? Print (Mynn. _1(i) +"\ t")

???}

??? println ()

??? println ("MYNN_W1")

??? Val tmpw1=Mynn. _3 (0)

??? for (i <-0 totmpw1. rows -1) {

????? for (J <-0 totmpw1. cols -1) {

??????? Print (tmpw1(i,J) +"\ t")

?????}

????? println ()

???}

??? Val Nnmodel=newneuralnet ().

????? SetSize (Mynn. _1).

????? Setlayer (Mynn. _2).

????? Setactivation_function ("Sigm").

????? Setoutput_function ("Sigm").

????? SETINITW (Mynn. _3).

????? Nntrain (train_d,nnopts)

?

??? //5 NN Model Test

??? Val nnforecast=Nnmodel. Predict (train_d)

??? Val nnerror=Nnmodel. Loss (nnforecast)

??? println (S"Nnerror = $nnerror.")

??? Val printf1=nnforecast. Map (F=> (F.label ). Data(0), F.Predict_label. Data (0)). Take ($)

??? println (" predicted result - actual value: Pre-measured value: Error ")

??? for (i <-0 untilprintf1. Length)

????? println (printf1(i). _1 +"\ t" +printf1(i). _2 +"\ t" + (printf1(i). _2 -printf1(i). _1))?

Reprint please specify the source:

Http://blog.csdn.net/sunbow0

?

?

?

Spark MLlib Deep Learning convolution neural network (depth learning-convolutional neural network) 3.3