Python + numpy, theano, cifar

Operators in Python are determined based on the object type. If a and B are lists, A + B is the sum of two lists and a list,

If you want to perform elementwise operations to add the elements in a and B, you need to write code.

So I wrote a paragraph for code. The speed is really slow.

Numpy provides elementwise operations such as numpy. Add and numpy. multiply. The speed is much faster.

For the same operation, it takes 30 minutes to use the python for code and add a subscript for access.

Numpy takes seconds...

Numpy can also easily convert the array type, using numpy. astype or directly setting the type of the new object in numpy. asarray.

Very easy to use.

It seems that theano is targeted at neural networks and has optimized several operations than numpy, such as grad and scan.

Tensor content in theano is mainly processed by tensor, and the operation object is a symbol variable. For example, T. Grad processes an algebraic form and produces an algebraic form.

In the theano directory, the function Scan processes data. For example, the function defines the input data, output data, and calculation formula. It also processes symbol variables. Scan can be seen as a function for cyclic computing.

Like numpy, theano functions are also elementwise.

Function only defines the input and output types, and obtains a further formula, which is compiled into C code.

After the function is compiled, the C function can be called directly in Python. This is what we finally do.

Because multiple functions may use the same data block. Therefore, theano uses SHARED variables. One memory block can be used in multiple functions. Maybe (I guess) a function is a DLL file.

Theano has its own memory block. (The share variable in theano is hard to understand. Why shared? When shared? How borrow ?). It is clear now.

James on Google group gave me an explanation, which worked very well.

Good question. Shared vars were introduced to make GPU Programming
Easier, you're right about that. But they have another use too-
Shared variables allow update expressions in theano. function. Update
Expressions are a common pattern for iterative algorithms, and shared
Variables are automatically eligible for In-Place updates, which can
Be faster.

Borrow = false means you are guaranteed to get back something that is
Not aliased to any memory that will be changed by subsequent callto
A function. It's similar to numpy. Array (V. get_value (Borow = true ))
Snapshot t that in some cases (like when V is on GPU) No copy is
Necessary. Borrow = false in set_value also means that no theano
Function will change your value by side-effect.

 

The use of shared variables and Memory aliased technology can save memory and reduce memory copying. GPU operations are essential.

 

Theano's process of writing code is modeling, data, and testing. It is very suitable for scientific computing.

Of course, any problem is modeling, data, and testing. Only other languages are modeling, encoding, data, and testing. Theano supports symbol variables, so we feel that modeling is encoding. Easy to use.

Numpy does not use Symbolic variables.

Run logistic_sgd.py on cifar. The accuracy is very low. The speed is fast.

Optimization complete with best validation score of 77.708333%, with test performance 77.645833%
The code run for 75 epochs, with 2.071545 epochs/sec

 

Next, try MLP again. The MLP is not as effective as the logistic classifer above.

Epoch 52, minibatch 2000/2000, validation error 90.050000%
Epoch 53, minibatch 2000/2000, validation error 90.040000%
Epoch 54, minibatch 2000/2000, validation error 90.060000%
Epoch 55, minibatch 2000/2000, validation error 90.100000%

The best score for changing a computer is 85%.

CNN does not seem to be able to try it directly, because the image size is 32*32, not 29*29, and the convolution layer will fail. And then change it. Try again tomorrow.

CNN changed the convolution size of the first layer to adapt to cifar, with a score of 53%. CNN is awesome. featuremap feels more reasonable than DBN layers. DBN must support multiple featuremap and shared weights at the same layer.

RBM. When the loss function is computed, It is CD-15, which is really slow. It is changed to CD-1, which does not mean that CD-1 is also very useful. Wait for the weekend to start another CD-15.

DBN

PIL library, you can save the array as an image. It also supports numpy arrays.

IMG = Pil. image. fromarray ()

IMG. Save ('......')

Easy to use.