<title>Deep Learning for NLP</title> Deep Learning-nlplecture 2:introduction to Teano
Enter link description here
Neural Networks can be expressed as one long function of vector and matrix operations.
(A neural network can be represented as a long function of a vector and a matrix operation.) )
Common frameworks (Common frame)
- C + +
If you are need maximum performance,start from scratch (and if you need the highest performance then start programming from the beginning:)
- Matlab
- Caffe
Ported Matlab ' s implementation nets to C. (Caffe is a purely C++/cuda architecture that supports command line, Python, and Matlab interfaces)
Mainly used for machine-vision (mainly for machine vision. )
- Torch:
Based on Lua, (based on Lua,)
Used by lots of companies (Google deep MIND,FACEBOOK,IBM) (used by many companies, including Google depth brain, FACEBOOK,IBM, etc.)
- Theano
Python based framework (Python basic framework)
Main framework used in the research community (key frame used by the study team)
For comparison:http://fastml.com/torch-vs-theano/
Introduction to Theanoadvantage (pros)
- Python Library with tight integration of Numpy (close integration of Python libraries Numpy)
Easy syntax for matrix operations (simple matrix arithmetic syntax)
- Transparent use of the GPU (speed-up of up to 140x) (Transparent GPU usage (acceleration 140x))
- Efficient symbolic differentiation (Theano computes the gradient) (valid symbolic differential (Theano compute gradient))
- Speed and stability optimizations (velocity and stability optimization)
- Calculations is dynamically mapped to C code (compute dynamically maps to C codes)
We do our computations as fast as we would had written it in C (we had the same fast speed as the program we wrote in C code)
Great performance (>10 faster than Java in my experiments) (great performance (10 times times faster than in my experiment with Java))
Disadvantages (Cons)
- Debugging is really hard (difficult to debug)
Some Note on the installation of Theano (some precautions for Theano installation)
- Theano utilizes BLAS (basic Linear Algebra subprograms) (Theano using BLAS (subroutine of basic linear algebra))
Building blocks for fast vector and matrix operations (building blocks for speed vectors and matrix operations)
Often written in Fortran, sometimes in assembler (often written in Fortran, but sometimes assembler)
- For performance optimization installation a BLAS package (installs a BLAS pack for optimized performance)
- Benchmark different BLAS packages (different base of BLAS package)
- I use a manually compiled Openblas implementation (I'm using a manual compile Openblas to implement)
Installation notes:http://deeplearning.net/software/theano/install_ubuntu.html
Theano–flow (Theano flowchart)
The execution of a Theano script is a bit different (Theano scripts are executed slightly differently)
Theano-computation Graph (Theano calculation chart)
import theano
import theano.tensor as T
a=T.fvector()
b=T.fvector()
c=T.fmatrix()
res1=a+b
res2=T.dot(res1,c)
Deep Learning for NLP Learning translation notes (2)
