Deep Learning for NLP Learning translation notes (2)

Source: Internet
Author: User
Tags theano

<title>Deep Learning for NLP</title> Deep Learning-nlplecture 2:introduction to Teano

Enter link description here

Neural Networks can be expressed as one long function of vector and matrix operations.

(A neural network can be represented as a long function of a vector and a matrix operation.) )

Common frameworks (Common frame)
    • C + +
      If you are need maximum performance,start from scratch (and if you need the highest performance then start programming from the beginning:)
    • Matlab
    • Caffe
      Ported Matlab ' s implementation nets to C. (Caffe is a purely C++/cuda architecture that supports command line, Python, and Matlab interfaces)
      Mainly used for machine-vision (mainly for machine vision. )
    • Torch:
      Based on Lua, (based on Lua,)
      Used by lots of companies (Google deep MIND,FACEBOOK,IBM) (used by many companies, including Google depth brain, FACEBOOK,IBM, etc.)
    • Theano
      Python based framework (Python basic framework)
      Main framework used in the research community (key frame used by the study team)
      For comparison:http://fastml.com/torch-vs-theano/
Introduction to Theanoadvantage (pros)
    • Python Library with tight integration of Numpy (close integration of Python libraries Numpy)
      Easy syntax for matrix operations (simple matrix arithmetic syntax)
    • Transparent use of the GPU (speed-up of up to 140x) (Transparent GPU usage (acceleration 140x))
    • Efficient symbolic differentiation (Theano computes the gradient) (valid symbolic differential (Theano compute gradient))
    • Speed and stability optimizations (velocity and stability optimization)
    • Calculations is dynamically mapped to C code (compute dynamically maps to C codes)
      We do our computations as fast as we would had written it in C (we had the same fast speed as the program we wrote in C code)
      Great performance (>10 faster than Java in my experiments) (great performance (10 times times faster than in my experiment with Java))
Disadvantages (Cons)
    • Debugging is really hard (difficult to debug)
Some Note on the installation of Theano (some precautions for Theano installation)
    • Theano utilizes BLAS (basic Linear Algebra subprograms) (Theano using BLAS (subroutine of basic linear algebra))
      Building blocks for fast vector and matrix operations (building blocks for speed vectors and matrix operations)
      Often written in Fortran, sometimes in assembler (often written in Fortran, but sometimes assembler)
    • For performance optimization installation a BLAS package (installs a BLAS pack for optimized performance)
    • Benchmark different BLAS packages (different base of BLAS package)
    • I use a manually compiled Openblas implementation (I'm using a manual compile Openblas to implement)
      Installation notes:http://deeplearning.net/software/theano/install_ubuntu.html
Theano–flow (Theano flowchart)

The execution of a Theano script is a bit different (Theano scripts are executed slightly differently)

Theano-computation Graph (Theano calculation chart)

import theano
import theano.tensor as T

a=T.fvector()
b=T.fvector()
c=T.fmatrix()

res1=a+b
res2=T.dot(res1,c)

Deep Learning for NLP Learning translation notes (2)

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.