machine learning and neural networks

ImageNet classification with deep convolutional neural Networks reading notes(2013-07-06 22:16:36) reprint Tags: deep_learning imagenet Hinton Category: machine learning (after deciding to read a paper each time, the notes are recorded on the blog.) )This article, published in NIPS2012,

to a center. The activation function of the radial basis neural network is \vert as the ∥dist∥\vert of the distance between the input vector and the weight vector dist as the independent variable. activation function of radial neural network The general expression is R (∥dist∥) =e−∥dist∥2 R (\vert dist \vert) = E^{-\vert Dist \vert^2}With the decrease of the distance between the weights and the input vecto

takes Alpha as an argument and obtains its optimal value by learning from the data. The alpha of the randomized Relu is randomly selected within the specified interval and is fixed at the test stage. Some scholars have put the best two kinds of CNN networks together with different activation functions to do experiments on cifar-10,cifar-100 and NDSB datasets, and evaluate the pros and cons of four kinds of

Some methods of himself analysis (II.) will be supplemented in the future. --by weponCombined with the literature "deep Learning for computer Vision", here are some points of attention and questions about convolutional neural networks. The excitation function is to choose a nonlinear function, such as tang,sigmoid,rectified liner. In CNN, Relu is used more be

networks. 3 Local response normalization of LRN The Relu function does not need normalization to prevent saturation, and if no neuron produces a positive activation value, learning will occur in this neuron; however, the authors find that local normalization helps generalization. Normalized formula: General initialization parameter k=2,n=5, and, here's n is the number of neurons in each layer. 4) overlapp

Idea: Using RNN to model users ' browsing order, using FNN to simulate CF, two networks learning togetherRNN Network structure:The state of the output layer represents a page that a user browses, which can be seen as a one-hot representation, and STATE0 to 3 is the page that is browsed in turn. Because RNN input number is limited, if the user browses too many pages, then will lose the first of those pages,

ImageNet classification with deep convolutional neural Networks reading notes(after deciding to read a paper each time, the notes are recorded on the blog.) ）This article, published in NIPS2012, was Hinton and his students, in response to doubts about deep learning, used deep learning for imagenet, the largest database

weight update, is by a lot of weight multiplied, the smaller, a bit like the gradient disappears meaning (this sentence is I added) 8: If training rnn or LSTM, It is important to ensure that the norm of the gradient is constrained to 15 or 5 (provided that the gradient is first normalized), which is significant in RNN and lstm. 9: Check the gradient below, if it is your own calculation. 10: If you use LSTM to solve the problem of long-time dependencies, remember to initialize bias 12: As far as

characters of the license plate with optical character recognition. For each detected plate, we proceed to segment the plateFor each character, and use an artificial neural network (ANN) machine-Learning Algorithm to recognize the character. 1. OCR Segmentation First, we obtain a plate image patch as the input to the segmentation OCR function with an equalized h

of the word vector effect is also possible.Channel (Channels): An image can take advantage of (R, G, B) as a different channel, while the input channel of the text is usually a different way of embedding (such as Word2vec or glove), In practice, the use of static word vectors and fine-tunning-word vectors as different channel methods are also used.One dimensional convolution (conv-1d): The image is a two-dimensional data, the word vector expression of the text is one-dimensional data, so in tex

1 reasons for normalization of data using bnA) The essence of neural network learning process is to study the distribution of data, once the training data and test data distribution is different, then the network generalization ability is greatly reduced;b) on the other hand, once each batch of training data is distributed differently (batch gradient drops), then the network will be in each iteration to lea

Som is a unsupervised learning neural network, first affixed with a recently written simple application that uses the SOM to compress and restore images, leaving a pit: 1. Have time to summarize the concept of SOM, learn the process, and optimize the algorithm. 2. Re-implement the code again in Python and C + + as a programming exercise ...The training process is generally as follows:Decomposing the image a

nonlinearity of the network, but also maintain the sensation field of the previous layer, so it has a good effect on the detection of small objects. The original 5x5 convolution kernel is replaced by two 3x3 convolution cores, reducing the parameters, increasing the nonlinearity of the network and the module sensing field. Hypernet:concatenation of Multi-scale Intermediate outputs Hypernet the convolution level of different convolution stages, it has a good effect on the detection