convolutional neural network theory

parameter. In practice, it is acceptable to spend a lot of computational resources in training to get a compact model that can quickly process new data.Next we will see that in order to get the parameters of the neural network, we essentially do a maximum likelihood estimation, which involves nonlinear optimization problems. This requires a log likelihood function for the parameter derivation number, we wi

Origin: The human visual cortex of the MeowIn the 1958, a group of wonderful neuroscientists inserted electrodes into the cat's brain to observe the activity of the visual cortex. and infer that the biological vision system starts from a small part of the object, After layers of abstraction, it is finally put together into a processing center to reduce the suspicious nature of object judgment. This approach runs counter to BP's network.The BP network

absrtact : This paper will analyze the basic principle of deep neural network to recognize graphic images in detail. For convolutional neural Networks, this paper will discuss in detail the principle and function of each layer in the network in the image recognition, such as

Main reference: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ RNN (recurrent neuralnetworks, cyclic neural network) For a common neural network, the previous information does not have an impact on the current understanding, for example, reading an article, we need to use the vocabulary learned before, and t

convolution layer of the error-sensitive items, because the reverse propagation when the output is smaller than the input, so the gradient at the time of transmission and traditional BP algorithm, So how to get the error-sensitive item of convolutional layer is the problem to consider. The third problem is to consider the pooling layer below the convolution layer, this is because we want to get the pooling layer error sensitivity, relying on the conv

visual comprehension of convolutional neural networks The first to suggest a visual understanding of convolutional neural Networks is Matthew D. Zeiler in the visualizing and understanding convolutional Networks. The following two blog posts can help you understand this a

ImageNet classification with deep convolutional neural Networks reading notes(after deciding to read a paper each time, the notes are recorded on the blog.) ）This article, published in NIPS2012, was Hinton and his students, in response to doubts about deep learning, used deep learning for imagenet, the largest database of image recognition, and eventually achieved very surprising results, The result is much

. Pretreatment: Mean removal;whitening (ZCA) Enhanced generalization capability: Data augmentation;weight regularization; adding noise to the network, including dropout,dropconnect,stochastic pooling. Dropout: The output of some neurons in the fully connected layer is randomly set to 0 at the full connection layer only. Dropconnect: Also only used on the full-connection layer, Random binary mask on weights. Stochastic Pooli

absrtact : This paper will analyze the basic principle of deep neural network to recognize graphic images in detail. For convolutional neural Networks, this paper will discuss in detail the principle and function of each layer in the network in the image recognition, such as

conclude that the problem it is best at solving is related to the time series. RNN is also the most natural neural network structure for dealing with such problems. The principal structure of a RNN is duplicated several times by the time series, and structure A is also called the loop body. How to design the network structure of loop body A is the key to solve

1, IntroductionDL solves VO problem: End-to-end vo with RCNN2. Network structureA.CNN based Feature ExtractionThe paper uses the Kitti data set.The CNN section has 9 convolutional layers, with the exception of CONV6, the other convolutional layers are connected to 1 layers of relu, and there are 17 layers.B, RNN based sequential modellingRNN is different from CNN

0-Background The so-called style conversion is based on a content image and a style image, merging the two, creating a new image that combines both contents and style.The required dependencies are as follows: Import OS import sys import scipy.io import scipy.misc import Matplotlib.pyplot as Plt from Matplotlib.pyplot import imshow from PIL import Image from nst_utils import * import NumPy as NP import te Nsorflow as TF %matplotlib inline 1-transfer Learning Migration learning is the applicat

redundant and unimportant parameters. Based on the method of low rank decomposition (Low-rank factorization), matrix/tensor decomposition is used to estimate the most informative parameters in deep CNN. Based on the migration/compression convolution filter (Transferred/compact convolutional filters) method, a special structure convolution filter is designed to reduce the complexity of storage and computation. Knowledge refinement (knowledge distillat

set, the KL distance is the indicator that describes the diversity, thus reducing the amount of computation. Traditional deep learning will need to do before the training of data enhancement, each sample is equal; This article contains some data enhancement not only does not play a good role, but brings the noise, it needs to do some processing, but also some of the data does not need to be enhanced, which reduces noise and saves calculation. Qa Q: Why did the active learning not b

Wang, Min, Baoyuan Liu, and Hassan Foroosh. "Factorized convolutional neural Networks." ArXiv preprint (2016). This paper focuses on the optimization of the convolution layer in the deep network, which has three unique features:-Can be trained directly . You do not need to train the original model first, then use the sparse, compressed bits and so on to compress.

of the "object" in the "the position with the maximum score Use a cost function this can explicitly model multiple objects present in the image. Because there may be many objects in the graph, the multi-class classification loss is not applicable. The author sees this task as multiple two classification questions, loss function and classification score as followsTrainingMuti-scale TestExperimentClassification MAP on VOC test: +3.1% compared with [56] MAP on VOC test: +7.

. We use the cublas. lib and curand. Lib libraries. One is matrix calculation and the other is random number generation. I applied for all the memory I needed at one time. After the program started running, there was no data exchange between the CPU and GPU. This proved to be very effective. The program performance is about dozens of times faster than the original C language version (if the network is relatively large, it can reach a speed-up ratio of

Minimalist notes Deepid-net:object detection with deformable part Based convolutional Neural Networks Paper Address Http://www.ee.cuhk.edu.hk/~xgwang/papers/ouyangZWpami16.pdf This is the CUHK Wang Xiaogang group 2017 years of a tpami, the first hair in the CVPR2015, increased after the experiment to cast the journal, so the contrast experiment are some alexnet,googlenet and other early

= Sess.run ([Me Rged, Accuarcy], feed_dict=feed_dict (False)) Test_writer.add_summary (summary, i) print (' Accuracy a T step%s:%s '% (I, ACC)) else:if i%100 = = 99:continue Run_options = Tf. Runoptions (TRACE_LEVEL=TF. Runoptions.full_trace) Run_metadata = tf. Runmetadata () Summary, _ = Sess.run ([merged, Train_step], feed_dict=feed_dict (True), Options=run_options, Run_metadata=run_metadata) train_writer.add_run_metadata (run_metadat A, ' step%03d '%i) t

ImageNet classification with deep convolutional neural Networks reading notes(2013-07-06 22:16:36) reprint Tags: deep_learning imagenet Hinton Category: machine learning (after deciding to read a paper each time, the notes are recorded on the blog.) )This article, published in NIPS2012, is Hinton and his students are using deep learning in response to doubts about deep learn