Generate a confrontation network-adversarial Examples against samples

[1] intriguing properties of neural networks

It is found that by adding some imperceptible non-random disturbances in the original image, the predicted results of the network can be greatly changed. These disturbances are trained to gain input by maximizing the predictive error, which is called a confrontation sample.

The confrontation example is relatively robust, that is, the confrontation sample generated by the neural network A, which is still in the network B, even if B is a different structure, parameter, and training data. Therefore, the neural network contains some intrinsic blind spots and non-display features, and its structure is related to the data distribution.

In order to explain the meaning of activation in the network, some researchers tried some training images to make the activation value of these features reach the maximum visual detection (Visual inspection). The study found that the difference between the factors is across the entire network space, rather than a unit of the network. The following figure. In fact, in the complex expression of deep neural network, the method of visual detection of unit-level local is worse than that of network-level global.

Here's a call for partial generalization (local generalization). A local generalization is that there is a higher recognition accuracy in the field of the training sample in the input space, that is, the imperceptible non-random disturbance does not affect the prediction result, which is called smoothing (smoothness) hypothesis. However, deep neural networks encode the input space by stacking nonlinear elements, which are non-local generalization , which means that the field of training samples in the input space is not able to obtain a high recognition probability, although these areas do not contain training samples, But it also expresses the same sample at different angles or distances. With this non-smoothing, we can use the optimization method to find the confrontation sample in the input space.

This confrontation sample is difficult to obtain by random sampling in the field of input space. This paper presents a method to find out the defects in modeling the local space around the training data, and proves that there is a large disturbance in the final output layer caused by the disturbance of small input space . This method is similar to the difficult-to-mistake sample mining (hard-negative mining), that is, the collection of classifier error large samples, so that the input distribution of further iterations to focus on the distribution of these difficult samples. By minimizing the distortion function d, such as the following formula, to obtain adversarial examples, the formula x represents the input image, l means that Label,x+r is the closest to X and the function f is classified as L image.

We can see that adversarial examples did deceive the network by getting adversarial examples under different training data and networks, and its errors in the test set are as follows.

Todo