Generate Combat Network Gan (ii) speech-related _ neural network

Multi-Task confrontation learning [1]

In order to gain robustness against noise, multi-task learning is introduced into three networks:
-Input Network (green), used as feature extractor
-Senone output Network (red), used as Senone classification
-Domain output Network (blue), domain here refers to the type of noise, a total of 17 kinds of noise

In order to increase the robustness of the noise, increased the GRL layer (gradient reversal layer), the network in the reverse transmission, the domain network over the gradient took −α, that is, increase the error rate of noise classification, In order to obtain the characteristics of senone-discriminative domain-invariant.
[2] and [1] are similar in thought. SEGAN[3]

Mainly used for speech enhancement (such as noise reduction) and so on.
Combined with conditional Gan and Lsgan, the L1 norm is used, and the final loss are as follows:
Mindvlsgan (d) =12ex∼pdata (X,XC) [(d (X,XC) −1) 2]+12exc∼pdata (XC), Z∼pz (z) [D (G (Z,XC)) 2]
Mingvlsgan (g) =12ex∼pdata (XC), Z∼pz (z) [(D (g (Z,XC)) −1) 2]+λ∥g (z,x~) −x∥1
Some of the parameters have the following meanings:
X:noise speech
Xc:clean speech
Z: Noise samples that obey the normal distribution

The training process is as follows:

Clean speech and noisy speech pair are required to maintain the original voice information while removing noise. Reference documents

[1]. Adversarial multi-task Learning of Deep neural Networks for robust Speech recognition
[2]. Invariant representations for noisy Speech recognition
[3]. Segan:speech Enhancement Generative Adversarial network