Model-driven deep learning (admm-net)

Flow: Model Family----depth network----depth learning

Model family: The model has a super-parameter, and gives different parameters corresponding to different models, it forms the model family

Algorithm family: Each model corresponds to a complete algorithm, the entire model family corresponds to an algorithm family to expand the algorithm family into a deep network, the network layer represents the number of iterations, the model's hyper-parameters become the parameters of the network (such as weight, etc.). The network can be trained with a small amount of tagged data.

Advantages relative to model-driven algorithms:

1, can learn the model super parameter, improve the adaptability of the model, improve the accuracy

Relative to data-driven benefits:

1, the network design has the model guidance

2, reduce the data demand

3, reduce the training time

For example, the ADMM algorithm for NMR reconstruction:

Model:

\ (x^*={arg\max}_{x}{\{\frac{1}{2}| | ax-y| | ^2+\sum_{l=1}^{l}\lambda_{l}g (d_{l}x) \}}\)

ADMM Algorithm Solution:

The different choices of \ (g,\lambda,l,d_{l}\) Form different models and constitute the model family.

Generalized Lagrangian functions:

ADMM algorithm iterative update process:

(\beta_{l}=\frac{\alpha_{l}}{\rho_{l}},a=pf\) (known), can be

\ (S (\cdot) \) is a nonlinear shrinkage function. \ (S (\cdot) \) is usually a smooth function.

Network structure:

including the reconstruction Layer \ (x^{(n)}\), convolutional layer \ (c^{(n)}=d_{l}x^{(n)}\), nonlinear transformation layer \ (z^{(n)}\), multiply sub-update layer \ (m^{(n)}\), where the nonlinear transformation function \ (s\) can be approximated by piecewise linear functions, Just learn the value of the function of the interpolation point.

Network Learning Parameters: Hyper-parameters in the model family, each layer can be different.

Network training:

The loss function is

Gradient Descent method training.

Reference documents:

Yangyan,sunjian,lihuibin,xuzongben, deep admm-net for compressive sensing MRI (NIPS2017)

https://arxiv.org/abs/1705.06869

Model-driven deep learning (admm-net)