caffemodel的剪枝與壓縮(一)_代碼

caffemodel裡每一層的卷積核維度為ouput-channel×input-channel×height×weight. 本篇部落格是針對每一個維度為input-channel×height×weight的卷積核, 計算出ouput-channel×input-channel個權重和,小於給定閾值的認為低效串連,給予置零操作.代碼寫的比較粗糙,注釋給的較為詳細,無需多言.

# coding:utf-8# by chen yhimport caffeimport numpy as npimport matplotlib.pyplot as pltdef weight_0(prototxt,model,layer,threshold):    caffe.set_mode_gpu()    net=caffe.Net(prototxt,model,caffe.TEST)    weight = net.params[layer][0].data    bias = net.params[layer][1].data    sum_l1=[]    for i in range(weight.shape[0]):        for j in range(weight.shape[1]):            sum_l1.append((i,j,np.sum(abs(weight[i,j,:,:]))))#i是核的順序,j是每個卷積核與前面某個channel的串連順序,求出每個串連的類似於L1範數的權重和,加上i,j是為了後續判斷weight的時候好直接處理到原weight    display(sum_l1,128)#從小到大排序後列印出前128個L1範數    l1_plot(sum_l1)#畫出L1範數關於out*input的座標圖,以確定多少個需要修剪.    weight_l1 = []    for i in sum_l1:        weight_l1.append(i[2]) #得到僅含有l1範數的列表    for i,weight_sum in enumerate(weight_l1):        if weight_sum < threshold:            out_channel_sort = sum_l1[i][0]            input_channel_sort = sum_l1[i][1]            weight[out_channel_sort, input_channel_sort, :, :] = 0 #小於閾值的,weight置0    net.save("new.caffemodel")def l1_plot(weight_l1):    weight_l1_n=[]    for i in weight_l1:        weight_l1_n.append(i[2])    weight_l1_n.sort()    x=[i for i in range(len(weight_l1_n))]    plt.plot(x,weight_l1_n)    plt.legend()    plt.show()def display(weight_l1,threshold):    weight_l1_n=[]    for i in weight_l1:        weight_l1_n.append(i[2])    weight_l1_n.sort()    print [weight_l1_n[i] for i in range(threshold)]root =  "/home/cyh/python_file/"prototxt = root+ "deploy.prototxt"model = root + "VGG_coco_SSD_300x300_iter_400000.caffemodel"weight_0(prototxt,model,'fc7',0.0001)

說明: 1.代碼中L1範數並不是嚴格的L1範數,只是每個weight*height的絕對值的和.

2.代碼僅僅實現了weight置0的步驟.如果希望減小儲存空間,需將該疏鬆陣列儲存為CSC或者CSR格式,可以移步看一下這個連結,有類似的原理介紹也有代碼,這個實現應該不難.

3.本文代碼和deep compression意義 是不一樣的,一是論文是以一定的比率修剪,我是以閾值修剪;二是論文好像是修剪單個權重,而我是以weight*height為單位,但是相同的是都不能提高forward inference時間, 因此所有後續的儲存,retrain等等我也沒有繼續,如果有做的小夥伴希望和我分享一下. 

明天有空的話我會注釋一下我寫的channel pruning的py檔案,然後再貼出來,這種方式就可以加快inference的時間了,而且也不用注意儲存等問題.

代碼給出連結

channel pruning 的py檔案已上傳,見連結