GPU coarse-grained parallel implementation and testing (optimization) for convolution operations

GPU coarse-grained parallel implementation and testing (optimization) for convolution operations

A Boundary expansion;

B The word block is aligned.

Matrix Size	Number	Kernel	CPU (s)	Cpu2gpu	Gpu-kernel	Gpu2cpu
5x4	1	5x4	<1ms	<1ms	<1ms	<1ms
12x9	1	5x4	<1ms	<1ms	<1ms	<1ms
18x19	1	5x4	<1ms	<1ms	<1ms	<1ms
118x29	1	5x4	<1ms	<1ms	<1ms	<1ms
138x59	1	5x4	<1ms	<1ms	<1ms	<1ms
158x159	1	5x4	0.005	<1ms	<1ms	<1ms
558x559	1	5x4	0.041	<1ms	0.001	<1ms
1128x1159	1	5x4	0.156	0.002	0.003	0.002
2128x2159	1	5x4	0.514	0.007	0.011	0.007
5128x5159	1	5x4	2.341	0.038	0.062	0.037
18128x4159	1	5x4	6.574	0.111	0.177	0.114
10128x11159	1	5x4	10.007	0.170	0.266	0.156
					17.04Gflps	1.44GBps
5x4	1	14x15	~	~	~	~
12x9	1	14x15	~	~	~	~
18x19	1	14x15	<1ms	<1ms	<1ms	<1ms
118x29	1	14x15	0.003	<1ms	<1ms	<1ms
138x59	1	14x15	0.011	0.001	<1ms	<1ms
158x159	1	14x15	0.028	<1ms	<1ms	<1ms
558x559	1	14x15	0.343	<1ms	0.006	<1ms
1128x1159	1	14x15	1.289	0.002	0.023	0.003
2128x2159	1	14x15	3.929	0.007	0.081	0.007
5128x5159	1	14x15	21.869	0.041	0.467	0.041
11128x4159	1	14x15	39.2	0.072	0.819	0.066
10128x11159	1	14x15	93.912	0.161	1.999	0.195
					23.71Gflps	372.86MBps
5x4	15	14x15	~	~	~	~
12x9	15	14x15	~	~	~	~
18x19	15	14x15	0.001	<1ms	<1ms	<1ms
118x29	15	14x15	0.003	<1ms	0.001	<1ms
138x59	15	14x15	0.099	0.001	0.002	<1ms
158x159	15	14x15	0.367	0.001	0.006	0.001
558x559	15	14x15	3.856	0.006	0.084	0.008
1128x1159	15	14x15	15.98	0.030	0.348	0.031
2128x2159	15	14x15	57.527	0.096	1.231	0.107
3058x2659	15	14x15	100.355	0.171	2.169	0.202
5128x5159	15	14x15	Pointer overflow
11128x4159	15	14x15
10128x11159	15	14x15
					23.39Gflops	366.07MBps

Analysis:

From the above table, the maximum throughput rate is 1.44gbps,pcie bus bandwidth of 5GBps, there is a certain amount of space. Single-precision floating-point multiplication is the highest effective performance of 23.71Gflops, a little bit higher than the previous 23.23Gflops, the single-precision floating point of the highest performance is still very different from the device, the reasons for analysis are as follows:

A. The data that the CPU transmits to the GPU is a one-dimensional array, and the internal GPU is computed in two dimensions, and it takes a lot of address computation to access the data.

B. When calculating a single convolution result within a thread, a two-dimensional loop is used, and when the convolution core is large, the operation time is exponentially increased (the main reason).

C. CPU and GPU data are stored continuously, when using the malloc () function to request a large amount of memory, the use of pointers to access data memory overflow phenomenon, resulting in the inability to test the larger volume of data, so also can not play GPU performance.

D. In order to simplify the operation of the batch convolution, the practice is to combine multiple images into a larger image, as a two-dimensional image, and then convolution, in the convolution, do a lot of extra convolution, in the results will be written, the need to eliminate, so added some branch operations.

Summarize:

1. This is a coarse-grained parallel implementation of convolution, and this week also implements a fine-grained parallel version, each block has a 16x16 thread, each block corresponds to a convolution operation, a thread corresponds to a multiplication operation, each block is about a convolution result. However, when testing, it was found that the version was slower. The reason may be that each block of a convolution, matrix and convolution kernel must be placed in global memory, access to global memory is much slower than access to shared memory, and the data reusability is low, resulting in inefficient.

2. Batch convolution, the use of three-dimensional block processing, you can avoid redundant convolution and write back the conditional branch, but in the CPU and GPU, the data is continuous storage, while the expansion of multiple images of the boundary is difficult, not yet found the ideal solution.

3. Coarse-grained and fine-grained are operations-level parallelism, and algorithmic-level parallelism is not yet implemented. The so-called algorithm-level parallelism, refers to the optimization of the convolution algorithm itself, the two-dimensional convolution into other space of the point multiplication operation, the efficiency should be higher.

GPU coarse-grained parallel implementation and testing (optimization) for convolution operations