Cuda: supercomputing for the masses (Super computing for large amounts of data)-Section 7

Section 7: Happy acceleration with the next generation of Cuda hardware

Original article link

Rob Farber is a senior researcher at the National Laboratory of the Pacific Northwest. He studied large-scale parallel operations in multiple national laboratories and was a partner of several new startups. You can send an email to [email protected] to communicate with him.

In section 6 of a series of articles on Cuda (compute unified device architecture, short for computing unified device architecture), I have discussed the use of Cuda profiler to detect global memory. In this section, I will look at the next generation of Cuda hardware.

Devices started with Cuda and Cuda have developed together, and each generation of the new product provides better performance. NVIDIA's recently introduced geforce 200-series and Tesla10-series products, proving the rapid pace of development, similar to the price of previous-generation products, but doubling its functionality. In addition, the 200 series has added some valuable (and indispensable) new features.

It turns out that g80 contains 0.698 billion transistors and gtx280 contains 1.4 billion transistors, which clearly shows that NVIDIA's next-generation Cuda hardware is twice the original hardware. My experience shows that NVIDIA does use these transistors well in the 200 series. Compared with the previous generation g80 hardware, my single-precision Code does not change, and the running speed on the new hardware is almost twice that of the original one.

The following describes the new features and functions of the 200-series architecture:

• The hardware now supports double-precision arithmetic operations (30 64-bit floating point units in gtx280 ).

• The global memory is larger, faster, and easier to use. The combination rules are slightly loose, making it easier to obtain high global memory performance. The global memory bandwidth is higher than 100 Gb/s, almost twice the size of the g80 architecture.

• The number of single-precision registers for each hardware thread has doubled (although the number of available registers does not change compared to the previous architecture ).

• 200-series shared storage supports 32-bit atomic signed and unsigned integer functions, and global storage supports 64-bit functions.

• 200-series now contains the warp vote function.

• The number of thread processors almost doubles-from 128 to 240-each multi-processor supports more active warp and active threads.

• The 200-series hardware feature has been improved to execute mad and Mul at the same time, helping some applications achieve better peak performance.

Let's further look at what this means for Cuda
For many applications, the most important new feature is hardware double-precision arithmetic operations. Nvidiagpus achieves super Computation for a large amount of data at a speed and large-scale concurrency. When dealing with super-large problems and datasets, digital noise can be quickly accumulated (due to inaccurate floating point), leading to spam results. For example, a physical simulation can suddenly display astonishing non-physical behaviors, and previous effective simulations may become unstable and start to generate infinite values of Nan or other meaningless values. Although not all computing problems are necessary, it is helpful to use a more precise floating point representation (such as a 64-bit double-precision floating point number. (In future articles, we will introduce how to quickly combine Single-precision and double-precision computing to accelerate the results .)

The 200-series architecture is the first architecture that includes hardware dual-precision. As expected for the first generation of products, we reserve some space for performance improvement (because the thread processors in the multi-processor all share a single dual-precision hardware unit ). My experience shows that dual precision is used only when you need to retain the numerical accuracy, and the performance will be slightly reduced. Of course, the problem varies depending on the situation.

Most programmers will find it easier to get the high performance of the 10-series architecture.

After NVIDIA doubles the number of registers, Cuda programmers are more likely to upload enough single-precision data to registers to reduce (or possibly) the majority of global memory bottlenecks. Because the dual-precision value needs to double the single-precision storage space (8 bytes to 4 bytes), the number of dual-precision registers on 200-series is consistent with the number of single-precision registers on the g80 and G92 architectures.

The bandwidth of global storage is almost doubled.

The 10 series motherboard provides a global memory bandwidth higher than 100 Gb/s. For 32-bit floating-point numbers, compared with the g80 and G92 architectures, the global memory bandwidth in the 10 series has fallen behind because the number of thread processors has almost doubled, as shown in table 1 below.

Architecture/card global memory bandwidth (GB/s) processing units the number of 32-bit operands available for each processing unit in the global memory per second (bandwidth only, unit: million)

G80/8800 ultra 104 128 203

10-series/GTX 280 141 240 146

Dual-precision (64-bit floating point number) performance is complicated because each multi-processor (eight-thread processor) has a dual-precision unit, doubling it will consume twice the bandwidth to cope. 200-series Cuda developers may find that applications with limited bandwidth when switching to dual-precision when using single-precision floating point numbers may become computing Limited.

This table confirms that the NVIDIA hardware design team did a great job because they provided a new product that was at the same price level as the previous generation, its global memory bandwidth and dual-precision functions almost doubled.

One of the strengths of Cuda applications is that the 200-series aggregation rules become loose, so you can access the global memory in a more efficient way. Section 5.2.2 of the Cuda Programming Guide discusses in more detail the protocols used to execute memory transaction processing. The following are three notable features:

Implements Federation for address requests in any mode, including multiple requests to the same address. The previous architecture requires sequential access to words. In fact, this means that more Cuda applications will achieve excellent global memory performance, and Cuda programmers will no longer need to invent (if possible) contingency plan to ensure the normal operation of its global memory access mode.

If the half-warp address is in n different segments, only n memory transactions are processed. For example, if n = 2, only two memory transactions are processed (1/4 of the original transaction processing) compared with the 16 transactions processed in the previous architecture ).

Unfortunately, unused words in the memory still need to be read, so they waste bandwidth, even though the hardware will perform the smallest memory transaction processing possible.

200-series supports atomic operations of signed and unsigned integers. Atomicexch () is an exception. It also supports single-precision floating points. Atomic functions perform read-Modify-write atomic operations on 32-bit or 64-bit words residing in global or shared storage. For example, atomicadd () reads the 32-bit character of an address in global or shared storage, adds an integer, and writes the result back to the same address. The operation is atomic, indicating that the operation is performed without the interference of other threads. That is, no other thread can access this address before the operation is completed.

The warp vote function can be used in the 200 series. If necessary, these functions are essential for performing fast predicate operations (for example, checking whether conditions are true or false) in all the threads of warp:

int __any(int predicate); 

Evaluates the specified predicate for all threads of the warp. if and only if the predicate evaluates non-zero for all threads, a non-zero value is returned.

Finally, the improvement of the ability to execute both mad and Mul operations on the hardware should help some applications achieve a flop (floating point operation) rate closer to the peak performance.

For more information, visit the Cuda zone forum. I also recommend that you download the latest version of Cuda programming guide from the NVIDIA website. The current version is 2.0b2, which includes discussions on the latest features and APIs.

This may be the best time to upgrade to the 200-series Cuda Startup Device. Fierce competition (http://www.tgdaily.com/content/view/38243/135) prompted for a significant adjustment to the price, so now may be the best time to trade!

Cuda: supercomputing for the masses (Super computing for large amounts of data)-Section 7