GPU Programming with CUDA Training Course

Introduction

• Overview of CUDA.
• Comparison of CUDA with OpenCL and SYCL.
• Summary of CUDA features and architecture.
• Setup of the Development Environment.

Initial Steps

• Creating a new CUDA project in Visual Studio Code.
• Examining the project structure and file organization.
• Compiling and executing the program.
• Outputting results using printf and fprintf.

CUDA API

• Understanding the function of the CUDA API within the host program.
• Querying device information and capabilities via the CUDA API.
• Allocating and deallocating device memory using the CUDA API.
• Transferring data between host and device with the CUDA API.
• Launching kernels and synchronizing threads using the CUDA API.
• Handling errors and exceptions through the CUDA API.

CUDA C/C++

• Comprehending the role of CUDA C/C++ in device-side programming.
• Writing kernels for GPU execution and data manipulation using CUDA C/C++.
• Utilizing CUDA C/C++ data types, qualifiers, operators, and expressions.
• Applying CUDA C/C++ built-in functions, including math, atomic, and warp operations.
• Using CUDA C/C++ built-in variables, such as threadIdx, blockIdx, and blockDim.
• Leveraging CUDA C/C++ libraries like cuBLAS, cuFFT, and cuRAND.

CUDA Memory Model

• Distinguishing between host and device memory models.
• Utilizing CUDA memory spaces, including global, shared, constant, and local.
• Working with CUDA memory objects, such as pointers, arrays, textures, and surfaces.
• Applying CUDA memory access modes, such as read-only, write-only, and read-write.
• Implementing CUDA memory consistency models and synchronization mechanisms.

CUDA Execution Model

• Distinguishing between host and device execution models.
• Defining parallelism using CUDA threads, blocks, and grids.
• Utilizing CUDA thread functions, such as threadIdx, blockIdx, and blockDim.
• Employing CUDA block functions, such as __syncthreads and __threadfence_block.
• Using CUDA grid functions, including gridDim, gridSync, and cooperative groups.

Debugging

• Identifying common errors and bugs in CUDA programs.
• Using the Visual Studio Code debugger to inspect variables, set breakpoints, and analyze the call stack.
• Debugging CUDA programs on Linux with CUDA-GDB.
• Detecting memory errors and leaks using CUDA-MEMCHECK.
• Debugging and analyzing CUDA programs on Windows using NVIDIA Nsight.

Optimization

• Understanding factors influencing CUDA program performance.
• Enhancing memory throughput using CUDA coalescing techniques.
• Reducing memory latency via CUDA caching and prefetching techniques.
• Optimizing memory access and bandwidth using CUDA shared and local memory techniques.
• Measuring and improving execution time and resource utilization through CUDA profiling and associated tools.