Course Description
Limitations in the advancement of high-end single-threaded processors have forced new paradigms in software and algorithm development. Research and high-performance applications often require massively parallel systems for simulation, data processing, and data analysis. Several architectures, including nVidia’s CUDA and Intel’s Xeon Phi, provide highly parallel performance at low cost. However, algorithms primarily designed for massively parallel systems are difficult to design and optimize.
In this course, we will focus on the design and development of algorithms that take advantage of highly parallel co-processors, such as the nVidia GPUs, in order to solve research related problems. This course will include an overview of GPU architectures and principles in programming massively parallel systems. Topics covered will include designing and optimizing parallel algorithms, using available heterogeneous libraries, and case studies in linear systems, n-body problems, deep learning, and differential equations.
Syllabus [ PDF ]
Lectures
| 01 [ PPTX ] | Introduction to GPU programming, a brief history of programmable graphics |
| 02 [ PPTX ] | Creating a build environment |
| 03 [ PPTX ] | Heterogeneous architectures |
| 04 [ PPTX ] | CUDA language and API |
| 05 [ PPTX ] | Threading algorithms |
| 06 [ PPTX ] | Convolution, programming assignment |
| 07 [ PPTX ] | C++ timers and profiling |
| 08 [ PPTX ] | Managing memory latency |
| 09 [ PPTX ] | Thread synchronization, shared memory |
| 10 [ PPTX ] | Threaded matrix multiplication |
| 11 [ PPTX ] | Task parallelism in CUDA and C/C++ |
| 12 [ PPTX ] [ YouTube ] |
Floating point numbers |
| 13 [ PPTX ] [ YouTube ] |
nVidia libraries |
| 14 [ PPTX ] [ YouTube ] |
C++ templates and CUDA Thrust |
| 15 [ PPTX ] [ YouTube ] |
MATLAB parallel programming |
Homework
| HW 0 | HW0 - Verify your ability to develop CUDA code on your platform: [ CMakeLists.txt ] [ main.cu ] |
| HW 1 [ PDF ] | CUDA API |
| HW 2 [ PDF ] | Grid launch parameters |
| HW 3 [ PDF ] | Memory optimization |
Programming
| Final Project | Project Ideas: [ PPTX ] Report Template: [ DOCX ] Rubric: [ PDF ] Slide Template: [ PPTX ] |
| Program 1 | Image convolution [ PDF ] Hereford Cathedral [ 256 ] [ 512 ] [ HiRes ] Sombrero Galaxy [ HiRes ] Bansberia Ruins [ HiRes ] Whirlpool Galaxy [ HiRes ] |