Question: If we want to use each thread in a grid to calculate one output element of a vector addition, what is the expression for mapping the thread/block indices to the data index i?

Choices:
- A. i = threadIdx.x + threadIdx.y;
- B. i = blockIdx.x + threadIdx.x;
- C. i = blockIdx.x * blockDim.x + threadIdx.x;
- D. i = blockIdx.x * threadIdx.x;

Explanation: You need both the block offset (blockIdx.x * blockDim.x) and the thread offset within the block (threadIdx.x).


Question: Each thread calculates two adjacent elements of a vector addition. What is the expression for the data index i of the first element processed by a thread?

Choices:
- A. i = blockIdx.x * blockDim.x + threadIdx.x * 2;
- B. i = blockIdx.x * threadIdx.x * 2;
- C. i = (blockIdx.x * blockDim.x + threadIdx.x) * 2;
- D. i = blockIdx.x * blockDim.x * 2 + threadIdx.x;

Explanation: This doubles the logical thread index so each thread starts at an even index (0,2,4,...) while remaining contiguous across blocks.


Question: Each thread calculates two elements. A block processes 2*blockDim.x consecutive elements in two sections: first section (each thread does one element), then second section (each thread does one element). What is the expression for the first element index i for a thread?

Choices:
- A. i = blockIdx.x * blockDim.x + threadIdx.x + 2;
- B. i = blockIdx.x * threadIdx.x * 2;
- C. i = (blockIdx.x * blockDim.x + threadIdx.x) * 2;
- D. i = blockIdx.x * blockDim.x * 2 + threadIdx.x;

Explanation: The first section starts at the block's base offset of 2*blockDim.x. Each thread handles i and then i + blockDim.x in the second section.


Question: Vector addition with length 8000, 1 output element per thread, block size 1024. Using the minimum number of blocks to cover all elements, how many threads are in the grid?

Choices:
- A. 8000
- B. 8196
- C. 8192
- D. 8200

Explanation: ceil(8000/1024) = 8 blocks, each with 1024 threads -> 8*1024 = 8192 threads.


Question: Allocate an array of v integers in device global memory with cudaMalloc. What is the correct expression for the second argument (size in bytes)?

Choices:
- A. n
- B. v
- C. n * sizeof(int)
- D. v * sizeof(int)

Explanation: cudaMalloc takes the size in bytes; for v integers that is v * sizeof(int).


Question: Allocate an array of n floats and have pointer A_d point to it. What is the appropriate first argument to cudaMalloc?

Choices:
- A. n
- B. (void*) A_d
- C. *A_d
- D. (void**) &A_d

Explanation: cudaMalloc's first parameter is a void** to receive the device pointer (i.e., the address of the pointer).


Question: Copy 3000 bytes from host array A_h to device array A_d. Which API call is correct?

Choices:
- A. cudaMemcpy(3000, A_h, A_d, cudaMemcpyHostToDevice);
- B. cudaMemcpy(A_h, A_d, 3000, cudaMemcpyDeviceToHost);
- C. cudaMemcpy(A_d, A_h, 3000, cudaMemcpyHostToDevice);
- D. cudaMemcpy(3000, A_d, A_h, cudaMemcpyHostToDevice);

Explanation: Syntax is cudaMemcpy(dst, src, sizeBytes, kind). Here we copy from host to device.


Question: How to declare variable err to receive return values of CUDA API calls?

Choices:
- A. int err;
- B. cudaError err;
- C. cudaError_t err;
- D. cudaSuccess_t err;

Explanation: CUDA API error return type is cudaError_t.


Question:
Given the CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int N) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < N) {
        b[i] = 2.7f * a[i] - 4.3f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 200000;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d, N);
}
```

(a) What is the number of threads **per block**?

Answer: 128

Explanation: Given by the kernel launch <<<..., 128>>>.


Question:
Given the CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int N) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < N) {
        b[i] = 2.7f * a[i] - 4.3f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 200000;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d, N);
}
```

(b) What is the **number of threads in the grid**?

Answer: 200064

Explanation: Blocks = ceil(200000/128) = (200000 + 127) // 128 = 1563; threads = 1563 * 128 = 200064.


Question:
Given the CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int N) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < N) {
        b[i] = 2.7f * a[i] - 4.3f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 200000;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d, N);
}
```

(c) What is the **number of blocks in the grid**?

Answer: 1563

Explanation: Computed as (N + 128 - 1) / 128 with N = 200000.


Question:
Given the CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int N) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < N) {
        b[i] = 2.7f * a[i] - 4.3f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 200000;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d, N);
}
```

(d) How many threads **execute the index computation line** `unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;`?

Answer: 200064

Explanation: All launched threads execute the index computation line.


Question:
Given the CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int N) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < N) {
        b[i] = 2.7f * a[i] - 4.3f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 200000;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d, N);
}
```

(e) How many threads **execute the assignment inside the `if (i < N)`** - i.e., `b[i] = 2.7f * a[i] - 4.3f;`?

Answer: 200000

Explanation: Only threads with i < N execute the body; extra 64 threads fail the predicate.


Question:
Given the following CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int M, unsigned int N) {
    unsigned int row = blockIdx.y * blockDim.y + threadIdx.y;
    unsigned int col = blockIdx.x * blockDim.x + threadIdx.x;
    if (row < M && col < N) {
        b[row*N + col] = a[row*N + col]/2.1f + 4.8f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int M = 150;
    unsigned int N = 300;
    dim3 bd(16, 32);
    dim3 gd((N - 1) / 16 + 1, (M - 1) / 32 + 1);
    foo_kernel<<<gd, bd>>>(a_d, b_d, M, N);
}
```

(a) What is the number of threads per block?

Answer: 512

Explanation: bd = (16,32) ⇒ threadsPerBlock = 16x32 = 512.


Question:
Given the following CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int M, unsigned int N) {
    unsigned int row = blockIdx.y * blockDim.y + threadIdx.y;
    unsigned int col = blockIdx.x * blockDim.x + threadIdx.x;
    if (row < M && col < N) {
        b[row*N + col] = a[row*N + col]/2.1f + 4.8f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int M = 150;
    unsigned int N = 300;
    dim3 bd(16, 32);
    dim3 gd((N - 1) / 16 + 1, (M - 1) / 32 + 1);
    foo_kernel<<<gd, bd>>>(a_d, b_d, M, N);
}
```

(b) What is the number of threads in the grid?

Answer: 48640

Explanation: gd = (19,5) ⇒ blocks = 19x5 = 95. Threads = 95x512 = 48,640.


Question:
Given the following CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int M, unsigned int N) {
    unsigned int row = blockIdx.y * blockDim.y + threadIdx.y;
    unsigned int col = blockIdx.x * blockDim.x + threadIdx.x;
    if (row < M && col < N) {
        b[row*N + col] = a[row*N + col]/2.1f + 4.8f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int M = 150;
    unsigned int N = 300;
    dim3 bd(16, 32);
    dim3 gd((N - 1) / 16 + 1, (M - 1) / 32 + 1);
    foo_kernel<<<gd, bd>>>(a_d, b_d, M, N);
}
```

(c) What is the number of blocks in the grid?

Answer: 95

Explanation: Blocks = gd.x x gd.y = 19 x 5 = 95.


Question:
Given the following CUDA code:

```c
__global__ void foo_kernel(float* a, float* b, unsigned int M, unsigned int N) {
    unsigned int row = blockIdx.y * blockDim.y + threadIdx.y;
    unsigned int col = blockIdx.x * blockDim.x + threadIdx.x;
    if (row < M && col < N) {
        b[row*N + col] = a[row*N + col]/2.1f + 4.8f;
    }
}

void foo(float* a_d, float* b_d) {
    unsigned int M = 150;
    unsigned int N = 300;
    dim3 bd(16, 32);
    dim3 gd((N - 1) / 16 + 1, (M - 1) / 32 + 1);
    foo_kernel<<<gd, bd>>>(a_d, b_d, M, N);
}
```

(d) How many threads execute the assignment `b[row*N + col] = a[row*N + col]/2.1f + 4.8f;`?

Answer: 45000

Explanation: Only threads with (row < M && col < N) execute it. Count = MxN = 150x300 = 45,000.


Question: A 2D matrix has width=400 and height=500 and is stored as a 1D array in row-major order. What is the linear index of the element at row=20, col=10?

Answer: 8010

Explanation: Row-major index = row*width + col = 20*400 + 10 = 8,010.


Question: A 2D matrix has width=400 and height=500 and is stored as a 1D array in column-major order. What is the linear index of the element at row=20, col=10?

Answer: 5020

Explanation: Column-major index = col*height + row = 10*500 + 20 = 5,020.

CUDA indexing column_major linearization


Question: A 3D tensor has width=400 (x), height=500 (y), and depth=300 (z). It is stored as a 1D array in row-major order with index mapping idx = z*height*width + y*width + x. What is the linear index of the element at x=10, y=20, z=5?

Answer: 1008010

Explanation: idx = 5*500*400 + 20*400 + 10 = 1,000,000 + 8,000 + 10 = 1,008,010.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

(a) How many warps are there per block?

Answer: 4

Explanation: Each block has 128 threads and a warp has 32 threads -> 128/32 = 4 warps per block.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

(b) How many warps are there in the entire grid?

Answer: 32

Explanation: Blocks = (1024 + 128 - 1)/128 = 8. Warps per block = 4. Total warps = 8 x 4 = 32.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the statement `if (threadIdx.x < 40 || threadIdx.x >= 104) { ... }`:
(i) How many warps in the grid are active on this statement?

Answer: 24

Explanation: Per block: warp 0 (0-31) active; warp 1 (32-63) partially active -> warp active; warp 2 (64-95) inactive; warp 3 (96-127) partially active -> warp active. So 3 active warps/block x 8 blocks = 24.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the statement `if (threadIdx.x < 40 || threadIdx.x >= 104) { ... }`:
(ii) How many warps in the grid are divergent on this statement?

Answer: 16

Explanation: Per block, warp 1 (32-63) and warp 3 (96-127) have mixed predicates (some threads true, some false) -> 2 divergent warps/block x 8 blocks = 16.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the same statement, what is the SIMD efficiency of warp 0 of block 0? Give a decimal in [0,1] with two decimals; do not include %.

Answer: 1.00

Explanation: Warp 0 covers threads 0-31; all satisfy `threadIdx.x < 40`. Active lanes = 32/32 = 100%.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the same statement, what is the SIMD efficiency of warp 1 of block 0? Give a decimal in [0,1] with two decimals; do not include %.

Answer: 0.25

Explanation: Warp 1 covers 32-63; only 32-39 (8 lanes) satisfy the predicate. Efficiency = 8/32 = 25%.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the same statement, what is the SIMD efficiency of warp 3 of block 0? Give a decimal in [0,1] with two decimals; do not include %.

Answer: 0.75

Explanation: Warp 3 covers 96-127; only 104-127 (24 lanes) satisfy the predicate. Efficiency = 24/32 = 75%.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the statement `if (i % 2 == 0) { ... }`:
(i) How many warps in the grid are active on this statement?

Answer: 32

Explanation: All warps reach the statement; within each warp, half the threads satisfy `i % 2 == 0`, but the warp itself is active. Total warps = 32.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the statement `if (i % 2 == 0) { ... }`:
(ii) How many warps in the grid are divergent on this statement?

Answer: 32

Explanation: Within every warp, half the lanes are even and half odd, so every warp diverges on this predicate.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the statement `if (i % 2 == 0) { ... }`:
(iii) What is the SIMD efficiency of warp 0 of block 0? Give a decimal between 0 and 1 with two decimals; do not include a % sign.

Answer: 0.50

Explanation: Exactly half the lanes (even indices) are active: 16/32 = 50%.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

Consider the loop `for (unsigned int j = 0; j < 5 - (i % 3); ++j) { ... }`.
(i) How many loop iterations (values of j) execute with no divergence across the entire grid?

Answer: 3

Explanation: Threads with i%3 in {0,1,2} have bounds 5,4,3 respectively. For j = 0,1,2 all threads execute; for j = 3 and 4 some threads do not, causing divergence. Hence 3 non-divergent iterations.


Question:
Consider the following CUDA kernel and host code:

```c
__global__ void foo_kernel(int* a, int* b) {
    unsigned int i = blockIdx.x*blockDim.x + threadIdx.x;
    if (threadIdx.x < 40 || threadIdx.x >= 104) {
        b[i] = a[i] + 1;
    }
    if (i % 2 == 0) {
        a[i] = b[i] * 2;
    }
    for (unsigned int j = 0; j < 5 - (i % 3); ++j) {
        b[i] += j;
    }
}

void foo(int* a_d, int* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1)/128, 128>>>(a_d, b_d);
}
```

For the same loop:
(ii) How many loop iterations (values of j) have divergence somewhere in the grid?

Answer: 2

Explanation: Iterations j = 3 and j = 4 are executed by only some threads (depending on i%3), so both are divergent.


Question: Vector addition of length 2000; one output element per thread; thread block size = 512 threads. Using the minimum number of blocks to cover all elements, how many threads are in the grid?

Answer: 2048

Explanation: Blocks = ceil(2000/512) = 4; threads = 4 x 512 = 2048.


Question: with 2000 elements and 512 threads per block (minimal blocks), how many warps do you expect to have divergence due to the boundary check (threads skipping work past N)?

Answer: 1

Explanation: Total warps = 2048/32 = 64. Only the warp covering thread indices 1984-2015 has some active (<=1999) and some inactive (>=2000) threads. The final warp (2016-2047) has all threads inactive (no divergence).


Question: A block with 8 threads executes a section before a barrier. Times (us) to reach the barrier are: 2.0, 2.3, 3.0, 2.8, 2.4, 1.9, 2.6, 2.9. Threads then wait at the barrier until the slowest arrives. What percentage of the aggregate thread time is spent waiting? Give a percentage to one decimal place; do not include %.

Answer: 17.1

Explanation: Max time = 3.0. Waiting per thread = (3.0 - t). Sum waits = 4.1 us. Aggregate time = 8 x 3.0 = 24 us. Percentage ~ 4.1/24 ~ 17.1%.


Question: An SM supports up to 1536 threads and up to 4 blocks concurrently. For a single SM, which block size among {128, 256, 512, 1024} yields the maximum number of resident threads, and how many threads does it schedule? Return the result as a tuple (BLOCK_SIZE, THREADS).

Answer: (512, 1536)

Explanation: For 128: min(4, 1536/128)=4 blocks -> 4x128=512 threads. For 256: 4 blocks -> 1024. For 512: 3 blocks (limited by total threads) -> 3x512=1536. For 1024: 1 block -> 1024. Max is 1536 with 512/block.


Question:
A device allows up to 64 blocks per SM and 2048 threads per SM. For each per-SM assignment below, state if it's possible and the occupancy (% of 2048 threads):
(a) 8 blocks x 128 threads
(b) 16 blocks x 64 threads
(c) 32 blocks x 32 threads
(d) 64 blocks x 32 threads
(e) 32 blocks x 64 threads Provide five semicolon-separated tuples (ans,occ) for (a)-(e), where ans is Yes/No and occ is an integer percent with no %.

Answer: (Yes,50);(Yes,50);(Yes,50);(Yes,100);(Yes,100)

Explanation: Check blocks ≤ 64 and total threads ≤ 2048 for each case; occupancy = total_threads / 2048.


Question:
A GPU has 2048 threads/SM, 32 blocks/SM, and 65,536 registers/SM. For each kernel, can it achieve full occupancy (2048 threads/SM)? If not, what limits it?
(a) 128 threads/block, 30 registers/thread
(b) 32 threads/block, 29 registers/thread
(c) 256 threads/block, 34 registers/thread Provide three semicolon-separated tuples (ans,limit,threads) for (a)-(c), where ans is Yes/No, limit in {none,blocks,registers}, threads is an integer.

Answer: (Yes,none,2048);(No,blocks,1024);(No,registers,1792)

Explanation: For each case, first bound by blocks from threads/SM, then check blocks limited by registers per block. Compare resulting resident threads to 2048.


Question: We multiply two 8x8 matrices C=AxB on a GPU. One thread computes one C[i,j]. Using shared-memory tiling with tile size T=2 (2x2 tiles), and assuming no caching except shared memory, how many total global-memory element loads (A and B only; ignore the 64 stores of C) are performed for the full multiplication? Give an integer; do not include units or x.

Answer: 2

Explanation: Baseline (no tiling, T=1): 64 outputs x 8 MACs/output x 2 loads/MAC = 1024 loads. With tiling of size T, loads = (8/T)^2 x (8/T) x 2T^2 = 1024/T. For T=2 -> 1024/2 = 512.


Question: Consider an 8x8 matrix multiplication with tile size T=2, what is the reduction factor in global-memory traffic versus the naive untiled case (T=1)? Give an integer; do not include units or x.

Answer: 2

Explanation: Untiled: 1024 loads. T=2: 512 loads. Reduction factor = 1024 / 512 = 2x.


Question: Now use tile size T=4 (4x4 tiles) for an 8x8 matrix multiplication. How many total global-memory element loads (A and B only; ignore C stores) are performed?

Answer: 256

Explanation: Using loads = 1024/T with N=8, for T=4 we have 1024/4 = 256.


Question: Consider an 8x8 matrix multiplication with tile size T=4. What is the reduction factor in global-memory traffic relative to the naive untiled case (T=1)? Give an integer; do not include units or x.

Answer: 4

Explanation: Untiled: 1024 loads. T=4: 256 loads. Reduction factor = 1024 / 256 = 4x, confirming linear scaling with tile dimension.


Question: Consider matrix addition C = A + B. Can shared memory be used to reduce global memory bandwidth consumption for this kernel? Briefly justify your answer.

Answer: No.

Explanation: In element-wise addition, each output element C[i,j] depends only on A[i,j] and B[i,j] once. Threads do not reuse neighbors' A or B values, so there is no inter-thread temporal locality to exploit. Caching A/B into shared memory would just add extra copies without reducing global loads.


Question: Assume register and shared memory capacities are not limiting. Give one important reason why using shared memory (instead of registers) to hold values fetched from global memory can be valuable.

Answer: Shared memory enables inter-thread data sharing within a block.

Explanation: Registers are private to a thread. Shared memory is visible to all threads in a block, so a value fetched once from global memory can be reused by multiple threads, reducing global traffic.


Question: For a tiled matrix-matrix multiplication kernel using 32x32 tiles, what is the reduction in global memory bandwidth usage for the input matrices M and N (compared to the untiled naive access), assuming ideal reuse within a tile? Provide the two reduction factors as M,N (integers); do not include x or text.

Answer: 32,32

Explanation: Within a 32x32 tile, each loaded element of M (resp. N) is reused across 32 multiply-accumulates along the tile dimension, replacing 32 separate global loads in the naive scheme. Thus, global loads per useful use drop by ~32x for both inputs.

CUDA tiling matrix_multiplication bandwidth reuse


Question: A CUDA kernel is launched with 1000 thread blocks, each with 512 threads. If a variable is declared as a local (per-thread) variable inside the kernel, how many distinct instances of this variable are created during execution?

Answer: 512,000

Explanation: Local variables are per-thread. Total threads = 1000 blocks x 512 threads/block = 512,000 instances.


Question: A CUDA kernel is launched with 1000 thread blocks, each with 512 threads. if a variable is declared in shared memory, how many distinct instances of this variable are created during execution?

Answer: 1,000

Explanation: Shared memory is per-block. There is exactly one instance per block -> 1000 instances total.


Question: For multiplying two NxN matrices without tiling, how many times is each input element requested from global memory?

Answer: N times.

Explanation: In the naive kernel each output element recomputes its dot product by reloading the same row/column elements. Each input element participates in N different outputs along the corresponding dimension, leading to N separate loads.


Question: For multiplying two NxN matrices with TxT tiling (ideal reuse within tiles), how many times is each input element requested from global memory?

Answer: N/T times.

Explanation: Each input element is fetched once per tile-stripe it participates in along the multiply dimension. Tiling reduces redundant loads by a factor of T, so loads per element drop from N to N/T.


Question: A CUDA kernel performs 36 floating-point operations and 7 global 32-bit (4-byte) memory accesses per thread. On a GPU with peak 200 GFLOP/s compute throughput and 100 GB/s memory bandwidth, is the kernel compute-bound or memory-bound? Justify briefly using a roofline-style argument.

Answer: Memory-bound.

Explanation: Arithmetic intensity = 36 FLOPs / (7x4 B) = 36/28 ~ 1.286 FLOP/B. Machine balance = 200 GFLOP/s ÷ 100 GB/s = 2.0 FLOP/B. Since 1.286 < 2.0, performance is limited by memory bandwidth.


Question: A CUDA kernel performs 36 floating-point operations and 7 global 32-bit (4-byte) memory accesses per thread. On a GPU with peak 300 GFLOP/s compute throughput and 250 GB/s memory bandwidth, is the kernel compute-bound or memory-bound? Justify briefly using a roofline-style argument.

Answer: Compute-bound.

Explanation: Arithmetic intensity = 36 FLOPs / (7x4 B) = 36/28 ~ 1.286 FLOP/B. Machine balance = 300 GFLOP/s ÷ 250 GB/s = 1.2 FLOP/B. Since 1.286 > 1.2, performance is limited by compute throughput.


Question:
You are given this tile-transpose kernel (abbrev.):

```cpp
dim3 blockDim(BLOCK_WIDTH,BLOCK_WIDTH);
dim3 gridDim(A_width/blockDim.x, A_height/blockDim.y);
BlockTranspose<<<gridDim, blockDim>>>(A, A_width, A_height);

__global__ void BlockTranspose(float* A_elements, int A_width, int A_height) {
    __shared__ float blockA[BLOCK_WIDTH][BLOCK_WIDTH];
    int baseIdx = blockIdx.x * BLOCK_WIDTH + threadIdx.x;
    baseIdx += (blockIdx.y * BLOCK_WIDTH + threadIdx.y) * A_width;
    blockA[threadIdx.y][threadIdx.x] = A_elements[baseIdx];
    // (no barrier here)
    A_elements[baseIdx] = blockA[threadIdx.x][threadIdx.y];
}
```
For which BLOCK_WIDTH values does this execute correctly?

Answer: Only BLOCK_WIDTH = 1.

Explanation: Without a barrier, some threads can read from shared memory before peers have written their elements. With a 1x1 block, there is only one thread and no race; for any larger tile, a race exists.

CUDA synchronization shared_memory barriers


Question:
You are given this tile-transpose kernel (abbreviated):

```cpp
dim3 blockDim(BLOCK_WIDTH, BLOCK_WIDTH);
dim3 gridDim(A_width / blockDim.x, A_height / blockDim.y);
BlockTranspose<<<gridDim, blockDim>>>(A, A_width, A_height);

__global__ void BlockTranspose(float* A_elements, int A_width, int A_height) {
    __shared__ float blockA[BLOCK_WIDTH][BLOCK_WIDTH];
    int baseIdx = blockIdx.x * BLOCK_WIDTH + threadIdx.x;
    baseIdx += (blockIdx.y * BLOCK_WIDTH + threadIdx.y) * A_width;
    blockA[threadIdx.y][threadIdx.x] = A_elements[baseIdx];
    // (no barrier here)
    A_elements[baseIdx] = blockA[threadIdx.x][threadIdx.y];
}
```
Explain the root cause of incorrect execution for BLOCK_WIDTH > 1 and give a minimal fix that makes it correct for any BLOCK_WIDTH >= 1. Give a single token naming the minimal synchronization fix.

Answer: __syncthreads()

Explanation: All threads must complete their writes to shared memory before any thread reads `blockA[tx][ty]`. Without a barrier, reads can see stale values (race). Adding `__syncthreads();` between the store and the load enforces correctness for any tile size.

CUDA synchronization shared_memory barriers


Question:
Consider the following CUDA code:

```cpp
__global__ void foo_kernel(float* a, float* b) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    float x[4];
    __shared__ float y_s;
    __shared__ float b_s[128];
    for (unsigned int j = 0; j < 4; ++j) {
        x[j] = a[j * blockDim.x * gridDim.x + i];
    }
    if (threadIdx.x == 0) { y_s = 7.4f; }
    b_s[threadIdx.x] = b[i];
    __syncthreads();
    b[i] = 2.5f*x[0] + 3.7f*x[1] + 6.3f*x[2] + 8.5f*x[3]
         + y_s*b_s[threadIdx.x] + b_s[(threadIdx.x + 3) % 128];
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d);
}
```
How many distinct instances of variable `i` exist during execution?

Answer: 1,024

Explanation: Blocks = (1024 + 127) / 128 = 8; threads per block = 128; total threads = 8 x 128 = 1,024. `i` is per-thread.


Question:
Consider the following CUDA code:

```cpp
__global__ void foo_kernel(float* a, float* b) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    float x[4];
    __shared__ float y_s;
    __shared__ float b_s[128];
    for (unsigned int j = 0; j < 4; ++j) {
        x[j] = a[j * blockDim.x * gridDim.x + i];
    }
    if (threadIdx.x == 0) { y_s = 7.4f; }
    b_s[threadIdx.x] = b[i];
    __syncthreads();
    b[i] = 2.5f*x[0] + 3.7f*x[1] + 6.3f*x[2] + 8.5f*x[3]
         + y_s*b_s[threadIdx.x] + b_s[(threadIdx.x + 3) % 128];
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d);
}
```
How many distinct instances of the local array `x[4]` are created during execution?

Answer: 1,024

Explanation: `x` is a per-thread local array. With 1,024 threads total, there are 1,024 instances.


Question:
Consider the following CUDA code:

```cpp
__global__ void foo_kernel(float* a, float* b) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    float x[4];
    __shared__ float y_s;
    __shared__ float b_s[128];
    for (unsigned int j = 0; j < 4; ++j) {
        x[j] = a[j * blockDim.x * gridDim.x + i];
    }
    if (threadIdx.x == 0) { y_s = 7.4f; }
    b_s[threadIdx.x] = b[i];
    __syncthreads();
    b[i] = 2.5f*x[0] + 3.7f*x[1] + 6.3f*x[2] + 8.5f*x[3]
         + y_s*b_s[threadIdx.x] + b_s[(threadIdx.x + 3) % 128];
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d);
}
```
How many distinct instances of the shared variable `y_s` are created during execution?

Answer: 8

Explanation: Shared memory is per-block. Blocks = (1024 + 127) / 128 = 8 -> 8 instances of `y_s`.


Question:
Consider the following CUDA code:

```cpp
__global__ void foo_kernel(float* a, float* b) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    float x[4];
    __shared__ float y_s;
    __shared__ float b_s[128];
    for (unsigned int j = 0; j < 4; ++j) {
        x[j] = a[j * blockDim.x * gridDim.x + i];
    }
    if (threadIdx.x == 0) { y_s = 7.4f; }
    b_s[threadIdx.x] = b[i];
    __syncthreads();
    b[i] = 2.5f*x[0] + 3.7f*x[1] + 6.3f*x[2] + 8.5f*x[3]
         + y_s*b_s[threadIdx.x] + b_s[(threadIdx.x + 3) % 128];
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d);
}
```
How many distinct instances of the shared array `b_s[128]` are created during execution?

Answer: 8

Explanation: Shared arrays are also per-block. With 8 blocks, there are 8 instances of `b_s`.


Question:
Consider the following CUDA code:

```cpp
__global__ void foo_kernel(float* a, float* b) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    float x[4];
    __shared__ float y_s;
    __shared__ float b_s[128];
    for (unsigned int j = 0; j < 4; ++j) {
        x[j] = a[j * blockDim.x * gridDim.x + i];
    }
    if (threadIdx.x == 0) { y_s = 7.4f; }
    b_s[threadIdx.x] = b[i];
    __syncthreads();
    b[i] = 2.5f*x[0] + 3.7f*x[1] + 6.3f*x[2] + 8.5f*x[3]
         + y_s*b_s[threadIdx.x] + b_s[(threadIdx.x + 3) % 128];
}

void foo(float* a_d, float* b_d) {
    unsigned int N = 1024;
    foo_kernel<<<(N + 128 - 1) / 128, 128>>>(a_d, b_d);
}
```
What is the amount of shared memory used per block (in bytes)? Give an integer (bytes); do not include units.

Answer: 516

Explanation: `y_s`: 1 float = 4 bytes; `b_s[128]`: 128 floats = 512 bytes; total = 516 bytes.


Question:
Consider the following CUDA code and compute the floating-point-operations-per-byte (OP/B) ratio per thread with respect to global memory traffic (assume 4-byte floats, count both reads and the final write):

```cpp
__global__ void foo_kernel(float* a, float* b) {
    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;
    float x[4];
    __shared__ float y_s;
    __shared__ float b_s[128];
    for (unsigned int j = 0; j < 4; ++j) {
        x[j] = a[j * blockDim.x * gridDim.x + i];
    }
    if (threadIdx.x == 0) { y_s = 7.4f; }
    b_s[threadIdx.x] = b[i];
    __syncthreads();
    b[i] = 2.5f*x[0] + 3.7f*x[1] + 6.3f*x[2] + 8.5f*x[3]
         + y_s*b_s[threadIdx.x] + b_s[(threadIdx.x + 3) % 128];
}
```
What is the OP/B value? Round to 3 decimals; provide only the numeric value.

Answer: 0.417

Explanation: Per thread FLOPs = 5 mul + 5 add = 10. Global traffic: 4 reads from `a` (16 B) + 1 read from `b` (4 B) + 1 write to `b` (4 B) = 24 B. OP/B = 10 / 24 ~ 0.417.


Question: GPU limits: 2048 threads/SM, 32 blocks/SM, 65,536 registers/SM, 96 KB shared memory/SM. Kernel uses 64 threads/block, 27 registers/thread, and **4 KB shared memory per block**. Can it reach full occupancy (2048 threads/SM)? If not, what limits it and what is the achieved occupancy? Provide a tuple (Yes|No,limiting_resource,threads) with limiting_resource in {none,shared_memory,blocks,registers}.

Answer: (No,shared_memory,1536)

Explanation: Max blocks by threads = 2048/64 = 32 (OK). Registers: 27x2048 = 55,296 < 65,536 (OK). Shared memory: at 4 KB per block, 96 KB/4 KB = 24 blocks fit, so threads = 24x64 = 1,536 -> 75% occupancy. Limiting factor: shared memory per SM.

CUDA occupancy resources shared_memory registers


Question: GPU limits: 2048 threads/SM, 32 blocks/SM, 65,536 registers/SM, 96 KB shared memory/SM. Kernel uses 256 threads/block, 31 registers/thread, and **8 KB shared memory per block**. Can it reach full occupancy (2048 threads/SM)? If not, what limits it and what is the achieved occupancy? Provide a tuple (Yes|No,limiting_resource,threads) with limiting_resource in {none,shared_memory,blocks,registers}.

Answer: (Yes,none,2048)

Explanation: Max blocks by threads = 2048/256 = 8. Registers: 31x2048 = 63,488 < 65,536 (OK). Shared memory: 8 blocks x 8 KB = 64 KB < 96 KB (OK). Blocks/SM limit is 32 (not binding). All constraints allow 8 blocks x 256 threads = 2048.

CUDA occupancy resources shared_memory registers


Question: A 2D tiled GEMM uses a BLOCK_SIZExBLOCK_SIZE thread block. Threads are indexed (ty, tx). In each phase, threads cooperatively load: M[row, ph*BLOCK_SIZE + tx] and (corner-turned) N[ph*BLOCK_SIZE + ty, col], where row = blockIdx.y*BLOCK_SIZE + ty and col = blockIdx.x*BLOCK_SIZE + tx. Arrays are row-major 4-byte floats. Warps have 32 lanes and lanes vary along x (a warp spans 32 consecutive tx at fixed ty). Which BLOCK_SIZE guarantees that both the M and N loads of every warp are fully coalesced into a single contiguous segment?

Choices:
- A. 8
- B. 16
- C. 32
- D. Any power of two

Explanation: With warps laid out along x, coalescing requires each warp to cover 32 consecutive tx at fixed ty so addresses are contiguous for both loads. BLOCK_SIZE=32 aligns one warp per row; 8 or 16 split a warp across multiple rows.


Question: Using the setup described in a BLOCK_SIZE x BLOCK_SIZE tiled GEMM with corner-turned N load, row-major floats, and 32-lane warps spanning x (BLOCK_SIZExBLOCK_SIZE tiled GEMM, corner-turned N load, row-major floats, warps span x with 32 lanes), if BLOCK_SIZE=16, what is the coalescing behavior of a warp's global loads for M and N?

Choices:
- A. Fully coalesced into a single contiguous segment per warp
- B. Two contiguous segments per warp (warp spans two 16-wide rows)
- C. Uncoalesced/random access
- D. Depends only on base address alignment

Explanation: A 32-lane warp spans two 16-wide rows at fixed ty, so both M and N loads become two 16-element contiguous segments per warp rather than one 32-element segment.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: a[i].

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Consecutive threads in a warp access consecutive elements a[i]; this is the canonical coalesced pattern.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: a_s[threadIdx.x], where a_s is declared in __shared__ memory.

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Coalescing applies to global memory transactions. Shared memory has different banking rules; coalescing classification is not applicable.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: b[j*blockDim.x*gridDim.x + i] with j fixed inside the loop.

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: For a fixed j, threads in a warp access consecutive indices offset by a constant base; addresses are contiguous across the warp.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: c[i*4 + j], with j fixed inside the loop.

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Across a warp, i increases by 1 so addresses stride by 4 elements (16 B) per thread, leading to multiple memory transactions (not contiguous).


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: bc_s[j*256 + threadIdx.x], where bc_s is __shared__ memory.

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Shared memory access; global-memory coalescing classification does not apply.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: a_s[threadIdx.x] (read), where a_s is __shared__ memory.

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Shared memory read; coalescing is a global-memory concept.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: d[i + 8] (global write).

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: The +8 is a constant offset; adjacent threads still write consecutive locations, so the warp issues contiguous transactions.


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: bc_s[threadIdx.x*4] (read), where bc_s is __shared__ memory.

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Shared memory read; coalescing classification is not applicable (banking rules apply instead).


Question: Classify the following memory access for coalescing on modern CUDA GPUs. Assume: 1D launch; i = blockIdx.x*blockDim.x + threadIdx.x; warps are 32 threads along x; arrays are float* (4 B) in global memory unless noted; shared memory accesses are not subject to global coalescing. Access: e[i*8] (global write).

Choices:
- A. Coalesced
- B. Uncoalesced
- C. Not applicable (shared memory)
- D. Unaligned but still fully coalesced due to caching

Explanation: Thread addresses stride by 8 elements (32 B) per lane across the warp; not a single contiguous segment, so it's uncoalesced.


Question: Arithmetic intensity (FLOP/B) for naive GEMM: One thread computes P[i,j] with a loop over k=0..n-1, doing one multiply and one add per k (2 FLOPs) and reading M[i,k] and N[k,j] from global memory each iteration. Assume 4-byte floats and ignore output writes. What is the arithmetic intensity?

Answer: 0.25

Explanation: Per output: 2n FLOPs and 2n reads x 4 B = 8n B -> (2n)/(8n) = 0.25 FLOP/B.


Question: Arithmetic intensity (FLOP/B) for tiled GEMM with BLOCK_SIZE=T: Each phase loads one TxT tile of M and one TxT tile of N from global memory and reuses them from shared memory to produce a TxT output tile. Assume 4-byte floats and ignore output writes. For T=32, what is the arithmetic intensity?

Answer: 8

Explanation: Global reads per output = 2n/T elements -> 8n/T bytes; FLOPs per output = 2n. Intensity = (2n)/(8n/T) = T/4 = 8 for T=32.

roofline arithmetic_intensity tiling GEMM


Question: Arithmetic intensity (FLOP/B) for tiled GEMM with thread coarsening factor C=4: The M tile loaded from global memory is reused across 4 adjacent output tiles; N tiles are not further reused beyond standard tiling. Assume 4-byte floats and ignore output writes. For T=32, what is the arithmetic intensity?

Answer: 12.8

Explanation: Reads per output: M contributes n/(4T), N contributes n/T -> (5n)/(4T) elements -> (5n)/T bytes. FLOPs per output = 2n. Intensity = (2n)/((5n)/T) = 2T/5 = 12.8 for T=32.

roofline arithmetic_intensity tiling coarsening GEMM


Question: Perform 1D discrete convolution with zero-padding (output length equals input length): N = {4, 1, 3, 2, 3}, F = {2, 1, 4}. Use P[i] = sum_{k=0..2} F[k] * N[i - 1 + k], treating out-of-bounds N as 0. What is P?

Choices:
- A. [8, 21, 13, 20, 7]
- B. [4, 12, 17, 14, 9]
- C. [2, 9, 12, 15, 11]
- D. [0, 8, 21, 13, 20]

Explanation: Zero-padding with radius r=1 yields P = [8, 21, 13, 20, 7].


Question: In 1D discrete convolution with zero-padding, what operation does the filter [0, 1, 0] primarily perform on a signal x?

Choices:
- A. Identity (pass-through): y[i] = x[i]
- B. Right shift by 1: y[i] = x[i-1]
- C. Left shift by 1: y[i] = x[i+1]
- D. 3-point moving average

Explanation: [0,1,0] preserves the center sample, acting as an identity under the stated assumptions.


Question: In 1D discrete convolution with zero-padding, what operation does the filter [0, 0, 1] primarily perform?

Choices:
- A. Identity (pass-through)
- B. Right shift by 1 sample
- C. Left shift by 1 sample
- D. 3-point moving average

Explanation: With the conventions used here, [0,0,1] produces a right shift by 1 (y[i] ~ x[i-1]).


Question: In 1D discrete convolution with zero-padding, what operation does the filter [1, 0, 0] primarily perform?

Choices:
- A. Right shift by 1 sample
- B. Left shift by 1 sample
- C. Identity (pass-through)
- D. High-pass smoothing

Explanation: With the conventions used here, [1,0,0] produces a left shift by 1 (y[i] ~ x[i+1]).


Question: In 1D discrete convolution, what is the primary effect of the filter [-1/2, 0, 1/2]?

Choices:
- A. Low-pass smoothing
- B. First-derivative (edge detection)
- C. Identity
- D. Right shift by 1

Explanation: It approximates a first derivative, responding to rapid changes (edges).


Question: In 1D discrete convolution, what is the primary effect of the filter [1/3, 1/3, 1/3]?

Choices:
- A. High-pass edge enhancer
- B. Left shift by 1
- C. 3-point moving average (smoothing)
- D. Identity

Explanation: Equal weights average the local neighborhood, smoothing noise.


Question: 1D convolution on an array of length N with an odd-sized filter of length M = 2r+1 (r = (M-1)/2). How many ghost (zero-padded) cells are there in total?

Choices:
- A. r
- B. 2r
- C. M - 1
- D. N + M

Explanation: Total ghost cells = r on the left + r on the right = 2r = M - 1.


Question: 1D convolution on array length N with odd-sized filter M (using zero-padding, counting multiplications even when reading zeros). How many total multiplications are performed?

Choices:
- A. N × M
- B. N × (M - 1)
- C. (N - M) × M
- D. 2N × M

Explanation: Each of N outputs multiplies M taps (zeros included) -> NxM.


Question: 2D convolution on an NxN image with an odd-sized MxM filter (M = 2r+1). With zero-padding, how many ghost cells surround the image in total?

Choices:
- A. 4Nr
- B. 2N(M-1)
- C. 4r(N + r)
- D. N^2 - (N - 2r)^2

Explanation: Padding adds r rows/cols around; total ghost cells = 4r(N + r).


Question: 2D convolution on an NxN image with an MxM filter (zero-padding, counting multiplications even on zeros). How many total multiplications are performed?

Choices:
- A. N^2 × M^2
- B. N × M
- C. (N - M + 1)^2 × M^2
- D. 2N × M

Explanation: Each of N^2 outputs multiplies M^2 taps -> N^2xM^2.


Question: 2D convolution on an N1xN2 image with an odd-sized M1xM2 filter. Let r1=(M1-1)/2 and r2=(M2-1)/2. With zero-padding, how many ghost cells are there in total?

Choices:
- A. 2(N1 r2 + N2 r1)
- B. 2(N1 r2 + N2 r1) + 4 r1 r2
- C. (N1 + N2)(r1 + r2)
- D. N1 N2 (r1 + r2)

Explanation: Edges contribute 2(N1 r2 + N2 r1); corners add 4 r1 r2.


Question: 2D convolution on an N1xN2 image with an M1xM2 filter (zero-padding, counting multiplications even on zeros). How many total multiplications are performed?

Choices:
- A. N1 N2 M1 M2
- B. N1 N2 (M1 + M2)
- C. (N1 - M1 + 1)(N2 - M2 + 1) M1 M2
- D. 2 N1 N2 M1

Explanation: Each of N1xN2 outputs multiplies M1xM2 taps -> N1 N2 M1 M2.


Question: A 2D tiled convolution uses an output tile of size TxT and an odd-sized filter with radius r=(M-1)/2. Input tiles are (T+2r)x(T+2r) due to halo. For an NxN output, how many thread blocks are needed?

Choices:
- A. ceil(N/T) × ceil(N/T)
- B. (N/T) × (N/T) (truncate)
- C. N × N
- D. ceil(N/(T+2r)) × ceil(N/(T+2r))

Explanation: Each block produces a TxT output tile; tiling the output requires ceil(N/T) in each dimension.


Question: In a tiled 2D convolution setup where the output tile is TxT and the halo radius is r, the block loads a (T+2r)x(T+2r) input tile into shared memory. Assuming one thread per input-tile element, how many threads are needed per block?

Choices:
- A. T^2
- B. (T+r)^2
- C. (T+2r)^2
- D. 2T(T+r)

Explanation: One thread per input-tile element -> (T+2r)^2 threads per block.


Question: In a tiled 2D convolution setup where the output tile is TxT and the halo radius is r, the block loads a (T+2r)x(T+2r) input tile into shared memory and allocates a shared-memory array to hold this input tile. How much shared memory is needed per block (in bytes) for single-precision floats?

Choices:
- A. T^2 × 4
- B. (T+2r)^2 × 4
- C. (T+2r) × 4
- D. 0

Explanation: The shared tile size is (T+2r)x(T+2r) floats; at 4 bytes/float -> (T+2r)^2 x 4 bytes.


Question: Consider a 2D convolution implementation that does NOT allocate any shared-memory input tile. Each thread block contains TxT threads, and each thread computes exactly one output element of a TxT output tile. All input reads are served directly from global memory (relying only on hardware caches). For an NxN output, how many thread blocks are required?

Choices:
- A. ceil(N/T) × ceil(N/T)
- B. (N/T) × (N/T) (truncate)
- C. N × N
- D. ceil(N/(T+2r)) × ceil(N/(T+2r))

Explanation: Each block covers a TxT region of the output, so the grid needs ceil(N/T) blocks along each dimension.


Question: Consider a 2D convolution implementation that does NOT allocate any shared-memory input tile. Each thread block contains TxT threads, and each thread computes exactly one output element of a TxT output tile. All input reads are served directly from global memory (relying only on hardware caches). How many threads are launched per block?

Choices:
- A. T^2
- B. (T+2r)^2
- C. 2T(T+r)
- D. N^2

Explanation: One thread per output element over a TxT tile yields TxT = T^2 threads per block.


Question: Consider a 2D convolution implementation that does NOT allocate any shared-memory input tile. Each thread block contains TxT threads, and each thread computes exactly one output element of a TxT output tile. All input reads are served directly from global memory (relying only on hardware caches). How much shared memory is needed per block (in bytes) to hold the input tile when using single-precision floats?

Choices:
- A. (T+2r)^2 × 4
- B. T^2 × 4
- C. 0
- D. 2(T+2r)^2 × 4

Explanation: By definition, this variant allocates no shared-memory input tile; it relies solely on hardware caches.


Question: A 3D seven-point stencil is applied on a cubic grid of size 120x120x120 (including boundary cells). The kernel only writes interior points (i=1..118, j=1..118, k=1..118). How many output grid points are computed per sweep?

Answer: 1643032

Explanation: Interior count = (120-2)^3 = 118^3 = 1,643,032.


Question: A basic 3D stencil kernel launches one thread per grid point over a 120x120x120 domain using blocks of size 8x8x8 threads (no overhang trimming). Using ceil division per dimension, how many thread blocks are launched in total?

Answer: 3375

Explanation: Blocks per dim = ceil(120/8) = 15 -> total blocks = 15^3 = 3,375.


Question: A shared-memory tiled 3D stencil uses IN_TILE_DIM = 8 and a radius r = 1, so OUT_TILE_DIM = IN_TILE_DIM - 2r = 6. Over a 120x120x120 domain, blocks are placed per OUT_TILE_DIM using ceil division per dimension. How many thread blocks are launched in total?

Answer: 8000

Explanation: Blocks per dim = ceil(120/6) = 20 -> total blocks = 20^3 = 8,000.


Question: A coarsened/tiled 3D stencil uses 2D thread blocks of IN_TILE_DIMxIN_TILE_DIM = 32x32 (z handled by coarsening) with radius r = 1, so OUT_TILE_DIM = 30. Over a 120x120x120 domain, blocks are placed on a 3D grid using ceil division by OUT_TILE_DIM in each dimension. How many thread blocks are launched in total?

Answer: 64

Explanation: Blocks per dim = ceil(120/30) = 4 -> total blocks = 4^3 = 64.


Question: A seven-point 3D stencil uses thread blocks of size IN_TILE_DIMxIN_TILE_DIM = 32x32 (z handled by coarsening). The block processes Z_COARSENING = 16 consecutive output z-planes. With radius r = 1, the block must load halo planes before the first and after the last output plane. How many input elements does a single block load over its lifetime? Assume each loaded plane is 32x32 elements.

Answer: 18432

Explanation: Planes loaded = (16 output) + 2 halo = 18 planes; per plane 32x32=1024 -> 18x1024 = 18,432 elements.


Question: Using IN_TILE_DIM=32 and radius r=1 (so OUT_TILE_DIM = 30) with Z_COARSENING = 16, how many output elements does a single block compute over its lifetime? Each output z-plane contributes OUT_TILE_DIM x OUT_TILE_DIM elements.

Answer: 14400

Explanation: Per plane: 30x30 = 900 outputs; over 16 planes: 900x16 = 14,400 outputs.


Question: For a 3D seven-point stencil with IN_TILE_DIM = 32 and radius r = 1 (so OUT_TILE_DIM = 30), and Z_COARSENING = 16, a block loads 18,432 input elements and computes 14,400 output elements. Assume 32-bit floats (4 bytes) and that each output performs 13 FLOPs (7 multiplies + 6 adds). What is the OP/B ratio for reads only? Do not include units; provide a decimal number.

Answer: 2.5390625

Explanation: FLOPs = 14,400x13 = 187,200. Bytes read = 18,432x4 = 73,728. OP/B = 187,200 / 73,728 ~ 2.5390625.


Question: With IN_TILE_DIM=32 and radius r=1, a coarsened 3D stencil stores three 32x32 tiles in shared memory at once (previous, current, next z-plane). Using 32-bit floats (4 bytes), how much shared memory (bytes) does a block need if register tiling is NOT used?

Answer: 12288

Explanation: 3 tiles x (32x32 elements) x 4 B = 3x1024x4 = 12,288 bytes.


Question: With IN_TILE_DIM=32 and radius r=1, if register tiling is used so that only one 32x32 tile is kept in shared memory at a time, how much shared memory (bytes) does a block need? Assume 32-bit floats (4 bytes).

Answer: 4096

Explanation: 1 tile x (32x32 elements) x 4 B = 1024x4 = 4,096 bytes.


Question: Assume each atomic operation to a single global memory variable (so updates serialize) has a total latency of 100 ns. What is the maximum throughput for these atomic operations in operations per second (ops/s)?

Answer: 10000000

Explanation: Throughput = 1 / (100 ns) = 1 / (100x10^-9 s) = 10,000,000 ops/s.


Question: A GPU supports atomic operations in L2 cache. Each atomic takes 4 ns if it hits in L2 and 100 ns if it goes to DRAM. If 90% of atomic operations hit in L2 and all updates target the same global variable (thus serialize), what is the approximate atomic throughput in ops/s?

Answer: 73529412

Explanation: Average latency = 0.9x4 ns + 0.1x100 ns = 13.6 ns. Throughput ~ 1 / 13.6 ns ~ 73,529,412 ops/s (rounded).


Question: A kernel performs 5 floating-point operations per atomic operation. Each atomic operation to a single global variable has latency 100 ns (so atomics serialize). What is the maximum floating-point throughput (in FLOP/s) limited by the atomic throughput?

Answer: 50000000

Explanation: Atomic throughput = 1/(100 ns) = 10,000,000 ops/s. FLOP/s = 10,000,000 x 5 = 50,000,000.


Question: A kernel replaces global atomics with shared-memory atomics (1 ns per atomic) and incurs an additional 10% total overhead to accumulate privatized results back to global memory. Each atomic corresponds to 5 floating-point operations. Assuming all updates still effectively serialize, what is the maximum floating-point throughput (in FLOP/s)?

Answer: 4545454545

Explanation: Effective per-atomic time = 1 ns x 1.1 = 1.1 ns ⇒ atomic throughput ~ 1/1.1 ns ~ 909,090,909 ops/s. FLOP/s ~ 909,090,909 x 5 ~ 4,545,454,545.

CUDA atomics shared_memory throughput FLOPS


Question: To atomically add the value of an integer variable Partial to a global-memory integer variable Total, which statement is correct? Use the signature: int atomicAdd(int* address, int val);

Choices:
- A. atomicAdd(Total, 1);
- B. atomicAdd(&Total, &Partial);
- C. atomicAdd(Total, &Partial);
- D. atomicAdd(&Total, Partial);

Explanation: atomicAdd expects a pointer to the destination and a by-value increment: atomicAdd(&Total, Partial).


Question:
Consider the CUDA kernel below and assume a single block with blockDim.x = 512 threads, warpSize = 32, and an input array of 1024 elements (each thread initially handles two elements via i = 2*threadIdx.x). During the 5th iteration of the loop (i.e., stride = 16), how many warps in the block have control-flow divergence?

```cpp
__global__ void simple_sum_reduction_kernel(float* input, float* output){
    unsigned int i = 2 * threadIdx.x;
    for (unsigned int stride = 1; stride <= blockDim.x; stride *= 2){
        if (threadIdx.x % stride == 0){
            input[i] += input[i + stride];
        }
        __syncthreads();
    }
    if (threadIdx.x == 0)
        *output = input[0];
}
```

Answer: 16

Explanation: There are 512 threads -> 16 warps. At stride=16, in each warp exactly 2 of 32 lanes execute (threads whose threadIdx.x is a multiple of 16), so every warp diverges -> 16 warps.


Question:
Consider the CUDA kernel below and assume a single block with blockDim.x = 512 threads, warpSize = 32, and an input array of 1024 elements. During the 5th iteration (strides: 512, 256, 128, 64, **32**), how many warps have control-flow divergence?

```cpp
__global__ void ConvergentSumReductionKernel(float* input, float* output) {
    unsigned int i = threadIdx.x;
    for (unsigned int stride = blockDim.x; stride >= 1; stride /= 2) {
        if (threadIdx.x < stride) {
            input[i] += input[i + stride];
        }
        __syncthreads();
    }
    if(threadIdx.x == 0) {
        *output = input[0];
    }
}
```

Answer: 0

Explanation: At stride=32, exactly lanes 0-31 (one full warp) are active and all take the same branch; other warps are fully inactive. No warp mixes taken/not-taken paths -> 0 divergent warps.

CUDA reduction warp_divergence branching


Question:
You want a *reversed* convergent access pattern where active threads add from lower indices (i - stride) instead of higher (i + stride). Starting from the kernel shown below, pick the option that correctly applies this change **without altering the iteration order**. Assume the input has 1024 elements and blockDim.x=512.

Original kernel:
```cpp
__global__ void ConvergentSumReductionKernel(float* input, float* output) {
    unsigned int i = threadIdx.x;
    for (unsigned int stride = blockDim.x; stride >= 1; stride /= 2) {
        if (threadIdx.x < stride) {
            input[i] += input[i + stride];
        }
        __syncthreads();
    }
    if (threadIdx.x == 0) {
        *output = input[0];
    }
}
```
Choices (showing only the lines to change):
A. `unsigned int i = threadIdx.x + blockDim.x;` and inside the loop `if (blockDim.x - threadIdx.x <= stride) { input[i] += input[i - stride]; }` and at the end `if (threadIdx.x == blockDim.x - 1) *output = input[i];`
B. Keep `unsigned int i = threadIdx.x;` but change the loop body to `if (threadIdx.x < stride) { input[i] += input[i - stride]; }`
C. `unsigned int i = threadIdx.x + blockDim.x;` and keep the original condition `if (threadIdx.x < stride) { input[i] += input[i - stride]; }`
D. Keep the original kernel unchanged (already reversed by iteration order).

Choices:
- A
- B
- C
- D

Explanation: To read i-stride safely, initialize i one blockDim.x ahead and mirror the active-lane condition. A ensures in-bounds access and preserves convergence; B/C can access negative indices or mis-activate lanes; D does not implement reversal.


Question:
You need to modify a block-level sum reduction to compute the **maximum** instead. Assume the GPU supports `atomicMax(float*, float)` and the usual `fmax` intrinsic. Which change set is **correct**?

A. Replace all `+`/`+=` with `fmax(..., ...)` in both the per-thread coarsening stage and the shared-memory tree, and replace the final `atomicAdd(output, value)` with `atomicMax(output, value)`.
B. Replace `+` with `fmax` only in the shared-memory tree; keep `atomicAdd` at the end.
C. Keep the tree as sum; at the end do `*output = fmax(*output, blockMax)` without atomics.
D. Compute max on the host after copying partial sums from device.

Choices:
- A
- B
- C
- D

Explanation: Max must be used consistently in *all* accumulation phases, and the final cross-block combine must be an atomic max. B mixes ops; C is racy; D changes the algorithmic contract.

CUDA reduction atomics parallel_algorithms


Question:
Consider a coarsened block-level reduction where each thread sums multiple elements spaced by `BLOCK_DIM`. The input length `N` may not be a multiple of the tile span. Which guard pattern prevents out-of-bounds reads while preserving correctness?

A.
```cpp
float sum = 0.0f;
if (i < N) {
  sum = input[i];
  for (unsigned int t = 1; t < COARSE_FACTOR*2; ++t)
    if (i + t*BLOCK_DIM < N) sum += input[i + t*BLOCK_DIM];
}
```
B.
```cpp
float sum = input[i];
for (unsigned int t = 1; t < COARSE_FACTOR*2; ++t)
  if (i + t*BLOCK_DIM < N) sum += input[i + t*BLOCK_DIM];
```
C.
```cpp
float sum = 0.0f;
if (i <= N) {
  sum = input[i];
  for (unsigned int t = 1; t <= COARSE_FACTOR*2; ++t)
    sum += input[i + t*BLOCK_DIM];
}
```
D.
```cpp
float sum = 0.0f;
if (i < N) sum = input[i];
for (unsigned int t = 1; t < COARSE_FACTOR*2; ++t)
  sum += input[i + t*BLOCK_DIM];
```

Choices:
- A
- B
- C
- D

Explanation: A guards both the base element and each strided read. B can read input[i] out of bounds when i ≥ N. C uses off-by-one bounds and unguarded inner reads. D leaves inner reads unguarded.


Question: A single CUDA block of 8 threads performs an inclusive prefix scan on X = [4, 6, 7, 1, 2, 8, 5, 2] using the classic Kogge-Stone pattern. Strides run with values s = 1, 2, 4. For each stride, every thread tid >= s adds the value from position tid - s that was produced in the previous stride, while threads with tid < s keep their current value. After completing all three strides, which vector remains in X?

Choices:
- A. [4, 10, 17, 18, 20, 28, 33, 35]
- B. [4, 10, 13, 8, 10, 10, 13, 7]
- C. [4, 6, 7, 1, 2, 8, 5, 2]
- D. [4, 6, 13, 14, 16, 26, 31, 33]

Explanation: Stride 1 adds neighbors offset by 1, stride 2 adds neighbors offset by 2 drawing from the stride-1 results, and stride 4 adds neighbors offset by 4. The cumulative effect is the inclusive prefix sums [4,10,17,18,20,28,33,35].


Question: For an inclusive Kogge-Stone scan across N = 2048 elements, the theoretical number of floating-point additions is given by N * log2(N) - (N - 1). Compute this value and provide it as an integer.

Answer: 20481

Explanation: log2(2048) = 11, so 2048 * 11 = 22528 and subtracting (2048 - 1) = 2047 gives 22528 - 2047 = 20481 additions.


Question: In the Kogge-Stone scan kernel shown below, assume blockDim.x = 1024 and warp size = 32.\n\n```cpp\n__global__ void koggeStone(float *X, float *Y, unsigned int N) {\n    __shared__ float XY[1024];\n    unsigned int i = blockIdx.x * blockDim.x + threadIdx.x;\n    if (i < N) {\n        XY[threadIdx.x] = X[i];\n    } else {\n        XY[threadIdx.x] = 0.0f;\n    }\n    for (unsigned int stride = 1; stride < blockDim.x; stride *= 2) {\n        __syncthreads();\n        float temp;\n        if (threadIdx.x >= stride) {\n            temp = XY[threadIdx.x] + XY[threadIdx.x - stride];\n        }\n        __syncthreads();\n        if (threadIdx.x >= stride) {\n            XY[threadIdx.x] = temp;\n        }\n    }\n    if (i < N) {\n        Y[i] = XY[threadIdx.x];\n    }\n}\n```\n\nFor how many stride values does control divergence occur within warp 0?

Choices:
- A. 3
- B. 5
- C. 16
- D. 32

Explanation: Warp 0 contains threads 0-31. Divergence happens while stride is 1, 2, 4, 8, or 16 because only a subset of that warp satisfies threadIdx.x >= stride. Once stride reaches 32 or larger, either all warp-0 threads are inactive or all evaluate the branch the same way.


Question: A Kogge-Stone scan processes N = 2048 elements using two blocks of 1024 threads. Within each block, the numbers of active threads at strides s = 1, 2, 4, 8, 16, 32, 64, 128, 256, 512 are 1023, 1022, 1020, 1016, 1008, 992, 960, 896, 768, and 512 respectively. Each active thread performs one addition per stride. After summing both blocks and adding 1024 more additions to propagate block sums, how many additions are executed in total? Provide the integer result.

Answer: 19458

Explanation: Per block the additions sum to 1023+1022+1020+1016+1008+992+960+896+768+512 = 9217. Two blocks contribute 2 * 9217 = 18434 additions, and adding 1024 for inter-block propagation yields 19458.


Question: A Brent-Kung scan on N = 2048 elements uses one block of 1024 threads. During the reduction tree phase, the active thread counts per stride are 1024, 512, 256, 128, 64, 32, 16, 8, 4, 2, 1. During the reverse tree phase, the active counts per stride are 1, 4, 7, 15, 31, 63, 127, 254, 509, 1019. If each active thread executes one addition at its stride, how many additions occur in total across both phases? Provide the integer total.

Answer: 4077

Explanation: The reduction phase contributes 1024+512+256+128+64+32+16+8+4+2+1 = 2047 additions. The reverse phase contributes 1+4+7+15+31+63+127+254+509+1019 = 2030 additions. The combined total is 2047 + 2030 = 4077.


Question:
The function below implements co_rank for merging two sorted arrays using zero-based indices.

```cpp
int co_rank(int k, const int* A, int m, const int* B, int n) {
    int i = (k < m) ? k : m;
    int j = k - i;
    int i_low = (0 > (k - n)) ? 0 : (k - n);
    int j_low = (0 > (k - m)) ? 0 : (k - m);
    while (true) {
        if (i > 0 && j < n && A[i - 1] > B[j]) {
            int delta = (i - i_low + 1) >> 1;
            j_low = j;
            j += delta;
            i -= delta;
        } else if (j > 0 && i < m && B[j - 1] >= A[i]) {
            int delta = (j - j_low + 1) >> 1;
            i_low = i;
            i += delta;
            j -= delta;
        } else {
            return i;
        }
    }
}
```

Let A = [1, 7, 8, 9, 10], B = [7, 10, 10, 12], and let C be the merged array of length m + n. For k = 8 (the start of the output suffix C[8:]), which pair (i, j) = (co_rank(k), k - co_rank(k)) does the function compute?

Choices:
- A. (4, 4)
- B. (5, 3)
- C. (8, 0)
- D. (3, 5)

Explanation: The merged array is [1,7,7,8,9,10,10,12]. Position k = 8 is past all five elements taken from A, so co_rank returns i = 5. The remaining offset is j = k - i = 3.


Question: Two sorted arrays of lengths 1030400 and 608000 are merged on the GPU using a basic co-rank kernel. Each thread is assigned exactly 8 output elements (elementsPerThread = 8), and every thread performs two binary searches via co_rank before calling a sequential merge. How many threads in total execute the binary searches when the entire merge is launched? Provide the integer count.

Answer: 204800

Explanation: The merged output has 1030400 + 608000 = 1638400 elements. Dividing by 8 elements per thread gives 1638400 / 8 = 204800 threads, and every thread runs both co_rank searches.


Question: The tiled merge kernel splits work by assigning exactly 8 output elements to each thread within a 1024-thread block. Before processing its tile, thread 0 in every block computes co_rank for the block's starting and ending output indices. Using the same arrays (1030400 and 608000 elements) and 8 elements merged per thread, how many threads in total perform binary searches against global memory? Provide the integer count.

Answer: 200

Explanation: Each 1024-thread block merges 8 x 1024 = 8192 elements. The total output length 1638400 divided by 8192 gives 200 blocks. Only thread 0 in each block issues the two co_rank calls, so 200 threads perform binary searches.


Question:
A benchmark with N = 10,000,000 elements produced the timings below (in milliseconds) and GPU speedups versus a CPU quicksort baseline:

- Naive Parallel Radix Sort: time = 14.099, speedup = 149.70x
- Memory-coalesced GPU sort: time = 18.055, speedup = 116.90x
- Memory-coalesced GPU sort (multiradix): time = 20.671, speedup = 102.11x
- Memory-coalesced GPU sort (multiradix + thread coarsening): time = 39.275, speedup = 53.74x
- GPU merge sort: time = 135.828, speedup = 15.54x

Which method delivered the highest speedup?

Choices:
- A. Memory-coalesced GPU sort
- B. Memory-coalesced GPU sort (multiradix)
- C. Memory-coalesced GPU sort (multiradix + thread coarsening)
- D. Naive Parallel Radix Sort

Explanation: 149.70x is the largest reported speedup, achieved by the Naive Parallel Radix Sort entry.


Question: Using the benchmark data above, compute the difference in reported speedup between Naive Parallel Radix Sort (149.70x) and GPU merge sort (15.54x). Give the result rounded to two decimal places as a decimal number (e.g., 12.34).

Answer: 134.16

Explanation: 149.70 - 15.54 = 134.16 when rounded to two decimal places.


Question: A single-kernel variant of the parallel radix sort attempts to synchronize across the entire grid to reuse the total count of zero bits. Beyond roughly 100,000 elements it deadlocks. Why is this approach fragile?

Choices:
- A. Because GPU quicksort launches interfere with the kernel
- B. Because the first block must wait for the last block without a device-wide synchronization primitive
- C. Because the radix counters overflow 32-bit integers
- D. Because thread coarsening increases register pressure beyond limits

Explanation: The README notes that the single-kernel try stalls because the first block needs the global zero count from the last block, effectively requiring a grid-wide barrier that the vanilla CUDA launch cannot provide.

radix_sort synchronization CUDA pitfalls


Question:
Consider the 4x4 sparse matrix

[ [1, 0, 7, 0],
  [0, 0, 8, 0],
  [0, 4, 3, 0],
  [2, 0, 0, 1] ].

Using zero-based indices and listing nonzeros in row-major order, which triple of arrays encodes this matrix in COO format (row indices, column indices, values)?

Choices:
- A. row=[0,0,1,2,3,3,3], col=[0,2,2,1,0,2,3], val=[1,7,8,4,2,3,1]
- B. row=[0,0,1,2,2,3,3], col=[0,2,2,1,2,0,3], val=[1,7,8,4,3,2,1]
- C. row=[0,1,1,2,2,3,3], col=[0,2,3,0,2,0,3], val=[1,7,8,4,3,2,1]
- D. row=[0,0,2,2,3,3,3], col=[0,2,1,2,0,2,3], val=[1,7,4,3,2,1,8]

Explanation: Row-major enumeration yields nonzeros (0,0),(0,2),(1,2),(2,1),(2,2),(3,0),(3,3) with values [1,7,8,4,3,2,1]; option B matches this ordering exactly.


Question: The same 4x4 matrix is stored in CSR format using zero-based indexing and row-major ordering of nonzeros. What is the CSR row pointer array rowPtr? Provide it as a tuple (r0,r1,r2,r3,r4).

Answer: (0,2,3,5,7)

Explanation: Row lengths are [2,1,2,2]; the prefix sums give rowPtr = [0,2,3,5,7].


Question: You know only that a sparse matrix has m rows, n columns, and z nonzeros. Which statement correctly identifies the additional information needed to determine memory usage for different formats?

Choices:
- A. COO storage cannot be determined from m, n, z because row indices depend on padding information.
- B. CSR needs the maximum row length to size its row pointer array.
- C. Both ELL and JDS require the maximum number of nonzeros in any row (and related row ordering details) to size their auxiliary arrays.
- D. JDS needs only m, n, z because the iterPtr array always has length m + 1.

Explanation: COO and CSR can be sized directly from z and m (rowPtr has length m+1). ELL and JDS must know the longest row to size padding/iterPtr, and JDS also needs the row permutation order.


Question:
A scale-free graph with 20,000 vertices was benchmarked using several BFS variants. The reported runtimes (milliseconds) and GPU speedups versus a sequential baseline were:

- Sequential BFS: 10.26 ms (reference)
- Push Vertex-Centric BFS: 1.44 ms (7.12x)
- Pull Vertex-Centric BFS: 0.44 ms (23.53x)
- Edge-Centric BFS: 0.19 ms (54.81x)
- Frontier-based BFS: 2.49 ms (4.11x)
- Optimized Frontier-based BFS: 2.60 ms (3.95x)
- Direction-Optimized BFS: 0.48 ms (21.54x)

Which variant achieved the highest reported speedup?

Choices:
- A. Pull Vertex-Centric BFS
- B. Edge-Centric BFS
- C. Direction-Optimized BFS
- D. Push Vertex-Centric BFS

Explanation: Edge-Centric BFS reached 54.81x, larger than the pull (23.53x), direction-optimized (21.54x), or push (7.12x) variants.


Question: For the scale-free graph with 10,000 vertices, the benchmark reported Sequential BFS = 4.71 ms and Direction-Optimized BFS = 0.35 ms. What is the absolute time saved (Sequential minus Direction-Optimized) in milliseconds, rounded to two decimals?

Answer: 4.36

Explanation: Time saved = 4.71 ms - 0.35 ms = 4.36 ms.


Question:
Consider a directed graph with vertices 0..7 and adjacency lists:
0->{5,2}, 1->{4}, 2->{3}, 3->{6}, 4->{}, 5->{1,7}, 6->{}, 7->{4,6}.
A level-synchronous vertex-centric push BFS launches one thread per vertex each iteration. Starting from source vertex 0, iteration 1 visits {0}; iteration 2 processes frontier {5,2}; iteration 3 processes frontier {1,7,3}. During iteration 3, how many threads actually iterate over their vertex's neighbor list?

Choices:
- A. 1
- B. 2
- C. 3
- D. 8

Explanation: Iteration 3's frontier contains three vertices (1, 7, 3); only their threads traverse neighbor lists, while the remaining five threads are idle.


Question:
The pooling forward routine below processes one feature map with stride K and averages when pooling_type is "avg":

```cpp
void poolingLayer_forward(int M, int H, int W, int K, float* Y, float* S, const char* pooling_type) {
    for (int m = 0; m < M; m++)
        for (int h = 0; h < H/K; h++)
            for (int w = 0; w < W/K; w++) {
                if (strcmp(pooling_type, "max") == 0)
                    S[m*(H/K)*(W/K) + h*(W/K) + w] = -FLT_MAX;
                else
                    S[m*(H/K)*(W/K) + h*(W/K) + w] = 0.0f;
                for (int p = 0; p < K; p++)
                    for (int q = 0; q < K; q++) {
                        float val = Y[m*H*W + (K*h + p)*W + (K*w + q)];
                        if (strcmp(pooling_type, "max") == 0) {
                            if (val > S[m*(H/K)*(W/K) + h*(W/K) + w])
                                S[m*(H/K)*(W/K) + h*(W/K) + w] = val;
                        } else {
                            S[m*(H/K)*(W/K) + h*(W/K) + w] += val / (K*K);
                        }
                    }
            }
}
```

Take M=1, H=4, W=4, K=2, pooling_type="avg", and the input feature map
Y = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]].
What value does the kernel store at output position (h=0, w=0)? Provide the result rounded to two decimals.

Answer: 3.50

Explanation: The 2x2 window [[1,2],[5,6]] averages to 3.5, so the stored value is 3.50.


Question: For input size [8, 64, 128, 128], the custom max pooling benchmark reports 82.433 ms while PyTorch reports 9.454 ms. Compute the speedup of PyTorch relative to the custom kernel (custom_time / pytorch_time) and round to two decimals.

Answer: 8.72

Explanation: Speedup = 82.433 / 9.454 = 8.719..., which rounds to 8.72.


Question:
The CUDA kernel below unrolls an input tensor X into X_unroll for convolution.

```cpp
__global__ void unroll_Kernel(int C, int H, int W, int K, float* X, float* X_unroll) {
    int t = blockIdx.x * blockDim.x + threadIdx.x;
    int H_out = H - K + 1;
    int W_out = W - K + 1;
    int W_unroll = H_out * W_out;
    if (t < C * W_unroll) {
        int c = t / W_unroll;
        int w_unroll = t % W_unroll;
        int h_out = w_unroll / W_out;
        int w_out = w_unroll % W_out;
        int w_base = c * K * K;
        for (int p = 0; p < K; p++)
            for (int q = 0; q < K; q++) {
                int h_unroll = w_base + p*K + q;
                X_unroll[h_unroll * W_unroll + w_unroll] = X[c * H * W + (h_out + p) * W + (w_out + q)];
            }
    }
}
```

Consider consecutive thread indices t and t+1 that map to the same channel (same c) and satisfy w_out < W_out - 1. For a fixed (p, q) iteration, what is the difference between their global memory read addresses in X?

Choices:
- A. 0
- B. 1
- C. W
- D. K*K

Explanation: With c and h_out identical and w_out incremented by one, the linearized address in X increases by exactly 1 element, yielding coalesced reads.


Question:
Consider the loop-fission version of the FHD computation (indices are zero-based):

```cpp
for (int m = 0; m < M; m++) {
    rMu[m] = rPhi[m]*rD[m] + iPhi[m]*iD[m];
    iMu[m] = rPhi[m]*iD[m] - iPhi[m]*rD[m];
}
for (int m = 0; m < M; m++) {
    for (int n = 0; n < N; n++) {
        rFhD[n] += rMu[m]*cArg(m,n) - iMu[m]*sArg(m,n);
        iFhD[n] += iMu[m]*cArg(m,n) + rMu[m]*sArg(m,n);
    }
}
```

Assume M = 4 and N = 3. Before the very first update to rFhD occurs in the second loop, how many times have the statements computing rMu and iMu (lines 2-3) executed? Give the integer count.

Answer: 4

Explanation: The first loop completes all M iterations before the second loop starts, so lines 2-3 execute once for each m = 0,1,2,3, totalling 4 executions.


Question:
The original loop nests iterate with n outer and m inner:

```cpp
for (int n = 0; n < N; n++) {
    for (int m = 0; m < M; m++) {
        body(n, m);
    }
}
```

After loop interchange, the outer loop runs over m first. For M = 2 and N = 3, what is the sequence of (n,m) pairs visited in the **interchanged** version?

Choices:
- A. (0,0), (1,0), (2,0), (0,1), (1,1), (2,1)
- B. (0,0), (0,1), (0,2), (1,0), (1,1), (1,2)
- C. (0,0), (0,1), (1,0), (1,1), (2,0), (2,1)
- D. (0,0), (0,1), (1,1), (1,2), (0,2), (1,0)

Explanation: With m outermost, we hold m=0 and sweep n=0..2, then set m=1 and sweep n=0..2, giving (0,0),(0,1),(0,2),(1,0),(1,1),(1,2).


Question:
A CUDA kernel assigns each thread a unique n index:

```cpp
int n = blockIdx.x * FHD_THREADS_PER_BLOCK + threadIdx.x;
float xn = x[n];
for (int m = 0; m < M; m++) {
    float term = kx[m] * xn;
    // ... uses ky[m], kz[m], rMu[m], iMu[m]
}
```

Which statement best explains why caching kx[n] in a register would be useless for this kernel?

Choices:
- A. kx[n] equals zero for all n, so caching offers no benefit.
- B. Threads need kx values indexed by m inside the loop, not by n, so kx[n] is never referenced.
- C. kx is stored in shared memory already, making registers redundant.
- D. Registers cannot hold floating-point values across loop iterations.

Explanation: The loop repeatedly accesses kx[m] for m = 0..M-1; the value kx[n] is never used, so loading it would waste instructions and registers.


Question:
The host routine below launches a gather kernel in chunks of constant memory. Assume `CHUNK_SIZE = 256`.

```cpp
void cenergyParallelGather(float* energygrid, dim3 grid_dim, float gridspacing, float z,
                           const float* host_atoms, int numatoms) {
    int num_chunks = (numatoms + CHUNK_SIZE - 1) / CHUNK_SIZE;
    for (int chunk = 0; chunk < num_chunks; ++chunk) {
        int start_atom = chunk * CHUNK_SIZE;
        int atoms_in_chunk = (start_atom + CHUNK_SIZE <= numatoms) ? CHUNK_SIZE
                                                                   : (numatoms - start_atom);
        // cudaMemcpyToSymbol atoms_in_chunk elements and launch kernel
    }
}
```

If `numatoms = 1000`, how many times will the body of the loop (and thus the constant-memory copy) execute? Give the integer count.

Answer: 4

Explanation: num_chunks = ceil(1000 / 256) = (1000 + 255) / 256 = 1255 / 256 = 4 when using integer division with the bias term.


Question:
For a coarsening factor of 8, the README compares one iteration of the original gather kernel (original gather kernel) to one iteration of the coarsened kernel (coarsened kernel). The counts per outer iteration are:
- original gather kernel: 32 memory loads, 88 floating-point operations, 8 branch evaluations.
- coarsened kernel: 11 memory loads, 61 floating-point operations, 17 branch evaluations.

Which statement best summarizes the trade-off introduced by thread coarsening?

Choices:
- A. Memory loads decrease, arithmetic decreases, but branch count rises.
- B. Memory loads increase, arithmetic increases, and branch count drops.
- C. Memory loads and arithmetic both decrease to zero, so only branches remain.
- D. All three categories (loads, arithmetic, branches) grow because more work is assigned to each thread.

Explanation: Coarsening reduces global loads (32->11) and arithmetic (88->61) but increases the number of branch evaluations (8->17).


Question: Section 18.3 notes two drawbacks of increasing the work per CUDA thread (high thread coarsening). Which pair captures those concerns?

Choices:
- A. Higher register usage can cut occupancy, and excessive coarsening can leave many cores idle.
- B. Device memory becomes read-only, and shared memory can no longer be allocated.
- C. Warp size expands beyond 32, and thread blocks stop synchronizing correctly.
- D. Atomic operations become mandatory, and constant memory broadcasts no longer function.

Explanation: More work per thread generally consumes more registers (reducing occupancy) and risks underutilizing parallelism if too few threads remain active.


Question: A 25-point stencil operates on a 64x64x2048 grid that is decomposed along z across 17 MPI ranks: 16 compute ranks plus 1 data server. Each compute rank receives 128 consecutive z-slices (ignoring halos). During Stage 2 of the described two-stage exchange (Stage 1: 4-slice boundaries; Stage 2: interior slices) the internal compute ranks process the interior z-slices after the four-slice boundaries are done in Stage 1. How many interior grid points does one internal compute rank update in Stage 2? Provide the integer count.

Answer: 376320

Explanation: Stage 1 covers four front and four back slices, leaving 120 interior slices. Interior work = 56x56x120 = 376,320 points per internal rank.


Question: Using the same 64x64x2048 grid split evenly over 16 compute ranks (plus 1 data server), how many halo grid points must an internal compute rank exchange in Stage 2 of the described two-stage exchange (Stage 1: 4-slice boundaries; Stage 2: interior slices)? Assume four halo slices on each side and single-precision floats.

Choices:
- A. 16,384 points (65,536 bytes)
- B. 32,768 points (131,072 bytes)
- C. 65,536 points (262,144 bytes)
- D. 524,288 points (2,097,152 bytes)

Explanation: Each side requires 4x64x64 = 16,384 points; two sides double that to 32,768 points ~ 131 KB when using 4-byte floats.


Question: The call `MPI_Send(ptr_a, 1000, MPI_FLOAT, 2000, 4, MPI_COMM_WORLD)` transmits 4,000 bytes. What is the size of each element?

Choices:
- A. 1 byte
- B. 2 bytes
- C. 4 bytes
- D. 8 bytes

Explanation: 4,000 byte total / 1,000 elements = 4 bytes per element, consistent with MPI_FLOAT.


Question: Which MPI statement is correct?

Choices:
- A. MPI_Send is the only nonblocking send routine.
- B. MPI_Recv blocks until a matching message arrives.
- C. MPI messages must be at least 128 bytes.
- D. Separate MPI ranks share the same global memory space.

Explanation: MPI_Recv is a blocking receive; the other statements are false.


Question:
The parent kernel below launches a child kernel for each Bezier line.

```cpp
typedef float2 float2;
struct BezierLine {
    float2 CP[3];
    float2 *vertexPos;
    int nVertices;
};
__global__ void computeBezierLines_parent(BezierLine *lines, int nLines) {
    int lidx = threadIdx.x + blockDim.x * blockIdx.x;
    if (lidx < nLines) {
        float curvature = 1.0f; // assume curvature chosen elsewhere
        lines[lidx].nVertices = min(max((int)(curvature * 16.0f), 4), 1024);
        cudaMalloc((void**)&lines[lidx].vertexPos, lines[lidx].nVertices * sizeof(float2));
        int childBlocks = (lines[lidx].nVertices + 31) / 32;
        computeBezierLine_child<<<childBlocks, 32>>>(lidx, lines, lines[lidx].nVertices);
    }
}
```
If `nLines = 1024` and the parent is launched with `computeBezierLines_parent<<<16, 64>>>(...)`, how many child kernels are launched in total?

Choices:
- A. 16
- B. 32
- C. 256
- D. 1024

Explanation: Each of the 1024 parent threads (16 blocks x 64 threads) passes the `lidx < nLines` check and launches exactly one child grid, so 1024 child kernels execute.


Question: For a launch with (`nLines = 1024`, `<<<16,64>>>`), suppose each parent thread creates a non-blocking CUDA stream before launching `computeBezierLine_child` and destroys it afterwards. How many streams are created across the entire parent launch? Provide the integer count.

Answer: 1024

Explanation: One stream is created per parent thread, and 16x64 = 1024 parent threads execute, so 1024 streams are created and later destroyed.


Question: A quadtree recursively subdivides a square region until each leaf contains at most one of 64 equidistant points. Including the root node, what is the maximum depth of the quadtree?

Choices:
- A. 4
- B. 8
- C. 16
- D. 64

Explanation: Depth 0 holds all 64 points. Each subdivision splits into 4 equal tiles. After three splits (depths 1-3) there are 64 leaves with one point each, so the maximum depth including the root is 4.


Question: In that quadtree, every node at depths 0, 1, and 2 launches a child kernel for each of its four quadrants. How many parent nodes issue a child-kernel launch in total?

Choices:
- A. 4
- B. 16
- C. 21
- D. 64

Explanation: Launching parents include the root (1 node), all depth-1 nodes (4 nodes), and all depth-2 nodes (16 nodes); 1 + 4 + 16 = 21 parent launches.