cuda.syncthreads() — every warp in the block waits here until all of them arrive.
C — thread view
A — global memory
B — global memory
Registers — selected warp
tileA — shared memory
tileB — shared memory
Code (current line highlighted)
This step, for the selected warp
Lane inspector
warp reads/writes distinct addresses
broadcast — all 32 lanes hit one address
accumulating in registers
stored to C
zero-filled (out of bounds) or zero padding
held in a WMMA fragment (lane mapping unspecified)
32-byte sector fetched
selected lane
held in a register, untouched this step
already loaded on an earlier pass
other warps (with "show all warps")
C cells tinted by owning warp (selected warp brighter)
Side by side
| Kernel | Warp covers | Global sectors per warp per k-step (A + B) | Shared reads per FMA (per thread) | Timing | What to look at |
|---|---|---|---|---|---|
| Lecture 8 / Week 2-4 · M = 1000, K = 1200, N = 800, fp32 · timings are T4 Nsight from the live session | |||||
| 1 · warps down columns | a column of C | 32 + 1 = 33 | — | ~64 ms | uncoalesced loads + stores, LG throttle |
| 2 · swap x/y | a row of C | 1 + 4 = 5 | — | ~16 ms | ALU busiest (index math) |
| 3 · 32×32 smem tiles | a row of C | (4 + 4) once per 32 k-steps = 0.25 | 2 | ~5 ms | MIO throttle (too many smem reads) |
| 4 · 1D block tiling, TM = 8 | 8 rows of C (each lane: an 8-tall column strip) | (32 + 32) once per 32 k-steps = 2, for 256 outputs | 9 / 8 = 1.125 | not read out | MIO throttle gone |
| Lecture 10 · M = N = K = 1024, fp32 · not run on a GPU for this page | |||||
| 5 · 2D block tiling, 4×4 patches | 8 rows × 64 cols (two rows of 16 patches) | (16 + 16) once per 16 k-steps = 2, for 512 outputs | 8 / 16 = 0.5 | — | compute reads: 2-way bank conflicts on tileA and tileB |
| 6 · warp tiling + transposed A | 16 × 32 region (4 rows of 8 patches) | (16 + 16 distinct) per 16 k-steps = 2, for 512 outputs | 8 / 16 = 0.5 | — | compute reads conflict-free; the grouped tileA and tileB stores are 4-way (bank model) |
| 7 · kernel factory, 11 configs | WM × WN, chosen per config | depends on config | (TM+TN)/(TM·TN): 0.5, 0.375 or 0.25 | — | same algorithm as 6; the host searches tile shapes |
| Lecture 12 · fp16 in, fp32 accumulate · WMMA 16×16×16 · not run on a GPU for this page | |||||
| TC-01 · one warp, one tile | the whole 16×16 C (collectively) | 16 + 16 once, for 256 outputs | no shared memory | — | B declared col_major; a fragment's lane mapping is unspecified |
| TC-02 · padded any-size | one 16×16 C tile per warp, 4 warps per block | (16 + 16) per 16 k-steps = 2, for 256 outputs | no shared memory | — | K padded to 1008; the 4 warps each fetch the same B tile separately |