Reading references¶
Checked 2026-09-23. Books supply a learning framework; executable API details are checked against upstream documentation. No paid book text or examples are vendored. Only public publisher descriptions, contents and available excerpts were reviewed, not the full paid books. The code in this repository is a compact teaching implementation of the cited algorithms, with its own contracts, tests and experiments.
Books¶
Open the downloaded previews and book readers. The local files contain Manning's short Chapter 1 brief and O'Reilly's 14-page table of contents. Full ebooks are not included; publisher access links are provided in the index.
- Harshwardhan Fartale, GPU Programming with Triton, Manning, ISBN 9781633434233. Publisher · public welcome excerpt. The publisher currently lists MEAP early access, 4 of 11 chapters available, begun August 2026, with final publication estimated for spring 2027. Treat its scope as a developing roadmap, not a finished reference. Its block-programming, memory access, reductions, fusion and tiling topics guide labs 1–5. Advanced attention topics are follow-on reading; no unavailable chapter content is assumed.
- Chris Fregly, AI Systems Performance Engineering, O'Reilly, 2025, ISBN 9798341627772. Publisher and contents. This is a wider systems book with a substantial Triton component. Read the profiling and benchmarking topics in chapter 13, and chapter 14, “PyTorch Compiler, OpenAI Triton, and XLA Backends,” for the compiler path, kernel launch tuning and autotuning. Use it alongside labs 5–6 and the benchmark methodology. It motivates measuring an entire pipeline as well as one kernel; this small repository only measures isolated forward operators.
Free primary technical references¶
| Reference | Used for |
|---|---|
| Triton installation and supported platforms | Linux setup and hardware expectations |
| Vector addition tutorial | Program IDs, block offsets, masked tails |
| Fused softmax tutorial | Stable row reductions, fusion and memory traffic reasoning |
| Matrix multiplication tutorial | Tile reuse, FP32 accumulation and configuration search |
| Fused attention tutorial | Tiled attention and online numerator/denominator rescaling in advanced workshop A3 |
| Online normalizer calculation for softmax | Mergeable max/sum state in workshops A1-A2 |
| Layer normalization tutorial | Row normalization patterns; our RMSNorm omits mean subtraction and adds a residual |
| NVIDIA CUDA Best Practices Guide | Coalescing, memory hierarchy, occupancy and measurement |
| NVIDIA CUDA Programming Guide | Blocks, synchronization and warp primitives |
| Triton debugging | Interpreter and device-side diagnostics |
| Nsight Compute guide | Interpreting counters and measurement limitations |
Reading order: block programming → memory and masks → reductions → fusion → tiling → measurement and compiler inspection. Run a small experiment after each topic, then describe the result without looking at the source.