Nathan's Blog

It traces the execution from the PyTorch function, through the launcher's setup (grid, block sizes), to the highly-optimized Triton JIT kernel code.

#FlashAttention #Triton #LLMs #GPUKernel #DeepLearning