LLM Inference video series | unicodesnwrites

Part 1

How LLM Inference Works

From prompt to tokens: prefill, decode and why a model writes one token at a time.

Part 2

Why models cache keys and values, and what that saves on every new token.

Part 3

Decoding is limited by memory bandwidth, not compute, and that shapes everything else.

Part 4

Static, dynamic and continuous batching, and how servers keep the GPU busy.

Part 5

Shrinking weights to 8 and 4 bits to fit bigger models and read memory faster.

COMING 15 NOV, 19:30 IST

Part 6

Tiling attention so it never writes the full attention matrix to slow memory.

COMING 22 NOV, 19:30 IST

Part 7

A small draft model guesses ahead and the big model checks several tokens at once.

COMING 29 NOV, 19:30 IST

Part 8

Temperature, top-k and top-p: how the next token is actually picked.

COMING 6 DEC, 19:30 IST

Part 9

Tensor, pipeline and data parallelism for serving models that do not fit on one GPU.

COMING 13 DEC, 19:30 IST