PagedAttention

Appears in 2 tutorials

vLLM's memory-management technique enabling efficient batched serving.

As used in Fine-Tuning & Model Customization →

vLLM's memory-management technique enabling efficient batched serving. (M14)

vLLM's KV-cache manager that stores the cache in small fixed-size blocks (pages) allocated on demand via a block table, eliminating waste and fragmentation and enabling much bigger batches. Borrowed from OS virtual memory.

Fine-Tuning & Model Customization → LLM Infrastructure →