← Dictionary / Scheduler / KV-cache manager / executor

Scheduler / KV-cache manager / executor

Appears in 1 tutorial

vLLM's three conceptual roles: decides what runs (continuous batching), manages KV memory (PagedAttention), and runs the model on the GPU.

As used in LLM Infrastructure →

vLLM's three conceptual roles: decides what runs (continuous batching), manages KV memory (PagedAttention), and runs the model on the GPU.

Appears in

LLM Infrastructure →