gpu-memory-utilization
Flag for the fraction of GPU memory vLLM may use; the key throughput/stability knob (bigger KV cache vs less headroom).
Flag for the fraction of GPU memory vLLM may use; the key throughput/stability knob (bigger KV cache vs less headroom).