EN DE
← ← Back to Index

serving-llms-vllm

mlops › inference › vllm v1.0.0
vLLM Inference Serving PagedAttention Continuous Batching High Throughput Production OpenAI API Quantization Tensor Parallelism

Description

vLLM: high-throughput LLM serving, OpenAI API, quantization.

Relevance to Our Projects

vLLM: high-throughput LLM serving, OpenAI API, quantization.

SKILL.md Preview

# vLLM - High-Performance LLM Serving

## When to use

Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

## Quick start

vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill/decode requests).

**Installation**:
```bash
pip install vllm
``

← ← Back to Index