docs/vllm.md - Atlas

Launch offer — 40% off every tier for a limited time. Prices shown are already reduced.

docs/vllm.md 25 lines · 507 B · markdown

# vllm

**Licence:** Apache-2.0  |  **Language:** Python  |  **Source:** https://github.com/vllm-project/vllm

## Role in the stack

High-throughput LLM serving

## Why it's here

PagedAttention serving with the best throughput per GPU for open models. The default when you have real GPU capacity.

## Using it

See the project README in the fetched source:

```bash
./fetch-sources.sh ./sources
tar xzf ./sources/vllm.tar.gz -C ./sources
```

## Running it

Started by `docker compose up -d` on port 8000.