#monitoring

Monitoring vLLM in Production: Metrics, PromQL, Alerts, and Runbooks

Published: 2026-01-28 Updated: 2026-06-18 Version scope: This guide targets vLLM 0.23.x and its default V1 engine. Metric names and semantics change between releases, so verify the /metrics output of the exact version and serving configuration you run before copying queries or alerts into production. A vLLM server can pass Kubernetes health checks while users experience slow or unstable responses. Requests may wait in the scheduler, long prompts may make prefill expensive, or token streaming may become uneven. This guide focuses on the metrics needed to diagnose latency, scheduler backlog, KV-

Andrey Krisanov · EN

SRE日报：70+ 信源 + AI 中文解读，每天 5 分钟盯完云运维圈

作者：SRE运维博客 博客地址：https://www.cnsre.cn/ 文章地址：https://www.cnsre.cn/posts/26

SRE运维博客 · ZH

Rachels Don't Run

While monitoring late-night calls at an AI companionship service, lonely customer support agent Leah impersonates an artificial dream girl to chat with the regular…

Vimeo Short of the Week · EN