Model Serving Signals

Engine-agnostic signal contract for model servers on Kubernetes: common metric semantics, engine profiles (e.g., vLLM, SGLang, TensorRT-LLM), a mapping exporter, a conformance suite, and autoscaling integration.

Model-serving signal contract

See the model server contract for the complete metric table, including standard names, types, units, semantics, and engine-specific metric mappings.

Design

The motivation, scope, and design of this project are described in the model-serving-signals proposal doc.

Extending

• Adding a new signal (metric)
• Adding a new engine

Community, discussion, contribution, and support

Learn how to engage with the Kubernetes community on the community page.

You can reach the maintainers of this project at:

• Slack channel
• Mailing List

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.