Pinned Loading

1. llm-d/llm-d llm-d/llm-d Public

   Achieve state of the art inference performance with modern accelerators on Kubernetes

   Python 4.8k 854

2. llm-d/llm-d-router llm-d/llm-d-router Public

   llm-d Router: The intelligent entry point for inference requests

   Go 414 459

3. llm-d/llm-d-kv-cache llm-d/llm-d-kv-cache Public

   Distributed KV cache scheduling & offloading libraries

   Go 186 165

4. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 93.9k 23.5k