vllm

Past 30d, sorted by best first

09/27	8	vllm-project/vLLM Semantic Router v0.4.0 System-level router that routes and manages requests across heterogeneous LLMs with signal-driven decisions, semantic caching, and security. Go 6019☆ 403d old #golang #kubernetes #go #ai-gateway #huggingface-candle
09/20	5	micytao/vLLM Playground v0.1.9 Modern web interface for managing and interacting with vLLM servers, supporting GPU/CPU modes, macOS Apple Silicon optimizations, and OpenShift/Kubernetes deployment. JavaScript 535☆ 329d old #javascript #ai #learning #llms #vllm