llm-inference

Past 14d, sorted by best first

10/16	8	kubernetes-sigs/The LeaderWorkerSet API (LWS) v1.0.0 API for deploying and managing groups of pods as a single unit with leader and worker roles for multi-host inference workloads. Go 880☆ 968d old #golang #go #sig-apps #llm-inference
10/03	7	lordmathis/llamactl v0.22.0 Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard. Go 156☆ 444d old #golang #llm #go #llama-cpp #llamacpp
10/04	6	tetherto/QVAC sdk-v0.21.0 Local-first, cross-platform SDK for building peer-to-peer AI apps with local model inference, speech, translation, and RAG. TypeScript 697☆ 288d old #javascript #typescript #ai #llm #llama