GitHub Release Tracker
All JS React Ruby Go Postgres Frontend Node
24h 7d 14d 30d | Newest Best Latest version only Popular

llm-inference

Past 30d, sorted by best first, all versions
13 results Markdown version
08/11 7
kubernetes-sigs/The LeaderWorkerSet API (LWS) v0.10.0
API for deploying and managing groups of pods as a single unit with leader and worker roles for multi-host inference workloads.
Go 789☆ 909d old #golang #go #sig-apps #llm-inference
08/10 6
tetherto/QVAC sdk-v0.17.0
Local-first, cross-platform SDK for building peer-to-peer AI apps with local model inference, speech, translation, and RAG.
TypeScript 556☆ 238d old #javascript #typescript #ai #llm #llama
08/04 6
ome-projects/OME (Open Model Engine) v1.2.2
Kubernetes operator for enterprise-grade management and serving of large language models with model lifecycle automation, runtime selection, and GPU scheduling.
Go 490☆ 449d old #golang #go #k8s #llm-inference #model-as-a-service
08/26 5
tetherto/QVAC sdk-v0.18.2
Local-first, cross-platform SDK for building peer-to-peer AI apps with local model inference, speech, translation, and RAG.
TypeScript 556☆ 238d old #javascript #typescript #ai #llm #llama
08/24 5
tetherto/QVAC sdk-v0.18.1
Local-first, cross-platform SDK for building peer-to-peer AI apps with local model inference, speech, translation, and RAG.
TypeScript 556☆ 238d old #javascript #typescript #ai #llm #llama
08/14 5
tetherto/QVAC sdk-v0.17.1
Local-first, cross-platform SDK for building peer-to-peer AI apps with local model inference, speech, translation, and RAG.
TypeScript 556☆ 238d old #javascript #typescript #ai #llm #llama
08/10 5
thushan/Olla v0.0.29
High-performance LLM proxy and load balancer providing intelligent routing, automatic failover, and unified model discovery across local and remote inference backends.
Go 273☆ 445d old #golang #ai #go #ollama #proxy
08/17 3
Avarok-Cybersecurity/Atlas Inference Engine b236
Pure Rust LLM inference engine focused on fast, hardware-optimized local model execution across GPUs and CPUs.
JavaScript 650☆ 104d old #javascript #cuda #llm-inference #gb10 #mamba
08/17 3
Avarok-Cybersecurity/Atlas Inference Engine b234
Pure Rust LLM inference engine focused on fast, hardware-optimized local model execution across GPUs and CPUs.
JavaScript 650☆ 104d old #javascript #cuda #llm-inference #gb10 #mamba
08/16 3
Avarok-Cybersecurity/Atlas Inference Engine b233
Pure Rust LLM inference engine focused on fast, hardware-optimized local model execution across GPUs and CPUs.
JavaScript 650☆ 104d old #javascript #cuda #llm-inference #gb10 #mamba
08/16 3
Avarok-Cybersecurity/Atlas Inference Engine b232
Pure Rust LLM inference engine focused on fast, hardware-optimized local model execution across GPUs and CPUs.
JavaScript 650☆ 104d old #javascript #cuda #llm-inference #gb10 #mamba
08/16 3
Avarok-Cybersecurity/Atlas Inference Engine b231
Pure Rust LLM inference engine focused on fast, hardware-optimized local model execution across GPUs and CPUs.
JavaScript 650☆ 104d old #javascript #cuda #llm-inference #gb10 #mamba
08/16 3
Avarok-Cybersecurity/Atlas Inference Engine b230
Pure Rust LLM inference engine focused on fast, hardware-optimized local model execution across GPUs and CPUs.
JavaScript 650☆ 104d old #javascript #cuda #llm-inference #gb10 #mamba