GitHub Release Tracker

09/28

8

Nehanth/pooled v1.0.0

Browser-based peer-to-peer LLM inference that pools model layers across devices in a room for fast chat and in-tab coding agents.

JavaScript 568☆ 34d old #javascript #browser #llm #inference #local-llm

10/09

7

kibae/pg_onnx v1.31.0

ONNX Runtime integration enabling machine learning inference within PostgreSQL databases.

C++ 63☆ 1121d old #c++ #ai #deep-learning #cuda #contributions-welcome

09/27

7

llm-d/llm-d Router v0.11.0

llm-d Router provides load and prefix-cache aware inference routing with prioritization and flow control, supporting standalone and Gateway API deployment via an endpoint picker.

Go 373☆ 517d old #golang #ai #kubernetes #go #networking

09/18

7

kaito-project/AIKit v0.23.0

Platform to host, deploy, build, and fine-tune large language models with OpenAI API compatibility.

Go 539☆ 1120d old #golang #docker #llm #go #large-language-models

10/05

6

RunanywhereAI/RunAnywhere cpp-desktop-v0.20.37...

On-device mobile SDKs for running LLMs, speech-to-text, and text-to-speech locally.

C++ 10314☆ 446d old #swift #c++ #llm #ios #kotlin

09/14

6

NVIDIA/NVIDIA Cloud Functions (NVCF) deploy/stacks/observ...

Platform for deploying, managing, and running GPU-accelerated inference, streaming, and batch workloads across worker clusters.

Go 230☆ 187d old #golang #kubernetes #go #gpu #cloud-functions

10/07

5

defilantech/LLMKube llmkube-0.10.2

Kubernetes operator managing self-hosted LLM inference on NVIDIA GPUs and Apple Silicon, with autoscaling, model routing, and OpenAI-compatible API.

Go 232☆ 333d old #golang #ai #kubernetes #go #gpu

09/28	8	Nehanth/pooled v1.0.0 Browser-based peer-to-peer LLM inference that pools model layers across devices in a room for fast chat and in-tab coding agents. JavaScript 568☆ 34d old #javascript #browser #llm #inference #local-llm
10/09	7	kibae/pg_onnx v1.31.0 ONNX Runtime integration enabling machine learning inference within PostgreSQL databases. C++ 63☆ 1121d old #c++ #ai #deep-learning #cuda #contributions-welcome
09/27	7	llm-d/llm-d Router v0.11.0 llm-d Router provides load and prefix-cache aware inference routing with prioritization and flow control, supporting standalone and Gateway API deployment via an endpoint picker. Go 373☆ 517d old #golang #ai #kubernetes #go #networking
09/18	7	kaito-project/AIKit v0.23.0 Platform to host, deploy, build, and fine-tune large language models with OpenAI API compatibility. Go 539☆ 1120d old #golang #docker #llm #go #large-language-models
10/05	6	RunanywhereAI/RunAnywhere cpp-desktop-v0.20.37... On-device mobile SDKs for running LLMs, speech-to-text, and text-to-speech locally. C++ 10314☆ 446d old #swift #c++ #llm #ios #kotlin
09/14	6	NVIDIA/NVIDIA Cloud Functions (NVCF) deploy/stacks/observ... Platform for deploying, managing, and running GPU-accelerated inference, streaming, and batch workloads across worker clusters. Go 230☆ 187d old #golang #kubernetes #go #gpu #cloud-functions
10/07	5	defilantech/LLMKube llmkube-0.10.2 Kubernetes operator managing self-hosted LLM inference on NVIDIA GPUs and Apple Silicon, with autoscaling, model routing, and OpenAI-compatible API. Go 232☆ 333d old #golang #ai #kubernetes #go #gpu