* [pooled v1.0.0](https://github.com/Nehanth/pooled) – Browser-based peer-to-peer LLM inference that pools model layers across devices in a room for fast chat and in-tab coding agents.

* [pg\_onnx v1.31.0](https://github.com/kibae/pg_onnx) – ONNX Runtime integration enabling machine learning inference within PostgreSQL databases.

* [llm-d Router v0.11.0](https://github.com/llm-d/llm-d-router) – llm-d Router provides load and prefix-cache aware inference routing with prioritization and flow control, supporting standalone and Gateway API deployment via an endpoint picker.

* [AIKit v0.23.0](https://github.com/kaito-project/aikit) – Platform to host, deploy, build, and fine-tune large language models with OpenAI API compatibility.

* [RunAnywhere cpp-desktop-v0.20.37...](https://github.com/RunanywhereAI/runanywhere-sdks) – On-device mobile SDKs for running LLMs, speech-to-text, and text-to-speech locally.

* [NVIDIA Cloud Functions (NVCF) deploy/stacks/observ...](https://github.com/NVIDIA/nvcf) – Platform for deploying, managing, and running GPU-accelerated inference, streaming, and batch workloads across worker clusters.

* [LLMKube llmkube-0.10.2](https://github.com/defilantech/LLMKube) – Kubernetes operator managing self-hosted LLM inference on NVIDIA GPUs and Apple Silicon, with autoscaling, model routing, and OpenAI-compatible API.