Best Deployment and Serving AI agents (2026)

Deployment and Serving

LiteRT-LM is a lightweight, cross-platform inference framework designed for efficiently running Large Language Mode…

Deployment and Serving

LiteRT is a lightweight runtime optimized for deploying machine learning models on resource-constrained devices lik…

Deployment and Serving

LMCache is a high-performance key-value (KV) cache layer designed to optimize and accelerate large language model (…

Deployment and Serving

LocalAI is a self-hosted, open-source alternative to the OpenAI API, providing a REST API for running AI models loc…

Deployment and Serving

MindsDB is an open-source platform that integrates machine learning models directly into databases, enabling users …

Deployment and Serving

mini-sglang is a lightweight framework designed for efficient deployment and serving of large language models (LLMs…

Deployment and Serving

MLRun is an open-source MLOps framework designed to streamline the deployment and management of machine learning wo…

Deployment and Serving

MLServer is an open-source inference server designed for deploying and serving machine learning models, supporting …

Deployment and Serving

Model Runner is a Docker-based tool for deploying and serving AI models, including LLMs, by containerizing them for…

Deployment and Serving

Mosec is a Rust-powered, multi-stage pipelined model server designed for efficient AI model deployment. It supports…

Deployment and Serving

nndeploy is an open-source framework designed to simplify the deployment of AI models by providing a high-performan…

Deployment and Serving

Nuclio is a high-performance serverless platform designed for deploying and managing data, I/O, and compute-intensi…

Deployment and Serving

OpenVINO (Open Visual Inference and Neural Network Optimization) is an open-source toolkit designed to optimize and…

Deployment and Serving

OptiLLM is an open-source, OpenAI API-compatible inference proxy that optimizes LLM performance using 20+ advanced …

Deployment and Serving

PowerInfer is a high-efficiency inference engine for large language models (LLMs) that optimizes CPU/GPU resource u…

Deployment and Serving

Prompt2Model is an open-source system that converts natural language task descriptions into small, specialized AI m…

Deployment and Serving

RamaLama is an open-source tool designed to streamline the deployment and serving of AI models locally using OCI co…

Deployment and Serving

RunAnywhere provides SDKs for deploying AI models (LLMs, speech-to-text, text-to-speech) on mobile devices, support…

Deployment and Serving

Seldon Core is an open-source platform designed to deploy, serve, and scale machine learning models in Kubernetes e…

Deployment and Serving

SIE is an open-source inference server and production cluster designed for deploying AI models focused on embedding…