Open source · Apache 2.0 · 47+ providers

The LLM gateway that
cuts your API bill in half

A3M Router is an open-source LLM proxy that analyzes every query and routes it to the cheapest capable model — without you changing a line of code. 96.77% accuracy on RouterArena at $0.08 per 1K tokens.

npm install adaptive-memory-multi-model-router
$ pip install adaptive-memory-multi-model-router Enter
View Live Demo Star on GitHub
zsh — a3m-router
$ npx a3m-router serve
A3M Router v2.14 — listening on http://localhost:8787/v1
Providers: 47 active | Cache: enabled | Guardrails: on
────────────────────────────────────────────
$ curl http://localhost:8787/v1/chat/completions
-d '{"messages":[{"role":"user","content":"Write a Python quicksort"}]}'
────────────────────────────────────────────
Routing query to: groq/llama-3.3-70b-versatile
Confidence: 0.94 | Tier: budget | Est. cost: $0.00012
Cache hit: false | Latency: 847ms
────────────────────────────────────────────
200 OK — routed in 0.3ms — provider: groq
Score: 96.77% accuracy on RouterArena
Cost: $0.0768 / 1K tokens (avg) — savings vs GPT-4o: 94%
96.77%
RouterArena Accuracy
47+
LLM Providers
$0.08
Per 1K Tokens (avg)
63%
Cost Savings
Everything you need to route LLMs at scale
Built for production from day one. Intelligent routing, caching, security, and analytics — all in one drop-in gateway.
⚡
Core

Intelligent Query Routing

Analyzes query complexity using keyword-based classification and routes to the cheapest capable provider. Code queries go to code models. Trivial queries use free models. Premium only when needed.

💰
Core

Parallel Ensemble Mode

Call multiple providers simultaneously and score every response with weighted signals: domain match, specificity, structure alignment, verb matching, and cost tier. Cheapest that fully satisfies wins.

🔒
Caching

Semantic Cache

Trigram Jaccard similarity cache eliminates redundant API calls. Exact duplicates return instantly for free. Near-duplicates use fuzzy matching to maximize cache hit rate.

🛡
Security

Security Guardrails

Built-in prompt injection detection, PII redaction, content filtering, and rate limiting. Optional NeMo Guardrails integration for jailbreak detection and hallucination prevention.

📊
Analytics

Real-time Cost Analytics

Monitor spend, latency, cache hits, and provider health in real-time. Set per-provider budgets and get alerts when thresholds are exceeded.

🔄
Resilience

Circuit Breaker & Retry

Automatic failover when providers degrade or fail. Circuit breaker pattern keeps your app resilient. Exponential backoff with jitter for transient failures.

Drop-in replacement for api.openai.com
No code changes needed. Point your existing OpenAI SDK to localhost and A3M handles the rest.
Step 1 — Install

One command, anywhere

Works on macOS, Linux, and Windows. No Docker required. Runs locally or on a server.

npm install -g
  adaptive-memory-multi-model-router
# or
pip install
  adaptive-memory-multi-model-router
Step 2 — Start

One command to serve

A3M starts an OpenAI-compatible proxy on port 8787. Set your API key once and forget it.

export OPENROUTER_API_KEY=...
npx a3m-router serve

# Listens at
http://localhost:8787/v1
Step 3 — Use

Change one line of code

Point your OpenAI SDK base URL to A3M's proxy. Everything else works automatically.

openai.OpenAI({
  base_url: 'http://localhost:8787/v1'
})
# That's it. 47+ providers,
# intelligent routing, caching.
A3M vs OpenRouter vs LiteLLM
How does A3M stack up against the alternatives? Here's a feature-by-feature breakdown.
Feature A3M Router OpenRouter LiteLLM
✓ Open-source (Apache 2.0)
 ✓ ✗ Closed ✓
✓ Intelligent cost-based routing
 ✓ Keyword classifier ~ Latent routing ~ Configurable fallbacks
✓ Parallel ensemble execution
 ✓ Multi-provider scoring ✗ ✗
✓ Semantic cache (Jaccard trigram)
 ✓ Built-in ✗ ~ Redis external
✓ Security guardrails & PII redaction
 ✓ Built-in + NeMo ✗ ~ Virtual keys only
✓ Real-time cost analytics
 ✓ Built-in dashboard ~ Usage tracking only ~ LangSmith hook
✓ RouterArena benchmark
 ✓ 96.77% ✗ ✗
✓ Zero-config drop-in OpenAI proxy
 ✓ ✓ ✓
✓ Self-hosted / local models
 ✓ Ollama, LM Studio, vLLM ~ OpenRouter only ✓
✓ Circuit breaker & retry logic
 ✓ Built-in ✗ ✓
View on GitHub
47+ providers, one endpoint
From free local models to the latest GPT-4o and Gemini 2.0. A3M routes to the right one automatically.
Groq
$0.59 / 1M
Llama 3.3 70B
Cerebras
$0.60 / 1M
Llama 3.3 70B
DeepSeek
$1.50 / 1M
V3, Chat, Coder
Mistral
$2.00 / 1M
Large, Small
OpenAI
$15 / 1M
GPT-4o, o1, o3
Anthropic
$18 / 1M
Claude 3.5, 3 Opus
Google
$10 / 1M
Gemini 2.0, 1.5
xAI
$5 / 1M
Grok 2, Grok Beta
Ollama
Free
Local models
Command R
Free
CommandCode, R+
Together AI
$0.50 / 1M
DeepSeek V3, Qwen
MiniMax
$0.50 / 1M
MiniMax-Text-01
Fireworks Cohere Perplexity Qwen GLM-4 LM Studio vLLM OpenRouter + 35 more
Benchmark-proven accuracy
A3M scores 96.77% on RouterArena — the independent LLM routing benchmark.
RouterArena Score
0.9404 / 1.0
A3M Router96.77%
Avg. Baseline87.32%
Higher is better. Source: RouterArena official evaluation, 8,400 queries.
Cost per 1K Tokens
$0.08 avg
A3M (routed)$0.0768
GPT-4o only$15.00
94% cost reduction vs GPT-4o-only baseline. Routed on 8,400 RouterArena queries.
View RouterArena on GitHub
🎯

Try it right now — no install required

The live demo runs in your browser. Enter any prompt and watch A3M route it to the optimal provider in real time. See the confidence score, routing decision, and response.

Open Live Demo on HuggingFace

Free to use. No API key required for the demo.

Install in 30 seconds
Works anywhere Node.js or Python runs. Linux, macOS, Windows.
npm
# Install globally
npm install -g \
  adaptive-memory-multi-model-router

# Or use via npx (no install)
npx a3m-router serve

# Start routing on port 8787
pip
# Install from PyPI
pip install \
  adaptive-memory-multi-model-router

# Use the Python client
from a3m_router import A3MRouter
router = A3MRouter()
result = router.route("...")
Frequently Asked Questions
What is an LLM router? +
An LLM router is a gateway that intelligently directs each query to the optimal language model provider based on query characteristics, cost, availability, and capability requirements — instead of hardcoding a single provider. This enables cost optimization, automatic failover, and quality maximization on a per-query basis. A3M Router uses a keyword-based classifier to make routing decisions in under a millisecond.
How does A3M compare to OpenRouter? +
OpenRouter is a paid proxy service with a closed-source routing layer. A3M Router is fully open source (Apache 2.0) — you can self-host it, audit the routing logic, and contribute. A3M uses parallel multi-provider ensemble execution with confidence-weighted scoring, while OpenRouter uses sequential routing. A3M also includes built-in semantic caching, security guardrails, and real-time cost analytics at no extra cost.
How does parallel ensemble routing work? +
A3M sends the same query to multiple providers simultaneously (e.g., Groq, DeepSeek, and Mistral). Each response is scored using weighted signals: domain match to provider specialties, query specificity, structure alignment, verb matching, and cost tier. The cheapest provider with a score above the confidence threshold wins. This means a provider that responds first isn't guaranteed to win — quality and cost are factored in.
How much can I save? +
On the RouterArena benchmark (8,400 real queries), A3M achieves 96.77% accuracy at an average cost of $0.0768 per 1K tokens — compared to GPT-4o at $15/1M. That's a 94% cost reduction with better accuracy than the baseline. Combined with semantic caching (which eliminates repeat queries), real-world savings are even higher.
Can I use local models (Ollama, LM Studio)? +
Yes. A3M supports Ollama, LM Studio, and vLLM as free local providers. A3M will route simple or repetitive queries to your local models, reserving paid APIs for complex tasks. This gives you a free tier that costs nothing to run.
Does this work with LangChain, LlamaIndex, or CrewAI? +
Yes. Because A3M is an OpenAI-compatible proxy, any framework that supports custom base URLs works out of the box. This includes LangChain, LlamaIndex, AutoGen, and CrewAI. A3M also provides a first-class Python client with a cleaner API for those who prefer it over the OpenAI SDK.
How fast is the routing decision? +
Routing decisions take 0.1–0.5ms using A3M's keyword-based classifier — negligible compared to LLM inference time. The end-to-end latency is dominated by the selected provider's model speed. In parallel ensemble mode, A3M waits for the fastest responders, so the overall latency is often faster than waiting for a single premium model.

Start reducing your LLM costs today

Open source, self-hostable, and works with your existing code in minutes.

npm install -g adaptive-memory-multi-model-router
View on GitHub
NPM | PyPI | Live Demo