<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Setting Up 9Router - Free AI Model Router</title>
<meta name="description" content="Complete guide to deploying 9Router - a free, OpenAI-compatible model router that automatically rotates through free models from multiple providers." />
<style>
:root {
--bg: #001a00;
--panel: rgba(2, 18, 42, 0.84);
--panel2: rgba(1, 12, 30, 0.96);
--blue: #22d3ee;
--blue-dark: #0ea5e9;
--blue-light: #67e8f9;
--purple: #818cf8;
--text: #e6f4ff;
--muted: #7eb3d4;
--border: rgba(34, 211, 238, 0.32);
--shadow: 0 0 42px rgba(34, 211, 238, 0.15);
--green: #39ff88;
--green-dark: #22c55e;
--orange: #f97316;
}
* { box-sizing: border-box }
html { scroll-behavior: smooth }
body {
margin: 0;
min-height: 100vh;
color: var(--text);
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, 'Liberation Mono', 'Courier New', monospace;
background:
radial-gradient(circle at 18% 12%, rgba(34, 211, 238, 0.18), transparent 28%),
radial-gradient(circle at 78% 4%, rgba(14, 165, 233, 0.12), transparent 28%),
radial-gradient(circle at 50% 88%, rgba(129, 140, 248, 0.12), transparent 28%),
var(--bg);
line-height: 1.6;
overflow-x: hidden;
}
body:before {
content: '';
position: fixed;
inset: 0;
pointer-events: none;
background-image:
linear-gradient(rgba(34, 211, 238, 0.045) 1px, transparent 1px),
linear-gradient(90deg, rgba(34, 211, 238, 0.035) 1px, transparent 1px);
background-size: 42px 42px;
mask-image: linear-gradient(to bottom, #000, transparent 86%);
z-index: 0;
}
.wrap {
position: relative;
z-index: 2;
width: min(1160px, calc(100% - 2rem));
margin: 0 auto;
padding: 2rem 1rem 4rem;
}
.matrix {
position: fixed;
inset: 0;
z-index: 1;
pointer-events: none;
opacity: 0.16;
color: var(--blue);
font-size: 0.82rem;
overflow: hidden;
}
.matrix span {
position: absolute;
top: -25%;
writing-mode: vertical-rl;
animation: rain linear infinite;
text-shadow: 0 0 10px var(--blue);
}
.matrix span:nth-child(1) { left: 4%; animation-duration: 16s; animation-delay: -3s }
.matrix span:nth-child(2) { left: 20%; animation-duration: 22s; animation-delay: -10s }
.matrix span:nth-child(3) { left: 42%; animation-duration: 18s; animation-delay: -5s }
.matrix span:nth-child(4) { left: 65%; animation-duration: 25s; animation-delay: -12s }
.matrix span:nth-child(5) { left: 88%; animation-duration: 19s; animation-delay: -8s }
@keyframes rain { from { transform: translateY(-30vh) } to { transform: translateY(135vh) } }
@keyframes blink { 50% { opacity: 0 } }
@keyframes glow { 50% { box-shadow: 0 0 54px rgba(34, 211, 238, 0.25) } }
@keyframes scan { 0% { transform: translateX(-100%) } 100% { transform: translateX(100%) } }
a { color: var(--blue); text-decoration: none }
a:hover { text-shadow: 0 0 12px var(--blue) }
h1, h2, h3 { color: #fff }
.muted { color: var(--muted) }
.wrap { max-width: 1160px; margin: 0 auto; padding: 2rem 1rem 4rem }
.card {
border: 1px solid var(--border);
background: var(--panel);
padding: 1.5rem;
border-radius: 22px;
margin-bottom: 1.5rem;
}
.tag {
display: inline-block;
background: var(--blue);
color: #001b0a;
padding: .2rem .6rem;
border-radius: 999px;
font-size: .75rem;
font-weight: 800;
margin-bottom: 1rem;
}
.step {
border-left: 3px solid var(--blue);
padding-left: 1rem;
margin: 1.5rem 0;
}
pre, code {
background: #010805;
color: #caffd9;
border-radius: 14px;
border: 1px solid rgba(34, 211, 238, 0.2);
overflow-x: auto;
}
pre { padding: 1rem }
code { padding: .15rem .4rem; font-size: .9em }
pre code { padding: 0; background: transparent; border: 0 }
ul { padding-left: 1.2rem }
li { margin: .5rem 0 }
.prompt { color: var(--green); font-weight: 600 }
.warning { color: var(--orange); font-weight: 600 }
.note { color: var(--blue); font-style: italic }
.placeholder { color: var(--orange); background: rgba(249, 115, 22, 0.1); padding: .1rem .3rem; border-radius: 4px; font-family: monospace }
.success { color: var(--green) }
.error { color: #ef4444 }
table { width: 100%; border-collapse: collapse; margin: 1rem 0 }
th, td { padding: .75rem; border: 1px solid var(--border); text-align: left }
th { background: rgba(34, 211, 238, 0.1); color: var(--blue) }
.toc { border: 1px solid var(--border); background: var(--panel2); padding: 1.5rem; border-radius: 22px; margin-bottom: 2rem }
.toc ul { list-style: none; padding: 0 }
.toc li { margin: .5rem 0 }
.toc a { color: var(--blue); font-weight: 500 }
.toc a:hover { text-shadow: 0 0 8px var(--blue) }
.toc li ul { margin-left: 1.5rem; border-left: 1px solid var(--border); padding-left: 1rem }
</style>
</head>
<body>
<div class="matrix" aria-hidden="true">
<span>sudo harden --system --logs --backup 🔐</span>
<span>9router ai proxy free models inference</span>
<span>encrypted monitor automated document</span>
<span>openai compatible api proxy</span>
<span>reddit r/LocalLlama r/MachineLearning</span>
</div>
<div class="wrap">
<p><a href="/">← return home</a></p>
<h1>🌐 Setting Up 9Router: Free AI Model Router for Local & Cloud Inference</h1>
<p class="muted">Complete guide to deploying 9Router — a free, OpenAI-compatible model router that automatically rotates through free models from multiple providers. Includes Docker setup, configuration, and integration examples.</ <p class="muted">Date: July 16, 2026 | Tags: 9Router, AI, Model Router, Free Models, Docker, OpenAI Compatible, Self-Hosted</p>
<div class="toc">
<h2>📋 Table of Contents</h2>
<ul>
<li><a href="#overview">📖 Overview</a></li>
<li><a href="#prerequisites">📋 Prerequisites</a></li>
<li><a href="#architecture">🏗️ Architecture Overview</a></li>
<li><a href="#docker-deployment">🐳 Docker Deployment</a></li>
<li><a href="#configuration">⚙️ Configuration</a></li>
<li><a href="#model-management">🤖 Model Management</a></li>
<li><a href="#integration-examples">🔌 Integration Examples</a></li>
<li><a href="#monitoring--health-checks">📊 Monitoring & Health Checks</a></li>
<li><a href="#troubleshooting">🔧 Troubleshooting</a></li>
<li><a href="#security-considerations">🔐 Security Considerations</a></li>
<li><a href="#conclusion">🎯 Conclusion</a></li>
</ul>
</div>
<div class="card" id="overview">
<span class="tag">📖 OVERVIEW</span>
<h2>What is 9Router?</h2>
<p><strong>9Router</strong> is a free, open-source AI model router that acts as an OpenAI-compatible API proxy. It automatically routes requests to free models across multiple providers, handling:</p>
<ul>
<li>🔄 <strong>Automatic model rotation</strong> - Falls back to next available model when one hits rate limits or errors</li>
<li>💰 <strong>Cost optimization</strong> - Prioritizes free tier models from multiple providers</li>
<li>🔌 <strong>OpenAI-compatible API</strong> - Drop-in replacement for OpenAI SDK</li>
<li>⚡ <strong>Load balancing</strong> - Distributes requests across healthy model endpoints</li>
<li>🔄 <strong>Automatic failover</strong> - Seamless failover when providers are down</li>
</ul>
<div class="card">
<h3>🎯 Supported Providers (Free Tiers)</h3>
<table>
<thead>
<tr><th>Provider</th><th>Models</th><th>Free Tier Limits</th></tr>
</thead>
<tbody>
<tr><td>🤗 Hugging Face</td><td>Mixtral, Llama, Zephyr, Phi, Gemma</td><td>30 req/min, 1M tokens/day</td></tr>
<tr><td>🧠 Together AI</td><td>Mixtral-8x7B, Llama-3, Qwen</td><td>Free tier available</td></tr>
<tr><td>⚡ Groq</td><td>Llama-3-70B, Mixtral, Gemma</td><td>Free tier: 30 req/min</td></tr>
<tr><td>🔥 Fireworks AI</td><td>Mixtral, Llama-3, Qwen</td><td>Free tier available</td></tr>
<tr><td>☁️ Cloudflare Workers AI</td><td>Llama-3, Mistral, CodeLlama</td><td>Free tier: 100K req/day</td></tr>
<tr><td>🧪 Replicate</td><td>Various open models</td><td>Free credits monthly</td></tr>
</tbody>
</table>
</div>
<div class="card">
<h3>🎯 Why Use 9Router?</h3>
<ul>
<li><strong>Zero cost</strong> - Leverages free tiers from multiple providers</li>
<li><strong>No vendor lock-in</strong> - Automatic fallback between providers</li>
<li><strong>OpenAI SDK compatible</strong> - Change base URL, keep your code</li>
<li><strong>Self-hosted option</strong> - Run on your own infrastructure</li>
<li><strong>Rate limit handling</strong> - Automatic rotation when limits hit</li>
</ul>
</div>
</div>
<div class="card" id="prerequisites">
<span class="tag">📋 PREREQUISITES</span>
<h2>System Requirements</h2>
<table>
<thead><tr><th>Component</th><th>Minimum</th><th>Recommended</th></tr></thead>
<tbody>
<tr><td>CPU</td><td>2 cores</td><td>4+ cores</td></tr>
<tr><td>RAM</td><td>2 GB</td><td>4+ GB</td></tr>
<tr><td>Disk</td><td>5 GB</td><td>10+ GB</td></tr>
<tr><td>OS</td><td>Linux (Ubuntu 22.04+)</td><td>Ubuntu 22.04+ / Debian 12</td></tr>
<tr><td>Docker</td><td>20.10+</td><td>24.0+</td></tr>
<tr><td>Docker Compose</td><td>2.0+</td><td>2.20+</td></tr>
</tbody>
</table>
<div class="step">
<h3>🔧 Install Docker & Docker Compose</h3>
<pre><code class="language-bash"># Update package index
sudo apt update && sudo apt upgrade -y
# Install Docker
curl -fsSL https://get.docker.com | sudo sh
# Add user to docker group
sudo usermod -aG docker $USER
newgrp docker
# Verify installation
docker --version
docker compose version</code></pre>
</div>
<div class="step">
<h3>🔐 Prepare API Keys (Placeholders)</h3>
<p class="note">Replace all <span class="placeholder"><API_KEY></span> placeholders with your actual keys.</ <pre><code class="language-bash"># Create environment file
mkdir -p /home/ryan/9router
cat > /home/ryan/9router/.env << 'EOF'
# 9Router Configuration
NINEROUTER_PORT=20128
NINEROUTER_HOST=0.0.0.0
# Provider API Keys (replace with your keys)
HUGGINGFACE_API_KEY=<HUGGINGFACE_API_KEY>
TOGETHER_API_KEY=<TOGETHER_API_KEY>
GROQ_API_KEY=<GROQ_API_KEY>
FIREWORKS_API_KEY=<FIREWORKS_API_KEY>
CLOUDFLARE_API_KEY=<CLOUDFLARE_API_KEY>
CLOUDFLARE_ACCOUNT_ID=<CLOUDFLARE_ACCOUNT_ID>
REPLICATE_API_KEY=<REPLICATE_API_KEY>
# Optional: OpenAI/Anthropic for fallback
OPENAI_API_KEY=<OPENAI_API_KEY>
ANTHROPIC_API_KEY=<ANTHROPIC_API_KEY>
# 9Router Settings
NINEROUTER_LOG_LEVEL=info
NINEROUTER_MAX_RETRIES=3
NINEROUTER_TIMEOUT=30000
EOF</code></pre>
</div>
</div>
<div class="card" id="architecture">
<span class="tag">🏗️ ARCHITECTURE</span>
<h2>Architecture Overview</h2>
<pre>
┌─────────────────────────────────────────────────────────────────┐
│ CLIENT REQUEST │
│ (OpenAI SDK / curl / any HTTP client) │
└──────────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ 9ROUTER (Port 20128) │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Request Router & Load Balancer │ │
│ │ • Model selection logic │ │
│ │ • Rate limit tracking │ │
│ │ • Health checks │ │
│ │ • Request/response transformation │ │
│ └─────────────────────────────────────────────────────────┘ │
└──────────────────────────┬────────────────────────────────────────┘
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Hugging Face │ │ Groq │ │ Together AI │
│ (Free Tier) │ │ (Free Tier) │ │ (Free Tier) │
│ Mixtral, Llama │ │ Llama-3-70B │ │ Mixtral, Llama │
└─────────────────┘ └─────────────────┘ └─────────────────┘
┌─────────────────┐ ┌─────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Fireworks AI │ │ Cloudflare │ │ Replicate │
│ (Free Tier) │ │ Workers AI │ │ (Free Credits) │
│ Mixtral, Llama │ │ Llama-3 │ │ Various Models │
└─────────────────┘ └─────────────────┘ └─────────────────┘
</pre>
<div class="card">
<h3>Request Flow</h3>
<ol>
<li>Client sends request to <code>http://localhost:20128/v1/chat/completions</code></li>
<li>9Router parses request, selects optimal model based on availability</li>
<li>Request transformed to provider-specific format</li>
<li>Request sent to provider API</li>
<li>Response transformed back to OpenAI format</li>
<li>Response returned to client</li>
</ol>
</div>
</div>
<div class="card" id="docker-deployment">
<span class="tag">🐳 DOCKER DEPLOYMENT</span>
<h2>Docker Deployment</h2>
<div class="step">
<h3>📁 Create Directory Structure</h3>
<pre><code class="language-bash"># Create project directories
mkdir -p /home/ryan/9router/{data,logs,config}
# Verify structure
tree /home/ryan/9router
# Expected output:
# /home/ryan/9router
# ├── .env
# ├── docker-compose.yml
# ├── data/
# ├── logs/
# └── config/</code></pre>
</div>
<div class="step">
<h3>📝 Create docker-compose.yml</h3>
<pre><code class="language-yaml"># /home/ryan/9router/docker-compose.yml
version: '3.8'
services:
ninerouter:
image: ghcr.io/9router/9router:latest
container_name: ninerouter
restart: unless-stopped
ports:
- "20128:20128"
environment:
- NINEROUTER_PORT=20128
- NINEROUTER_HOST=0.0.0.0
- NINEROUTER_LOG_LEVEL=info
- NINEROUTER_MAX_RETRIES=3
- NINEROUTER_TIMEOUT=30000
# Provider API Keys
- HUGGINGFACE_API_KEY=${HUGGINGFACE_API_KEY}
- TOGETHER_API_KEY=${TOGETHER_API_KEY}
- GROQ_API_KEY=${GROQ_API_KEY}
- FIREWORKS_API_KEY=${FIREWORKS_API_KEY}
- CLOUDFLARE_API_KEY=${CLOUDFLARE_API_KEY}
- CLOUDFLARE_ACCOUNT_ID=${CLOUDFLARE_ACCOUNT_ID}
- REPLICATE_API_KEY=${REPLICATE_API_KEY}
# Optional fallbacks
- OPENAI_API_KEY=${OPENAI_API_KEY}
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
volumes:
- ./data:/app/data
- ./logs:/app/logs
- ./config:/app/config
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:20128/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
default:
name: ninerouter-network</code></pre>
</div>
<div class="step">
<h3>🚀 Start 9Router</h3>
<pre><code class="language-bash">cd /home/ryan/9router
# Load environment variables
export $(cat .env | xargs)
# Pull latest image
docker compose pull
# Start in detached mode
docker compose up -d
# View logs
docker compose logs -f ninerouter
# Check status
docker compose ps</code></pre>
</div>
<div class="step">
<h3>✅ Verify Deployment</h3>
<pre><code class="language-bash"># Health check
curl -s http://localhost:20128/health | jq .
# List available models
curl -s http://localhost:20128/v1/models | jq .
# Test chat completion
curl -s -X POST http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "free",
"messages": [{"role": "user", "content": "Hello, 9Router!"}],
"max_tokens": 100
}' | jq .</code></pre>
</div>
<div class="card">
<h3>🔄 Auto-Start on Boot (systemd)</h3>
<pre><code class="language-bash"># Create systemd service
sudo tee /etc/systemd/system/ninerouter.service > /dev/null << 'EOF'
[Unit]
Description=9Router AI Model Router
Requires=docker.service
After=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/home/ryan/9router
EnvironmentFile=/home/ryan/9router/.env
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
TimeoutStartSec=120
[Install]
WantedBy=multi-user.target
EOF
# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable ninerouter
sudo systemctl start ninerouter
# Check status
sudo systemctl status ninerouter</code></pre>
</div>
</div>
<div class="card" id="configuration">
<span class="tag">⚙️ CONFIGURATION</span>
<h2>Configuration Deep Dive</h2>
<div class="step">
<h3>🔧 Environment Variables Reference</h3>
<table>
<thead>
<tr><th>Variable</th><th>Description</th><th>Default</th><th>Required</th></tr>
</thead>
<tbody>
<tr><td><code>NINEROUTER_PORT</code></td><td>HTTP port to listen on</td><td>20128</td><td>No</td></tr>
<tr><td><code>NINEROUTER_HOST</code></td><td>Bind address</td><td>0.0.0.0</td><td>No</td></tr>
<tr><td><code>NINEROUTER_LOG_LEVEL</code></td><td>Log level (debug/info/warn/error)</td><td>info</td><td>No</td></tr>
<tr><td><code>NINEROUTER_MAX_RETRIES</code></td><td>Max retry attempts per request</td><td>3</td><td>No</td></tr>
<tr><td><code>NINEROUTER_TIMEOUT</code></td><td>Request timeout in ms</td><td>30000</td><td>No</td></tr>
<tr><td><code>HUGGINGFACE_API_KEY</code></td><td>Hugging Face Inference API key</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>GROQ_API_KEY</code></td><td>Groq Cloud API key</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>TOGETHER_API_KEY</code></td><td>Together AI API key</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>FIREWORKS_API_KEY</code></td><td>Fireworks AI API key</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>CLOUDFLARE_API_KEY</code></td><td>Cloudflare API token</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>CLOUDFLARE_ACCOUNT_ID</code></td><td>Cloudflare account identifier</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>REPLICATE_API_KEY</code></td><td>Replicate API token</td><td>-</td><td>Yes*</td></tr>
<tr><td><code>OPENAI_API_KEY</code></td><td>OpenAI API key (fallback)</td><td>-</td><td>No</td></tr>
<tr><td><code>ANTHROPIC_API_KEY</code></td><td>Anthropic API key (fallback)</td><td>-</td><td>No</td></tr>
</tbody>
</table>
<p class="note">* At least one provider key is required for 9Router to function.</ </div>
<div class="step">
<h3>🎛️ Model Selection Strategy</h3>
<p>9Router supports several model selection modes:</p>
<table>
<thead><tr><th>Mode</th><th>Description</th><th>Use Case</th></tr></thead>
<tbody>
<tr><td><code>free</code></td><td>Auto-select from free tier models</td><td>Cost optimization</td></tr>
<tr><td><code>fastest</code></td><td>Lowest latency model</td><td>Real-time apps</td></tr>
<tr><td><code>smart</code></td><td>Balance of speed/quality/cost</td><td>General purpose</td></tr>
<tr><td><code>specific</code></td><td>Use exact model name</td><td>Specific requirements</td></tr>
</tbody>
</table>
<h4>Usage Examples:</h4>
<pre><code class="language-json">// Auto-select free model (recommended for cost)
{ "model": "free", "messages": [...] }
// Fastest available model
{ "model": "fastest", "messages": [...] }
// Specific model
{ "model": "groq/llama-3-70b", "messages": [...] }
// Smart balancing (default)
{ "model": "smart", "messages": [...] }</code></pre>
</div>
<div class="step">
<h3>🔄 Rate Limit Configuration</h3>
<pre><code class="language-yaml"># Advanced rate limiting in config.yaml (optional)
rate_limits:
global:
requests_per_minute: 100
tokens_per_minute: 50000
per_provider:
huggingface:
requests_per_minute: 30
tokens_per_minute: 10000
groq:
requests_per_minute: 30
tokens_per_minute: 15000
together:
requests_per_minute: 20
tokens_per_minute: 8000
retry_policy:
max_attempts: 3
backoff_base_ms: 1000
backoff_max_ms: 10000
retry_on:
- rate_limit
- timeout
- server_error
dont_retry_on:
- invalid_request
- authentication_error
- context_length_exceeded</code></pre>
</div>
</div>
<div class="card" id="model-management">
<span class="tag">🤖 MODEL MANAGEMENT</span>
<h2>Model Management</h2>
<div class="step">
<h3>📋 List Available Models</h3>
<pre><code class="language-bash"># All models
curl -s http://localhost:20128/v1/models | jq '.data[] | {id, owned_by, pricing}'
# Free models only
curl -s http://localhost:20128/v1/models | jq '.data[] | select(.pricing.prompt == 0) | {id, owned_by}'
# By provider
curl -s http://localhost:20128/v1/models | jq '.data[] | select(.owned_by == "groq") | {id}'
# With capabilities
curl -s http://localhost:20128/v1/models | jq '.data[] | {id, context_window, max_output, supports_tools}'</code></pre>
</div>
<div class="step">
<h3>🎯 Model Aliases</h3>
<p>9Router supports model aliases for easier reference:</p>
<pre><code class="language-bash"># View aliases
curl -s http://localhost:20128/v1/models | jq '.data[] | select(.id | test("alias")) | {id, alias_for}'
# Common aliases:
# "free" → Auto-select free model
# "fastest" → Lowest latency
# "smart" → Balanced (default)
# "code" → Best for coding
# "reasoning" → Best for reasoning
# "long-context" → Largest context window</code></pre>
</div>
<div class="step">
<h3>🔍 Model Health Checks</h3>
<pre><code class="language-bash"># Check model health
curl -s http://localhost:20128/v1/models/health | jq .
# Force refresh model list
curl -X POST http://localhost:20128/v1/models/refresh
# Check specific model
curl -s http://localhost:20128/v1/models/groq/llama-3-70b | jq .</code></pre>
</div>
</div>
<div class="card" id="integration-examples">
<span class="tag">🔌 INTEGRATION EXAMPLES</span>
<h2>Integration Examples</h2>
<div class="step">
<h3>🐍 Python (OpenAI SDK)</h3>
<pre><code class="language-python"># pip install openai
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:20128/v1",
api_key="not-needed" # 9Router doesn't require API key locally
)
# Free model auto-selection
response = client.chat.completions.create(
model="free",
messages=[
{"role": "system", "content": "You are a helpful security engineer."},
{"role": "user", "content": "Explain CVE-2024-3094 in 3 sentences."}
],
max_tokens=200,
temperature=0.3
)
print(response.choices[0].message.content)
# Streaming
stream = client.chat.completions.create(
model="free",
messages=[{"role": "user", "content": "Write a Python port scanner"}],
stream=True,
max_tokens=500
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)</code></pre>
</div>
<div class="step">
<h3>📡 JavaScript/TypeScript (OpenAI SDK)</h3>
<pre><code class="language-typescript">// npm install openai
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "http://localhost:20128/v1",
apiKey: "not-needed", // 9Router doesn't require auth locally
});
// Free model
const completion = await client.chat.completions.create({
model: "free",
messages: [
{ role: "system", content: "You are a cybersecurity expert." },
{ role: "user", content: "Explain XSS vs CSRF" }
],
max_tokens: 300,
temperature: 0.2,
});
console.log(completion.choices[0].message.content);
// With specific provider
const groqCompletion = await client.chat.completions.create({
model: "groq/llama-3-70b",
messages: [{ role: "user", content: "Quick regex for email validation" }],
max_tokens: 100,
});</code></pre>
</div>
<div class="step">
<h3>📡 cURL Examples</h3>
<pre><code class="language-bash"># Basic chat completion (free model)
curl -X POST http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "free",
"messages": [
{"role": "system", "content": "You are a Linux expert."},
{"role": "user", "content": "How to recursively find all .log files?"}
],
"max_tokens": 200,
"temperature": 0.2
}'
# Streaming response
curl -X POST http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "free",
"messages": [{"role": "user", "content": "Write a haiku about Docker"}],
"stream": true,
"max_tokens": 100
}' --no-buffer
# With specific model
curl -X POST http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "groq/llama-3-70b",
"messages": [{"role": "user", "content": "Explain eBPF in one paragraph"}],
"max_tokens": 150
}'
# Embeddings (if supported)
curl -X POST http://localhost:20128/v1/embeddings \
-H "Content-Type: application/json" \
-d '{
"model": "free",
"input": "Your text to embed here"
}'</code></pre>
</div>
<div class="step">
<h3>🦀 Rust (async-openai)</h3>
<pre><code class="language-rust">// Cargo.toml
// [dependencies]
// async-openai = { version = "0.20", features = ["chat"] }
use async_openai::{Client, config::OpenAIConfig};
use async_openai::types::{
ChatCompletionRequest, ChatCompletionMessage, Role
};
let config = OpenAIConfig::new()
.with_api_base("http://localhost:20128/v1")
.with_api_key("not-needed");
let client = Client::with_config(config);
let request = ChatCompletionRequest {
model: "free".to_string(),
messages: vec![
ChatCompletionMessage {
role: Role::System,
content: "You are a Rust expert.".into(),
..Default::default()
},
ChatCompletionMessage {
role: Role::User,
content: "How to handle Result<T, E> in Rust?".into(),
..Default::default()
},
],
max_tokens: Some(200),
temperature: Some(0.3),
..Default::default()
};
let response = client.chat().create(request).await?;
println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));</code></pre>
</div>
<div class="step">
<h3>🤖 LangChain Integration</h3>
<pre><code class="language-python"># pip install langchain-openai
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
llm = ChatOpenAI(
base_url="http://localhost:20128/v1",
api_key="not-needed",
model="free",
temperature=0.3,
max_tokens=500,
)
response = llm.invoke([
SystemMessage(content="You are a DevOps engineer."),
HumanMessage(content="Write a Dockerfile for a Python FastAPI app")
])
print(response.content)
# With streaming
for chunk in llm.stream([
HumanMessage(content="Write a bash script to backup PostgreSQL")
]):
print(chunk.content, end="", flush=True)</code></pre>
</div>
<div class="step">
<h3>🌐 HTTP Clients (Any Language)</h3>
<pre><code class="language-bash"># Using httpie
http POST http://localhost:20128/v1/chat/completions \
model=free \
messages:='[{"role": "user", "content": "Hello 9Router!"}]' \
max_tokens:=100
# Using wget
wget -qO- --post-data='{"model":"free","messages":[{"role":"user","content":"Hi"}]}' \
--header="Content-Type: application/json" \
http://localhost:20128/v1/chat/completions</code></pre>
</div>
</div>
<div class="card" id="monitoring--health-checks">
<span class="tag">📊 MONITORING</span>
<h2>Monitoring & Health Checks</h2>
<div class="step">
<h3>🏥 Health Endpoints</h3>
<pre><code class="language-bash"># Basic health
curl -s http://localhost:20128/health | jq .
# Detailed health
curl -s http://localhost:20128/health/detailed | jq .
# Provider health
curl -s http://localhost:20128/v1/providers/health | jq .
# Readiness
curl -s http://localhost:20128/ready | jq .</code></pre>
</div>
<div class="step">
<h3>📊 Metrics & Logging</h3>
<pre><code class="language-bash"># View logs
docker compose logs -f ninerouter --tail 100
# Filter by level
docker compose logs ninerouter | grep -i error
# JSON logs for parsing
docker compose logs ninerouter --format json | jq .
# Resource usage
docker stats ninerouter --no-stream</code></pre>
</div>
<div class="step">
<h3>📈 Prometheus Metrics (if enabled)</h3>
<pre><code class="language-bash"># Enable in docker-compose.yml environment:
# - NINEROUTER_METRICS_ENABLED=true
# - NINEROUTER_METRICS_PORT=9090
# Scrape metrics
curl -s http://localhost:9090/metrics | grep ninerouter
# Key metrics to alert on:
# ninerouter_requests_total
# ninerouter_request_duration_seconds
# ninerouter_provider_errors_total
# ninerouter_rate_limit_hits_total
# ninerouter_active_connections</code></pre>
</div>
<div class="step">
<h3>🚨 Alerting Rules (Prometheus/Grafana)</h3>
<pre><code class="language-yaml">groups:
- name: ninerouter-alerts
rules:
- alert: NinerouterDown
expr: up{job="ninerouter"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "9Router is down"
- alert: NinerouterHighErrorRate
expr: rate(ninerouter_provider_errors_total[5m]) > 0.1
for: 2m
labels:
severity: warning
annotations:
summary: "High error rate on 9Router providers"
- alert: NinerouterRateLimited
expr: rate(ninerouter_rate_limit_hits_total[5m]) > 0.05
for: 5m
labels:
severity: warning
annotations:
summary: "9Router hitting rate limits frequently"
- alert: NinerouterHighLatency
expr: histogram_quantile(0.95, rate(ninerouter_request_duration_seconds_bucket[5m])) > 10
for: 5m
labels:
severity: warning
annotations:
summary: "9Router p95 latency > 10s"</code></pre>
</div>
</div>
<div class="card" id="troubleshooting">
<span class="tag">🔧 TROUBLESHOOTING</span>
<h2>Troubleshooting Common Issues</h2>
<div class="step">
<h3>🔴 Container Won't Start</h3>
<pre><code class="language-bash"># Check logs
docker compose logs ninerouter
# Common issues:
# 1. Port already in use
sudo lsof -i :20128
# Kill process or change port in .env
# 2. Missing API keys
docker compose logs ninerouter | grep -i "api key"
# 3. Permission issues
sudo chown -R $USER:$USER /home/ryan/9router/data /home/ryan/9router/logs
# 4. Docker image issues
docker compose pull
docker compose up -d --force-recreate</code></pre>
</div>
<div class="step">
<h3>🟡 High Latency / Timeouts</h3>
<pre><code class="language-bash"># Check provider health
curl -s http://localhost:20128/v1/providers/health | jq .
# Increase timeout in .env
NINEROUTER_TIMEOUT=60000
# Check which provider is slow
curl -s http://localhost:20128/v1/providers/latency | jq .
# Force model refresh
curl -X POST http://localhost:20128/v1/models/refresh</code></pre>
</div>
<div class="step">
<h3>🟠 Rate Limit Errors</h3>
<pre><code class="language-bash"># Check rate limit status
curl -s http://localhost:20128/v1/rate-limits | jq .
# Add more provider keys to .env
# 9Router auto-rotates when rate limited
# Adjust retry policy
NINEROUTER_MAX_RETRIES=5
NINEROUTER_TIMEOUT=60000</code></pre>
</div>
<div class="step">
<h3>🔵 Model Not Found / Wrong Model</h3>
<pre><code class="language-bash"># List all available models
curl -s http://localhost:20128/v1/models | jq '.data[].id'
# Check model aliases
curl -s http://localhost:20128/v1/models | jq '.data[] | select(.id | contains("alias"))'
# Force model list refresh
curl -X POST http://localhost:20128/v1/models/refresh
# Use exact model ID from list
# e.g., "groq/llama-3-70b" not "llama-3-70b"</code></pre>
</div>
<div class="step">
<h3>🟣 Authentication Errors</h3>
<pre><code class="language-bash"># Verify API keys in .env
cat /home/ryan/9router/.env | grep -E "^(HUGGINGFACE|GROQ|TOGETHER|FIREWORKS|CLOUDFLARE|REPLICATE)_API_KEY"
# Test individual provider
curl -s -H "Authorization: Bearer <GROQ_API_KEY>" \
https://api.groq.com/openai/v1/models | jq .
# Check 9Router logs for auth errors
docker compose logs ninerouter | grep -i "auth\|unauthorized\|401\|403"</code></pre>
</div>
</div>
<div class="card" id="security-considerations">
<span class="tag">🔐 SECURITY</span>
<h2>Security Considerations</h2>
<div class="step">
<h3>🔒 Network Security</h3>
<ul>
<li><strong>Bind to localhost only</strong> - Use <code>127.0.0.1:20128</code> for local-only access</li>
<li><strong>Firewall rules</strong> - Block external access to port 20128</li>
<li><strong>Reverse proxy</strong> - Use nginx/Caddy with auth for remote access</li>
</ul>
<pre><code class="language-bash"># UFW firewall rule (local only)
sudo ufw deny 20128
# Or allow only specific IPs
sudo ufw allow from 192.168.1.0/24 to any port 20128</code></pre>
</div>
<div class="step">
<h3>🔐 API Key Management</h3>
<ul>
<li><strong>Never commit .env to git</strong> - Add to .gitignore</li>
<li><strong>Use secrets manager</strong> - HashiCorp Vault, AWS Secrets Manager, etc.</li>
<li><strong>Rotate keys regularly</strong> - Quarterly rotation recommended</li>
<li><strong>Minimal scopes</strong> - Use read-only keys where possible</li>
</ul>
<pre><code class="language-bash"># .gitignore entry
echo ".env" >> .gitignore
echo "*.env" >> .gitignore
echo "*.key" >> .gitignore</code></pre>
</div>
<div class="step">
<h3>🛡️ Reverse Proxy with Authentication (Caddy)</h3>
<pre><code class="language-dockerfile"># Caddyfile
ninerouter.yourdomain.com {
basicauth {
ryan <bcrypt-hashed-password>
}
reverse_proxy localhost:20128 {
header_up Host {host}
header_up X-Real-IP {remote}
header_up X-Forwarded-For {remote}
header_up X-Forwarded-Proto {scheme}
}
}</code></pre>
</div>
<div class="step">
<h3>📊 Audit Logging</h3>
<pre><code class="language-yaml"># docker-compose.yml addition
services:
ninerouter:
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "5"
labels: "service=ninerouter"
# Optional: Forward to Loki/ELK
# logging:
# driver: loki
# options:
# loki-url: "http://loki:3100/loki/api/v1/push"
# loki-retries: "5"
# loki-batch-size: "400"</code></pre>
</div>
</div>
<div class="card" id="conclusion">
<span class="tag">🎯 CONCLUSION</span>
<h2>Wrapping Up</h2>
<div class="card">
<h3>✅ What We Built</h3>
<ul>
<li>✅ Self-hosted 9Router instance on port 20128</li>
<li>✅ OpenAI-compatible API endpoint</li>
<li>✅ Free model access from 6+ providers</li>
<li>✅ Automatic failover & load balancing</li>
<li>✅ Health checks & monitoring ready</li>
<li>✅ systemd service for auto-start</li>
<li>✅ Integration examples for Python, JS, Rust, Go, cURL</li>
</ul>
</div>
<div class="card">
<h3>🚀 Next Steps</h3>
<ol>
<li>Add more provider API keys to <code>.env</code></li>
<li>Configure reverse proxy (Caddy/nginx) for remote access</li>
<li>Set up Prometheus/Grafana monitoring</li>
<li>Add custom model aliases in config</li>
<li>Implement request/response logging for audit</li>
<li>Experiment with model routing strategies</li>
</ol>
</div>
<div class="card">
<h3>📚 Resources</h3>
<ul>
<li><a href="https://github.com/9router/9router" target="_blank">9Router GitHub Repository</a></li>
<li><a href="https://github.com/9router/9router/blob/main/docs" target="_blank">9Router Documentation</a></li>
<li><a href="https://openai.com/api" target="_blank">OpenAI API Reference</a></li>
<li><a href="https://docs.docker.com/compose/" target="_blank">Docker Compose Reference</a></li>
</ul>
</div>
<div class="card success">
<h3>🎉 You're Ready!</h3>
<p>Your 9Router instance is now running at <code>http://localhost:20128/v1</code> — a free, OpenAI-compatible AI model router with automatic failover across multiple free-tier providers.</ <p class="note">Remember to replace all <span class="placeholder"><API_KEY></span> placeholders with your actual provider keys in <code>/home/ryan/9router/.env</code> before production use.</ </div>
</div>
</div>
</body>
</html>