# Setting Up 9Router: A Free OpenAI-Compatible Model Router for Local & Cloud AI
**Date:** July 16, 2026
**Tags:** AI, 9Router, Model Router, OpenAI Compatible, Docker, Self-Hosted, Free Models, AI Infrastructure
---
## 🎯 Overview
[9Router](https://github.com/9router/9router) is a free, open-source, OpenAI-compatible model router that automatically routes requests to free models across multiple providers. Think of it as a **load balancer for LLMs** that gives you a single OpenAI-compatible endpoint while automatically handling:
- **Automatic failover** when a provider hits rate limits or goes down
- **Load balancing** across multiple free model providers
- **Model routing** based on task type (coding, chat, reasoning, etc.)
- **Unified API** - drop-in replacement for OpenAI SDK
- **Zero cost** - leverages free tiers from multiple providers
### Supported Providers (Free Tiers)
| Provider | Models | Free Tier Limits |
|----------|--------|------------------|
| **Groq** | Llama-3.1-70B, Llama-3.1-8B, Mixtral-8x7B, Gemma2-9B | 30 RPM, 6K TPM |
| **Cerebras** | Llama-3.1-70B, Llama-3.1-8B | Unlimited (rate limited) |
| **Together AI** | Llama-3.1-405B, Llama-3.1-70B, Mixtral | $1 free credit |
| **Fireworks** | Llama-3.1-70B, Llama-3.1-8B, Mixtral | $1 free credit |
| **DeepInfra** | Llama-3.1-405B, 70B, 8B, Mixtral | Pay per token (very cheap) |
| **Novita AI** | Llama-3.1, Qwen2.5, Yi | Pay per token |
| **OpenRouter** | 100+ models | Pay per token |
---
## 🏗️ Architecture Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ Your Application │
│ (OpenAI SDK / curl / any HTTP client) │
└─────────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ 9Router (Port 4000) │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ Request Router & Load Balancer │ │
│ │ • Model selection (task-aware routing) │ │
│ │ • Health checks & circuit breakers │ │
│ │ • Rate limit tracking per provider │ │
│ │ • Automatic failover & retry logic │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────┬───────────────────────────────────┘
│
┌─────────────────────┼─────────────────┐
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Groq │ │ Cerebras │ │ Together AI │
│ (Llama 3.1) │ │ (Llama 3.1) │ │ (Llama 3.1) │
└───────────────┘ └───────────────┘ └───────────────┘
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Fireworks │ │ DeepInfra │ │ OpenRouter │
│ (Llama 3.1) │ │ (Llama 3.1) │ │ (100+ models)│
└───────────────┘ └───────────────┘ └───────────────┘
```
---
## 🐳 Quick Start: Docker Deployment
### Prerequisites
- Docker & Docker Compose installed
- API keys for at least 2 providers (recommended for redundancy)
- Linux/macOS/WSL2 environment
### 1. Create Project Directory
```bash
mkdir -p ~/9router/{config,data,logs}
cd ~/9router
```
### 2. Create Environment File
```bash
cat > .env << 'EOF'
# 9Router Configuration
PORT=4000
HOST=0.0.0.0
LOG_LEVEL=info
# Provider API Keys (REQUIRED - at least 2 for failover)
# Get keys from each provider's dashboard
GROQ_API_KEY=<GROQ_API_KEY>
CEREBRAS_API_KEY=<CEREBRAS_API_KEY>
TOGETHER_API_KEY=<TOGETHER_API_KEY>
FIREWORKS_API_KEY=<FIREWORKS_API_KEY>
DEEPINFRA_API_KEY=<DEEPINFRA_API_KEY>
NOVITA_API_KEY=<NOVITA_API_KEY>
OPENROUTER_API_KEY=<OPENROUTER_API_KEY>
# Optional: Custom model routing rules
# ROUTING_CONFIG=/app/config/routing.yaml
# Optional: Metrics & monitoring
PROMETHEUS_ENABLED=true
METRICS_PORT=9090
EOF
```
### 3. Create Docker Compose File
```bash
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
ninerouter:
image: ghcr.io/9router/9router:latest
container_name: ninerouter
restart: unless-stopped
ports:
- "4000:4000"
env_file: .env
volumes:
- ./config:/app/config:ro
- ./data:/app/data
- ./logs:/app/logs
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:4000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
deploy:
resources:
limits:
memory: 512M
reservations:
memory: 128M
networks:
default:
name: ninerouter-network
EOF
```
### 4. Create Configuration File
```bash
cat > config/config.yaml << 'EOF'
# 9Router Configuration
# See: https://github.com/9router/9router/blob/main/docs/config.md
server:
host: "0.0.0.0"
port: 4000
read_timeout: 30s
write_timeout: 300s
idle_timeout: 60s
# Provider Configuration
providers:
groq:
enabled: true
api_key_env: "GROQ_API_KEY"
base_url: "https://api.groq.com/openai/v1"
models:
- "llama-3.1-70b-versatile"
- "llama-3.1-8b-instant"
- "mixtral-8x7b-32768"
- "gemma2-9b-it"
priority: 1
rate_limit:
requests_per_minute: 30
tokens_per_minute: 6000
timeout: 30s
max_retries: 3
cerebras:
enabled: true
api_key_env: "CEREBRAS_API_KEY"
base_url: "https://api.cerebras.ai/v1"
models:
- "llama3.1-70b"
- "llama3.1-8b"
priority: 2
rate_limit:
requests_per_minute: 60
tokens_per_minute: 12000
timeout: 30s
max_retries: 3
together:
enabled: true
api_key_env: "TOGETHER_API_KEY"
base_url: "https://api.together.xyz/v1"
models:
- "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo"
- "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo"
- "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"
- "mistralai/Mixtral-8x7B-Instruct-v0.1"
priority: 3
rate_limit:
requests_per_minute: 60
tokens_per_minute: 20000
timeout: 60s
max_retries: 3
fireworks:
enabled: true
api_key_env: "FIREWORKS_API_KEY"
base_url: "https://api.fireworks.ai/inference/v1"
models:
- "accounts/fireworks/models/llama-v3p1-70b-instruct"
- "accounts/fireworks/models/llama-v3p1-8b-instruct"
- "accounts/fireworks/models/mixtral-8x7b-instruct"
priority: 4
rate_limit:
requests_per_minute: 60
tokens_per_minute: 20000
timeout: 60s
max_retries: 3
deepinfra:
enabled: true
api_key_env: "DEEPINFRA_API_KEY"
base_url: "https://api.deepinfra.com/v1/openai"
models:
- "meta-llama/Meta-Llama-3.1-405B-Instruct"
- "meta-llama/Meta-Llama-3.1-70B-Instruct"
- "meta-llama/Meta-Llama-3.1-8B-Instruct"
- "mistralai/Mixtral-8x7B-Instruct-v0.1"
priority: 5
rate_limit:
requests_per_minute: 100
tokens_per_minute: 50000
timeout: 60s
max_retries: 3
novita:
enabled: true
api_key_env: "NOVITA_API_KEY"
base_url: "https://api.novita.ai/v3/openai"
models:
- "meta-llama/llama-3.1-405b-instruct"
- "meta-llama/llama-3.1-70b-instruct"
- "qwen/qwen2.5-72b-instruct"
- "01-ai/yi-1.5-34b-chat"
priority: 6
rate_limit:
requests_per_minute: 100
tokens_per_minute: 50000
timeout: 60s
max_retries: 3
openrouter:
enabled: true
api_key_env: "OPENROUTER_API_KEY"
base_url: "https://openrouter.ai/api/v1"
models:
- "meta-llama/llama-3.1-405b-instruct"
- "meta-llama/llama-3.1-70b-instruct"
- "mistralai/mixtral-8x7b-instruct"
- "google/gemma-2-27b-it"
priority: 7
rate_limit:
requests_per_minute: 100
tokens_per_minute: 100000
timeout: 60s
max_retries: 3
# Routing Rules
routing:
# Default strategy: least_latency
strategy: "least_latency"
# Task-aware routing
task_routing:
coding:
preferred_providers: ["groq", "cerebras", "together"]
preferred_models: ["llama-3.1-70b", "llama-3.1-8b"]
reasoning:
preferred_providers: ["together", "deepinfra", "openrouter"]
preferred_models: ["llama-3.1-405b", "llama-3.1-70b"]
chat:
preferred_providers: ["groq", "cerebras", "fireworks"]
preferred_models: ["llama-3.1-8b", "llama-3.1-70b"]
fast:
preferred_providers: ["groq", "cerebras"]
preferred_models: ["llama-3.1-8b", "gemma2-9b"]
# Fallback chain when primary fails
fallback_chain:
- "groq"
- "cerebras"
- "fireworks"
- "together"
- "deepinfra"
- "novita"
- "openrouter"
# Circuit Breaker Settings
circuit_breaker:
failure_threshold: 5
success_threshold: 2
timeout: 30s
half_open_requests: 3
# Rate Limiting
rate_limiting:
global:
requests_per_minute: 500
tokens_per_minute: 100000
per_ip:
requests_per_minute: 60
tokens_per_minute: 10000
# Logging
logging:
level: "info"
format: "json"
output: "stdout"
access_log: true
error_log: true
# Metrics
metrics:
enabled: true
path: "/metrics"
port: 9090
EOF
```
### 5. Start 9Router
```bash
# Pull the latest image
docker compose pull
# Start the service
docker compose up -d
# View logs
docker compose logs -f ninerouter
# Check status
docker compose ps
```
### 6. Verify Installation
```bash
# Health check
curl -s http://localhost:4000/health | jq .
# List available models
curl -s http://localhost:4000/v1/models | jq .
# Test chat completion
curl -s -X POST http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <ANY_STRING>" \
-d '{
"model": "auto",
"messages": [{"role": "user", "content": "Hello! What model are you?"}],
"max_tokens": 100,
"temperature": 0.7
}' | jq .
```
---
## 🔧 Advanced Configuration
### Custom Routing Rules
Create `config/routing.yaml` for advanced routing:
```yaml
# Custom routing rules
rules:
# Route coding tasks to Groq (fastest for coding)
- name: "coding_tasks"
condition: "contains(messages[-1].content, 'code') or contains(messages[-1].content, 'function') or contains(messages[-1].content, 'class')"
providers: ["groq", "cerebras"]
models: ["llama-3.1-70b-versatile", "llama3.1-70b"]
# Route long context to Together/DeepInfra
- name: "long_context"
condition: "prompt_tokens > 8000"
providers: ["together", "deepinfra", "novita"]
models: ["llama-3.1-405b", "llama-3.1-70b"]
# Route creative writing to Together/Fireworks
- name: "creative_writing"
condition: "contains(system_prompt, 'creative') or contains(system_prompt, 'story')"
providers: ["together", "fireworks", "novita"]
models: ["llama-3.1-70b", "llama-3.1-405b"]
# Default fallback
default:
providers: ["groq", "cerebras", "fireworks", "together", "deepinfra"]
strategy: "least_latency"
```
Update `docker-compose.yml` to mount the routing config:
```yaml
volumes:
- ./config:/app/config:ro
- ./config/routing.yaml:/app/config/routing.yaml:ro
```
### Model Aliases
Create `config/models.yaml` for friendly model names:
```yaml
aliases:
# Friendly names -> actual model IDs
"gpt-4": "llama-3.1-70b-versatile"
"gpt-4-turbo": "llama-3.1-70b-versatile"
"gpt-3.5-turbo": "llama-3.1-8b-instant"
"claude-3-opus": "llama-3.1-405b-instruct"
"claude-3-sonnet": "llama-3.1-70b-versatile"
"claude-3-haiku": "llama-3.1-8b-instant"
"gemini-pro": "llama-3.1-70b-versatile"
"gemini-flash": "llama-3.1-8b-instant"
# Coding optimized
"coder": "llama-3.1-70b-versatile"
"coder-fast": "llama-3.1-8b-instant"
# Reasoning optimized
"reasoner": "llama-3.1-405b-instruct"
"reasoner-fast": "llama-3.1-70b-versatile"
# Fast/cheap
"fast": "llama-3.1-8b-instant"
"cheap": "llama-3.1-8b-instant"
# Auto (let router decide)
"auto": "auto"
"best": "auto"
```
---
## 🔌 Integration Examples
### Python (OpenAI SDK)
```python
from openai import OpenAI
# Point to your 9Router instance
client = OpenAI(
base_url="http://localhost:4000/v1",
api_key="not-needed" # 9Router ignores this
)
# Simple chat
response = client.chat.completions.create(
model="auto", # Let 9Router choose best model
messages=[
{"role": "system", "content": "You are a helpful coding assistant."},
{"role": "user", "content": "Write a Python function to parse JSON safely"}
],
max_tokens=500,
temperature=0.3
)
print(response.choices[0].message.content)
# Streaming
stream = client.chat.completions.create(
model="coder", # Uses alias -> llama-3.1-70b
messages=[{"role": "user", "content": "Write a REST API in FastAPI"}],
stream=True,
max_tokens=2000
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```
### Python (Async)
```python
import asyncio
from openai import AsyncOpenAI
async def main():
client = AsyncOpenAI(
base_url="http://localhost:4000/v1",
api_key="not-needed"
)
# Concurrent requests
tasks = [
client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": f"Write a haiku about {topic}"}],
max_tokens=50
)
for topic in ["AI", "Linux", "Docker", "Rust", "Python"]
]
responses = await asyncio.gather(*tasks)
for i, resp in enumerate(responses):
print(f"{['AI','Linux','Docker','Rust','Python'][i]}: {resp.choices[0].message.content}")
asyncio.run(main())
```
### JavaScript/TypeScript
```javascript
import OpenAI from 'openai';
const client = new OpenAI({
baseURL: 'http://localhost:4000/v1',
apiKey: 'not-needed',
dangerouslyAllowBrowser: true // For browser usage
});
// Non-streaming
async function chat() {
const response = await client.chat.completions.create({
model: 'auto',
messages: [
{ role: 'system', content: 'You are a Linux expert.' },
{ role: 'user', content: 'How do I debug a segfault in C?' }
],
max_tokens: 500,
temperature: 0.3
});
console.log(response.choices[0].message.content);
}
// Streaming
async function streamChat() {
const stream = await client.chat.completions.create({
model: 'coder',
messages: [{ role: 'user', content: 'Write a systemd service file' }],
stream: true,
max_tokens: 1500
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content || '';
process.stdout.write(content);
}
}
```
### cURL Examples
```bash
# Simple chat
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer dummy" \
-d '{
"model": "auto",
"messages": [{"role": "user", "content": "Explain Docker in one paragraph"}],
"max_tokens": 200,
"temperature": 0.7
}'
# Streaming
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer dummy" \
-d '{
"model": "coder",
"messages": [{"role": "user", "content": "Write a bash script to backup a directory"}],
"stream": true,
"max_tokens": 1000
}'
# List models
curl -s http://localhost:4000/v1/models | jq '.data[] | {id: .id, owned_by: .owned_by}'
# Health check
curl -s http://localhost:4000/health | jq .
```
### Go
```go
package main
import (
"context"
"fmt"
"log"
"github.com/sashabaranov/go-openai"
)
func main() {
client := openai.NewClientWithConfig(openai.ClientConfig{
BaseURL: "http://localhost:4000/v1",
APIKey: "not-needed",
})
resp, err := client.CreateChatCompletion(
context.Background(),
openai.ChatCompletionRequest{
Model: "auto",
Messages: []openai.ChatCompletionMessage{
{Role: "user", Content: "Write a Go HTTP server"},
},
MaxTokens: 500,
Temperature: 0.3,
},
)
if err != nil {
log.Fatal(err)
}
fmt.Println(resp.Choices[0].Message.Content)
}
```
### Rust
```rust
use openai_rust::{ChatCompletionMessage, ChatCompletionRequest, MessageRole};
use reqwest::Client;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let client = Client::new();
let url = "http://localhost:4000/v1/chat/completions";
let request = ChatCompletionRequest {
model: "auto".to_string(),
messages: vec![
ChatCompletionMessage {
role: MessageRole::User,
content: Some("Explain Rust ownership".to_string()),
name: None,
function_call: None,
}
],
max_tokens: Some(300),
temperature: Some(0.3),
stream: Some(false),
..Default::default()
};
let response = Client::new()
.post(url)
.header("Authorization", "Bearer dummy")
.header("Content-Type", "application/json")
.json(&request)
.send()
.await?
.json::<serde_json::Value>()
.await?;
println!("{}", response["choices"][0]["message"]["content"]);
Ok(())
}
```
---
## 📊 Monitoring & Observability
### Prometheus Metrics
```bash
# Enable in config.yaml
metrics:
enabled: true
path: "/metrics"
port: 9090
```
```bash
# Scrape metrics
curl -s http://localhost:9090/metrics | grep ninerouter
# Key metrics to monitor:
# ninerouter_requests_total{provider,status}
# ninerouter_request_duration_seconds{provider,quantile}
# ninerouter_active_connections
# ninerouter_provider_health{provider}
# ninerouter_rate_limit_remaining{provider}
# ninerouter_circuit_breaker_state{provider}
```
### Grafana Dashboard
```json
{
"dashboard": {
"title": "9Router Metrics",
"panels": [
{
"title": "Requests per Second by Provider",
"targets": [{"expr": "rate(ninerouter_requests_total[5m])"}]
},
{
"title": "Latency (p50, p95, p99)",
"targets": [
{"expr": "histogram_quantile(0.50, ninerouter_request_duration_seconds_bucket)"},
{"expr": "histogram_quantile(0.95, ninerouter_request_duration_seconds_bucket)"},
{"expr": "histogram_quantile(0.99, ninerouter_request_duration_seconds_bucket)"}
]
},
{
"title": "Provider Health",
"targets": [{"expr": "ninerouter_provider_health"}]
},
{
"title": "Error Rate",
"targets": [{"expr": "rate(ninerouter_requests_total{status=~\"5..\"}[5m])"}]
}
]
}
}
```
### Log Aggregation (Loki)
```yaml
# docker-compose.yml addition
services:
ninerouter:
logging:
driver: loki
options:
loki-url: "http://loki:3100/loki/api/v1/push"
loki-retries: "5"
loki-batch-size: "400"
loki-timeout: "10s"
```
---
## 🚀 Production Deployment
### Systemd Service
```bash
sudo tee /etc/systemd/system/ninerouter.service << 'EOF'
[Unit]
Description=9Router AI Model Router
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/home/ryan/9router
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
TimeoutStartSec=120
TimeoutStopSec=30
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable ninerouter
sudo systemctl start ninerouter
```
### Reverse Proxy (Nginx)
```nginx
# /etc/nginx/sites-available/ninerouter
server {
listen 80;
server_name ai.yourdomain.com;
# Redirect HTTP to HTTPS
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name ai.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/ai.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ai.yourdomain.com/privkey.pem;
# Security headers
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
# Rate limiting
limit_req_zone $binary_remote_addr zone=api:10m rate=60r/s;
limit_req zone=api burst=20 nodelay;
location / {
proxy_pass http://localhost:4000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Timeouts
proxy_connect_timeout 30s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Buffering
proxy_buffering off;
proxy_cache off;
}
# Health check endpoint (no rate limit)
location /health {
proxy_pass http://localhost:4000/health;
access_log off;
}
}
```
### SSL with Let's Encrypt
```bash
# Install certbot
sudo apt install certbot python3-certbot-nginx
# Get certificate
sudo certbot --nginx -d ai.yourdomain.com
# Auto-renewal
sudo systemctl enable certbot.timer
```
---
## 🔒 Security Best Practices
### API Key Management
```bash
# Use Docker secrets in production
echo "<GROQ_API_KEY>" | docker secret create groq_api_key -
# In docker-compose.yml:
secrets:
groq_api_key:
external: true
services:
ninerouter:
secrets:
- groq_api_key
environment:
- GROQ_API_KEY_FILE=/run/secrets/groq_api_key
```
### Network Isolation
```yaml
# docker-compose.yml
networks:
ninerouter-network:
driver: bridge
internal: true # No internet access from container
# Allow only specific outbound connections via sidecar proxy
# Or use egress firewall rules on host
```
### Authentication (Optional)
```yaml
# config.yaml
auth:
enabled: true
type: "bearer" # or "api_key"
tokens:
- name: "app-server"
token: "<GENERATED_TOKEN>"
rate_limit: 1000/min
- name: "dev-machine"
token: "<GENERATED_TOKEN>"
rate_limit: 100/min
```
Generate tokens:
```bash
openssl rand -hex 32
```
---
## 🐛 Troubleshooting
### Common Issues
| Issue | Cause | Solution |
|-------|-------|----------|
| `connection refused` | Container not running | `docker compose ps`, check logs |
| `401 unauthorized` | Invalid/missing API key | Check `.env` file, verify keys |
| `429 rate limited` | Provider rate limit hit | Add more providers, check rate limits |
| `502 bad gateway` | Upstream provider down | Check circuit breaker, add more providers |
| `model not found` | Wrong model name | Check `/v1/models` for valid names |
| High latency | Provider overloaded | Check metrics, adjust routing |
### Debug Commands
```bash
# View real-time logs
docker compose logs -f ninerouter --tail 100
# Check container resources
docker stats ninerouter
# Test specific provider
curl -s http://localhost:4000/v1/models | jq '.data[] | select(.owned_by=="groq")'
# Test specific provider endpoint directly
curl -H "Authorization: Bearer <GROQ_API_KEY>" https://api.groq.com/openai/v1/models
# Check circuit breaker status
curl -s http://localhost:4000/admin/circuits | jq .
# View routing decisions
curl -s http://localhost:4000/admin/routes | jq .
# Reset circuit breakers
curl -X POST http://localhost:4000/admin/circuits/reset
```
### Log Analysis
```bash
# Search for errors
docker compose logs ninerouter | grep -i error
# Search for rate limits
docker compose logs ninerouter | grep -i "rate limit\|429"
# Search for circuit breaker trips
docker compose logs ninerouter | grep -i "circuit breaker\|open state"
# Provider selection debugging
docker compose logs ninerouter | grep -i "routing\|selected\|provider"
```
---
## 📈 Performance Tuning
### For Low Latency
```yaml
# config.yaml
server:
read_timeout: 10s
write_timeout: 60s
routing:
strategy: "least_latency" # or "round_robin" for balanced
circuit_breaker:
failure_threshold: 3
timeout: 10s
```
### For High Throughput
```yaml
server:
read_timeout: 30s
write_timeout: 600s
rate_limiting:
global:
requests_per_minute: 2000
tokens_per_minute: 500000
```
### Resource Limits
```yaml
# docker-compose.yml
deploy:
resources:
limits:
cpus: '2'
memory: 1G
reservations:
cpus: '0.5'
memory: 256M
```
---
## 📋 Maintenance
### Regular Tasks
```bash
#!/bin/bash
# /home/ryan/9router/maintenance.sh
# Update image
docker compose pull
# Restart with new image
docker compose up -d
# Clean up old images
docker image prune -f
# Rotate logs
docker compose exec ninerouter logrotate /etc/logrotate.d/9router
# Check disk space
df -h /home/ryan/9router
```
### Backup Configuration
```bash
# Backup config
tar -czf ~/backups/9router-config-$(date +%Y%m%d).tar.gz \
~/9router/config ~/9router/.env ~/9router/docker-compose.yml
# Restore
tar -xzf ~/backups/9router-config-20260716.tar.gz -C /
```
---
## 📚 Additional Resources
- **GitHub Repository:** https://github.com/9router/9router
- **Documentation:** https://github.com/9router/9router/blob/main/docs
- **API Reference:** https://github.com/9router/9router/blob/main/docs/api.md
- **Configuration Guide:** https://github.com/9router/9router/blob/main/docs/config.md
- **Discord Community:** https://discord.gg/9router
- **Issues/Bug Reports:** https://github.com/9router/9router/issues
---
## ✅ Summary Checklist
- [ ] Created project directory structure
- [ ] Configured `.env` with API keys (use placeholders)
- [ ] Created `docker-compose.yml` with all providers
- [ ] Created `config/config.yaml` with provider configs
- [ ] Created `config/config.yaml` with routing rules
- [ ] Created `config/models.yaml` with model aliases
- [ ] Started 9Router with `docker compose up -d`
- [ ] Verified health endpoint: `curl http://localhost:4000/health`
- [ ] Tested chat completion: `curl -X POST http://localhost:4000/v1/chat/completions`
- [ ] Listed models: `curl http://localhost:4000/v1/models`
- [ ] Set up systemd service for auto-start
- [ ] Configured reverse proxy (nginx/Caddy) for HTTPS
- [ ] Set up monitoring (Prometheus + Grafana)
- [ ] Configured log aggregation (Loki/ELK)
- [ ] Documented API keys in password manager
- [ ] Tested failover by stopping a provider
---
## 🎉 You're Ready!
Your 9Router instance is now running at `http://localhost:4000/v1` with:
- ✅ **OpenAI-compatible API** - Drop-in replacement for OpenAI SDK
- ✅ **7+ free model providers** with automatic failover
- ✅ **Smart routing** based on task type
- ✅ **Model aliases** for easy migration from OpenAI
- ✅ **Health checks** and monitoring ready
- ✅ **Production-ready** with systemd, reverse proxy, SSL
**Next Steps:** Add your actual API keys to `.env`, test with your applications, and enjoy free, resilient AI inference!---
*Last updated: July 16, 2026 | 9Router version: latest | Guide version: 1.0*