LLM Proxy

# What is LLM Proxy?

A Flask-based proxy that presents an OpenAI-compatible API endpoint while managing multiple LLM backends behind the scenes. Add models through the web UI, tag them as fast or smart, and the proxy handles auto-fallback, usage tracking, and cost calculation — no config files to hand-edit after initial setup.

# Features

Everything you need to manage LLM backends from a single endpoint.

# API Usage

Once running, point any OpenAI-compatible client at the proxy.

# Use the fast model (auto-fallback to smart on error)
curl http://localhost:8086/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fast",
    "messages": [{"role": "user", "content": "Tell me a joke!"}]
  }'

# Use the smart model for heavier tasks
curl http://localhost:8086/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "smart",
    "messages": [{"role": "user", "content": "Write a detailed analysis..."}]
  }'

# Use a specific model by its configured name
curl http://localhost:8086/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# List available models
curl http://localhost:8086/v1/models

# Quick Start

Get the proxy running in under a minute.

# 1. Install dependencies
./ubuntu-install.sh
# or manually: pip install -r requirements.txt

# 2. Configure
cp .env.example .env
# Edit .env — set PROXY_PORT, BIND_HOST, and FLASK_SECRET_KEY

# 3. Run
./start.sh

# 4. Open the web UI
open http://localhost:8086/

# 5. Add your first model through the browser
# No manual config file editing — just fill in the form!

Configuration

# Model Configuration

All model management is done through the web UI — no config files to hand-edit.

# Usage Reports

Visit /reports for matplotlib-powered cost and usage charts.