AI-Proxy Docs

Everything you need to run your automations on the subscriptions you already pay for — one endpoint, one key, every media type.

Install & Quickstart

Desktop app (recommended)

macOS (Homebrew):

brew tap meta-thinking/tap
brew trust --tap meta-thinking/tap     # Homebrew 6+: third-party taps need a one-time trust
brew install --cask --no-quarantine ai-proxy

Windows / Linux: download from the GitHub releases on meta-thinking/homebrew-tap (tags desktop-v*):

Windows: .msi installer
Linux: AppImage or .deb

First launch — the app is free forever and needs no account:

Open AI-Proxy. The gateway starts on http://localhost:8317/v1.
Go to AI Accounts and sign in to the subscriptions you own (Gemini, Claude, Codex/ChatGPT, Kimi, xAI) via OAuth in your browser. No per-token API keys — it can't silently bill you; on quota it fails loudly.
Open the Server tab and copy the two values every client needs:
- Base URL: http://localhost:8317/v1
- API key: the single ai-proxy-… key

That one key fronts every connected provider plus local Ollama; you pick the provider by model name (gemini-3-flash, claude-sonnet-4-6, …), not by key.

Where things live (macOS): ~/Library/Application Support/AI-Proxy/ — config.yaml (port + api-keys), auths/ (OAuth tokens), logs/. On Linux: ${XDG_CONFIG_HOME:-~/.config}/AI-Proxy/.

Pro plan (image / video / voice routes + one-click client configs): sign in at aiproxy.meta-thinking.net, upgrade at /pricing. On the free plan those routes return HTTP 402; text stays free forever.

Headless CLI (servers, Mac minis, no GUI)

brew tap meta-thinking/tap
brew trust --tap meta-thinking/tap
brew install ai-proxy-cli

Commands:

ai-proxy start            # start gateway (:8317) + engine, in the background
ai-proxy stop             # stop both
ai-proxy status           # health + endpoint info (prints Base URL + API key)
ai-proxy login <provider> # OAuth login: gemini | claude | codex | kimi | xai
ai-proxy key              # print the API key
ai-proxy models           # list models on the unified endpoint

First ai-proxy start bootstraps the config dir and generates the ai-proxy-… key automatically. ai-proxy login gemini runs a --no-browser OAuth flow — copy the URL it prints into any browser and approve.

Pro on a server: export your license key before starting — the free plan works without any key:

AIPROXY_LICENSE_KEY=ap_sk_… ai-proxy start

Quick smoke test:

curl -s http://localhost:8317/v1/chat/completions \
  -H "Authorization: Bearer $(ai-proxy key)" -H "Content-Type: application/json" \
  -d '{"model":"gemini-3-flash","messages":[{"role":"user","content":"hi"}]}'

How AI-Proxy works

Gateway architecture

AI-Proxy runs two HTTP servers in one process. The gateway owns the public port (default :8317, bound on all interfaces so Tailscale access like http://<mac>:8317 works). Behind it, the bundled CLIProxyAPI engine — the text brain that holds your OAuth sessions (Gemini, Claude, Codex, …) and Ollama — listens on port+1, localhost-only. Clients never talk to the engine directly.

             ┌────────────────────────── your machine ──────────────────────────┐
             │                                                                  │
 client ────►│  gateway :8317  (public, CORS, license gate, one api key)        │
             │   │                                                              │
             │   ├── /v1/audio/speech ──────────► ElevenLabs  (key injected)    │
             │   ├── /v1/audio/transcriptions ──► ElevenLabs Scribe             │
             │   ├── /v1/images/generations ───► engine chat → Gemini image     │
             │   ├── /v1/videos, /v1/videos/{id}► Luma / HeyGen async jobs      │
             │   ├── /v1/models ───────────────► engine list + "kind" tags      │
             │   └── everything else ──────────► engine :8318 (127.0.0.1 only)  │
             │                                    byte-for-byte, SSE-safe       │
             └──────────────────────────────────────────────────────────────────┘

The catch-all is a streaming-safe reverse proxy (FlushInterval: -1), so /v1/chat/completions SSE streams flow through untouched. Chat requests also get a model self-heal cascade: if the named model is retired, unavailable, or absent — or none was sent — the gateway tries an ordered fallback (requested → same-family current → live default → a local Ollama model → claude-sonnet-4-6 → gemini-3-flash) until one answers. It only cascades on not-found style errors; real errors (401, 429, bad request) pass straight through unchanged.

The one-key model

You hold exactly one credential: the ai-proxy-… key from config.yaml (api-keys: list, re-read per request — rotations apply instantly). Capability routes validate that key, then swap in the real provider key server-side from providers.json (ElevenLabs, Luma, HeyGen). Your automations never see provider keys. Errors are deterministic: 401 bad ai-proxy key, 424 provider not connected, 402 premium route on the Free plan, 502 engine down.

Chat (proxied to the engine)

curl http://localhost:8317/v1/chat/completions \
  -H "Authorization: Bearer $AI_PROXY_KEY" -H "Content-Type: application/json" \
  -d '{"model": "gemini-2.5-pro", "messages": [{"role": "user", "content": "hi"}], "stream": true}'

Tool calling (function-calling passthrough)

The gateway is transparent for the OpenAI function-calling protocol: the CLIProxyAPI engine translates your tools[] into each provider's native tool format and back, so tool_calls, finish_reason: "tool_calls", streamed tool-call deltas over SSE, tool_choice, parallel calls, and the role: "tool" result round-trip all work through the same /v1/chat/completions route — on the free text tier.

Crucially, the gateway never re-encodes your request body. The retired-model remap rewrites only the top-level model field (byte-level), so your tools[] and messages[] arrays reach the engine byte-identical — no reordering or mangling that would break an agent. Streaming stays live (FlushInterval: -1), and the chat path has no request timeout, so long agent loops aren't cut off.

curl http://localhost:8317/v1/chat/completions \
  -H "Authorization: Bearer $AI_PROXY_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{"type": "function", "function": {
      "name": "get_weather",
      "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}
    }}]
  }'
# → choices[0].finish_reason: "tool_calls", with a get_weather call in message.tool_calls

Provider limits are upstream, not in the gateway: Claude does forced tool_choice and parallel calls reliably; Gemini currently ignores forced tool_choice and returns one call at a time; small Ollama models emit weak arguments. Pick tool-capable models via the "tools" flag on /v1/models (below).

Images — `POST /v1/images/generations`

Standard OpenAI shape, backed by the free Gemini subscription. dall-e-*, gpt-image-*, or an empty model map to the Gemini image model (gemini-3.1-flash-image, overridable via GATEWAY_IMAGE_MODEL); any other model id passes through. n is clamped to 1–4 (generated in parallel). response_format: b64_json (default) or url — url returns a data-URL, no file hosting.

curl http://localhost:8317/v1/images/generations \
  -H "Authorization: Bearer $AI_PROXY_KEY" -H "Content-Type: application/json" \
  -d '{"model": "dall-e-3", "prompt": "pyramids at dawn, watercolor", "n": 1, "response_format": "b64_json"}' \
  | jq -r '.data[0].b64_json' | base64 -d > out.png

Audio

Speech returns raw audio bytes (ElevenLabs behind the scenes). voice accepts an ElevenLabs voice_id; anything else (including OpenAI names like alloy) falls back to your account's first voice. response_format: mp3 (default, audio/mpeg), wav/pcm, opus. Optional speed.

curl http://localhost:8317/v1/audio/speech \
  -H "Authorization: Bearer $AI_PROXY_KEY" -H "Content-Type: application/json" \
  -d '{"model": "elevenlabs-tts", "input": "Welcome to Cairo!", "voice": "<voice_id>", "response_format": "mp3"}' \
  --output speech.mp3

Transcriptions take multipart form data (ElevenLabs Scribe, scribe_v1). response_format=json (default) returns {"text": "..."}; text returns plain text.

curl http://localhost:8317/v1/audio/transcriptions \
  -H "Authorization: Bearer $AI_PROXY_KEY" \
  -F file=@note.m4a -F model=whisper-1 -F response_format=json

Videos — async job lifecycle

Long renders use an OpenAI-style job API. Model prefix picks the provider: heygen-* → HeyGen avatar video, anything else → Luma (pure text-to-video). Jobs are in-memory — they do not survive a restart.

POST /v1/videos {model, prompt}          →  {"id":"vid_…","object":"video","status":"in_progress"}
GET  /v1/videos/{id}                     →  status: queued | in_progress | completed | failed
GET  /v1/videos/{id}/content             →  302 redirect to the video (409 while not completed)

JOB=$(curl -s http://localhost:8317/v1/videos \
  -H "Authorization: Bearer $AI_PROXY_KEY" -H "Content-Type: application/json" \
  -d '{"model": "luma-video", "prompt": "felucca sailing the Nile at sunset"}' | jq -r .id)

curl -s http://localhost:8317/v1/videos/$JOB -H "Authorization: Bearer $AI_PROXY_KEY" | jq .status
curl -L http://localhost:8317/v1/videos/$JOB/content -H "Authorization: Bearer $AI_PROXY_KEY" -o tour.mp4

`GET /v1/models` — tagged catalog

The gateway forwards the engine's model list and tags every entry with a kind: text, image, audio, video, or embedding (by id heuristics — veo/sora/heygen → video, imagen/dall/flux → image, tts/whisper/eleven → audio, embed → embedding). Clients that don't know the field ignore it; clients that do can filter by capability.

Every entry also carries a boolean tools flag saying whether that model can do function-calling. Subscription text models (Gemini, Claude, GPT, Codex, Grok, Kimi) are true; image/audio/video models are false; Ollama is model-dependent (llama3.1+, qwen2.5+, mistral, command-r → true; gemma3/gemma4 and tiny models like *:1b → false). Filter for agent-ready models with select(.tools):

curl -s http://localhost:8317/v1/models -H "Authorization: Bearer $AI_PROXY_KEY" \
  | jq '.data[] | select(.kind == "image") | .id'

curl -s http://localhost:8317/v1/models -H "Authorization: Bearer $AI_PROXY_KEY" \
  | jq '.data[] | select(.tools) | .id'

Connect your tools

Everything speaks the OpenAI API. One Base URL, one ai-proxy-… key, choose the model per request. Authoritative model list: GET <base>/v1/models (each model is tagged with a "kind": text / image / audio / video).

Pick the Base URL by where the client runs:

Client location	Base URL
Same machine (opencode, Cursor, curl)	`http://localhost:8317/v1`
Docker container, proxy also in Docker (same network)	`http://ai-proxy:8317/v1`
Docker container, proxy on the host	`http://host.docker.internal:8317/v1`
Other Tailscale devices	`http://<mac-tailscale-name>:8317/v1`

Pro tip: the desktop app's Connect Clients tab (Pro) generates these per-tool configs one-click, pre-filled with your live key and reachable URL.

n8n

One OpenAI credential covers all providers — this is the recommended setup:

Credential type: OpenAI
API Key: ai-proxy-…
Base URL: http://ai-proxy:8317/v1 (keep the /v1; use host.docker.internal variant if the proxy runs on the host)
Model: any — gemini-3-flash, claude-sonnet-4-6, claude-opus-4-8, …

Native nodes also work — note the /v1 differences:

Credential	URL field	Value
OpenAI (all models)	Base URL	`http://ai-proxy:8317/v1` (keep `/v1`)
Anthropic (Claude only)	Base URL	`http://ai-proxy:8317` (no `/v1` — node appends `/v1/messages`)
Google Gemini (PaLM)	Host	`http://ai-proxy:8317` (no `/v1` — node appends `/v1beta/...`; Allowed Domains: All)

Same ai-proxy-… key for all three.

opencode

Generate a full provider config containing every live model on the proxy:

./scripts/gen-opencode-config.sh > ~/.config/opencode/opencode.jsonc            # this machine
./scripts/gen-opencode-config.sh <mac-name> > ~/.config/opencode/opencode.jsonc # other device

This writes an ai-proxy provider (baseURL http://localhost:8317/v1, your apiKey) whose models block mirrors /v1/models. Restart opencode and pick a model under "AI-Proxy (local subscriptions)". The generator also marks each tool-capable model, so opencode knows which models can drive its tool loop.

Buzz (Block's team chat with agents)

Buzz has no base-URL/API-key settings of its own — it drives agent harnesses over the Agent Client Protocol (ACP): Goose, Claude Code, or Codex. The chain that puts Buzz agents on your subscriptions:

Buzz → Goose → AI-Proxy (localhost:8317) → your subscriptions / Ollama

Install Goose (Block's OSS agent): brew install block-goose-cli
Point Goose at the gateway — ~/.config/goose/config.yaml:
```
GOOSE_PROVIDER: openai
GOOSE_MODEL: claude-sonnet-4-6   # any tool-capable id from /v1/models
OPENAI_HOST: http://localhost:8317
OPENAI_BASE_PATH: v1/chat/completions
```
plus OPENAI_API_KEY=ai-proxy-… in the environment. The desktop app's Connect Clients → Goose → Configure writes both in one click.
In Buzz, create an agent and pick Goose as its harness — every agent turn now rides the gateway (Goose is Buzz's default harness; buzz-acp literally spawns goose acp).

Tool calling & agents

The OpenAI function-calling protocol passes through the gateway end-to-end, and it runs on the free text route — tool calling is not a Pro feature. Verified working live:

tools[] on the request and structured tool_calls on the response
finish_reason: "tool_calls"
streamed tool-call deltas over SSE (live/incremental, not buffered)
the role: "tool" result round-trip
tool_choice forcing a specific tool
parallel tool calls

The gateway is transparent here: it does not re-encode your request body. The retired-model remap rewrites only the top-level model field, leaving your tools[] and messages[] arrays byte-identical.

List the tool-capable models — /v1/models now tags each model with "tools": true|false alongside its existing "kind":

curl -s http://localhost:8317/v1/models \
  -H "Authorization: Bearer ai-proxy-…" \
  | jq '.data[] | select(.tools) | .id'

Which models to pick:

Tier	Models	Notes
Best	`claude-sonnet-4-6`, `claude-opus-4-8`	Forced `tool_choice` and parallel tool calls both work
Good	`gemini-3-flash`, `gemini-3.1-pro-*`	Reliable for single tool calls
Self-hosted	Ollama `llama3.1+` / `qwen2.5+` / `mistral` / `command-r`	Work for tools (free, local)
Not tool-capable	`gemma3`, `gemma4`, tiny Ollama (`*:1b`), image/audio/video models	`tools: false` — filter them out

n8n AI Agent / Tools Agent node: use the OpenAI credential (same setup as above) and pick claude-sonnet-4-6 for the most reliable behaviour, or gemini-3-flash for simpler single-tool flows.

Provider limits (upstream, not the gateway): these come from the model/provider — the gateway just passes tools through unchanged.

Claude does forced tool_choice and parallel tool calls reliably.
Gemini currently ignores forced tool_choice and returns tool calls one at a time (no parallel).
Small Ollama models emit weak arguments.

So for anything that must force a specific tool or fan out to several calls at once — including the n8n AI Agent / Tools Agent node — choose claude-sonnet-4-6.

Cursor / any OpenAI SDK

Point the OpenAI-compatible settings at the gateway:

Base URL: http://localhost:8317/v1
API key: ai-proxy-…
Model: any from /v1/models

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8317/v1", api_key="ai-proxy-...")
resp = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "hi"}],
)

Plain curl

# chat (free)
curl -s http://localhost:8317/v1/chat/completions \
  -H "Authorization: Bearer ai-proxy-…" -H "Content-Type: application/json" \
  -d '{"model":"gemini-3-flash","messages":[{"role":"user","content":"hi"}]}'

# images (Pro) — any dall-e-*/gpt-image-* name maps to the Gemini image model
curl -s http://localhost:8317/v1/images/generations \
  -H "Authorization: Bearer ai-proxy-…" -H "Content-Type: application/json" \
  -d '{"model":"dall-e-3","prompt":"a tiny robot barista","n":1}' \
  | jq -r '.data[0].b64_json' | base64 -d > out.jpg

# audio (Pro) — TTS / STT via connected ElevenLabs account
curl -s http://localhost:8317/v1/audio/speech \
  -H "Authorization: Bearer ai-proxy-…" -H "Content-Type: application/json" \
  -d '{"model":"tts-1","input":"hello","voice":""}' -o out.mp3
curl -s http://localhost:8317/v1/audio/transcriptions \
  -H "Authorization: Bearer ai-proxy-…" -F file=@clip.mp3 -F model=whisper-1

# video (Pro) — async jobs; model prefix picks the provider (luma-* / heygen-*)
curl -s -X POST http://localhost:8317/v1/videos \
  -H "Authorization: Bearer ai-proxy-…" -H "Content-Type: application/json" \
  -d '{"model":"luma-video","prompt":"drone shot of the pyramids at sunrise"}'
curl -s  http://localhost:8317/v1/videos/<id>          -H "Authorization: Bearer ai-proxy-…"
curl -sL http://localhost:8317/v1/videos/<id>/content  -H "Authorization: Bearer ai-proxy-…" -o video.mp4

On the free plan the image/audio/video routes return HTTP 402 with an upgrade link; all text routes work forever without a plan.

Free vs Pro

AI-Proxy's core is free forever. Pro unlocks the multimodal routes on top of the same gateway and key. Prices live at aiproxy.meta-thinking.net/pricing.

Free (no account required)

The local gateway itself — the unified OpenAI-compatible endpoint (http://localhost:8317/v1 by default) with your single ai-proxy-… key from config.yaml.
All text models: /v1/chat/completions, /v1/models, and everything the text engine serves.
Tool calling / function calling for agents — the full OpenAI protocol on the free text route (not a Pro feature): tools[], structured tool_calls, finish_reason: "tool_calls", streamed tool-call deltas over SSE, the role: "tool" result round-trip, tool_choice forcing, and parallel tool calls. /v1/models tags each model with "tools": true|false so you can filter: jq '.data[] | select(.tools) | .id'.
Connecting your AI subscriptions via OAuth (Gemini, Claude, Codex/ChatGPT, Kimi, xAI).
Local Ollama models through the same endpoint.

Free plan features are exactly: local_gateway, oauth_providers, openai_compatible_api.

Pro (`premium_capabilities` on your plan)

Image — POST /v1/images/generations (standard OpenAI shape, backed by your free Gemini subscription; returns b64_json, or a data-URL when response_format: "url").
Voice — /v1/audio/speech and /v1/audio/transcriptions (ElevenLabs key injected server-side; clients only ever hold the ai-proxy key).
Video — POST /v1/videos async jobs + GET /v1/videos/{id} polling + GET /v1/videos/{id}/content redirect (Luma / HeyGen).
One-click quick configs in the app.
Priority support.

What happens when Free hits a Pro route

The gateway returns HTTP 402 with an upsell message — text routes are never affected:

{"error": {"message": "Image, video & voice are Pro features. Upgrade at https://aiproxy.meta-thinking.net/pricing — the text gateway stays free forever."}}

The gate covers /v1/audio/*, /v1/images/*, and /v1/videos. Everything else passes straight through to the text engine.

Your account

You don't need an account to use AI-Proxy — only to unlock Pro.

Signing in (device flow)

In the app, click Sign in. The app asks the account server to start a device-auth session and opens the returned URL in your browser.
Approve in the browser (sign-in at aiproxy.meta-thinking.net).
The app polls every few seconds until approval, then stores the issued ap_sk_… account token locally. The whole flow expires after a few minutes; if it times out, just start again.

What's stored where

On your machine: only the ap_sk_… token, in account.json next to config.yaml (macOS: ~/Library/Application Support/AI-Proxy/account.json), written with 0600 permissions. Deliberately a plain file rather than the OS keychain — no per-build keychain approval prompts.
In the cloud: your plan, purchase state, and email. The app refreshes this from GET /api/license/check and caches the last known license.

Signing out

Sign out deletes the local ap_sk_ token and clears the cached license. Nothing else on disk changes — your gateway config, provider OAuth logins, and Ollama setup are untouched.

Network failures fail open to Free

License checks never block the local gateway. If the account server is unreachable, the app keeps your cached plan and the gateway keeps the last known state (retrying periodically). The only thing a network error can do is keep Pro routes locked until a successful check says otherwise. Only an explicit 401 (revoked/invalid token) signs you out.

Troubleshooting & FAQ

The gateway is offline / clients get connection refused

Start AI-Proxy (menu bar app) or, headless: ai-proxy start. Check with ai-proxy status. If clients reach the port but get 502 AI-Proxy engine is not running., the internal text engine is down — restart AI-Proxy.

How do I change the port?

Edit port in config.yaml (macOS: ~/Library/Application Support/AI-Proxy/config.yaml) and restart. The default is 8317; the internal engine runs on port+1, bound to localhost only — don't point clients at it. The public port binds all interfaces, so LAN/Tailscale access (http://<machine>:8317/v1) works.

Ollama models don't show up

Make sure Ollama is installed and running (ollama serve, default port 11434), then check GET /v1/models — local models appear alongside subscription models, tagged with "kind". You can also install models from the app's Model Catalog tab.

Which models should I use for agents / tool calling?

Any model that advertises "tools": true in GET /v1/models. The gateway passes the OpenAI function-calling protocol through end-to-end (tools, tool_calls, streamed tool-call deltas, the role:"tool" round-trip, tool_choice, parallel calls), and tool calling runs on the free text route — it is not a Pro feature. Filter for tool-capable models with:

curl -s http://localhost:8317/v1/models -H "Authorization: Bearer $AIPROXY_KEY" \
  | jq '.data[] | select(.tools) | .id'

For picking one:

Most reliable: claude-sonnet-4-6 (or claude-opus-4-8). These are the only models that do forced tool_choice and parallel tool calls reliably — use them for the n8n AI Agent / Tools Agent node or anything that must force a specific tool or fan out to several at once.
Good for single tool calls: gemini-3-flash and gemini-3.1-pro-*. Note that Gemini currently ignores forced tool_choice and returns tool calls one at a time (no parallel) — that's an upstream provider limit, not the gateway.
Self-hosted: Ollama llama3.1+, qwen2.5+, mistral, command-r all report tools:true; small models (gemma3, *:1b) don't, and tiny models tend to emit weak tool arguments.

My agent sends a model name that no longer exists (or none) — does it break?

No. The gateway self-heals with a candidate cascade: if a request names a model that's retired, temporarily unavailable, or absent — or names no model at all — it tries an ordered fallback until one answers: requested → same-family current model → the live default → a local Ollama model → claude-sonnet-4-6 → gemini-3-flash. Retired provider ids (e.g. old Anthropic ids like claude-3-5-haiku-20241022) are also pruned from /v1/models and remapped in chat requests to their current replacements.

The cascade only triggers on not-found-style errors. Real errors pass straight through unchanged — a 401 auth failure or 429 rate limit still surfaces as itself, so you'll never silently drift onto a different model when the actual problem is your key or quota.

A provider stopped working / OAuth expired

Re-login from the app (AI Accounts → the provider) or headless: ai-proxy login <provider>. OAuth tokens live under auths/ in the app's data folder.

401 "Invalid AI-Proxy API key"

Your request's bearer key doesn't match api-keys in config.yaml. The gateway re-reads config.yaml on every request, so key rotations apply immediately — copy the current key (ai-proxy key on the CLI, or the app's endpoint panel).

Signed in, but the app says my account key is invalid (after a migration)

Account-server migrations can invalidate old ap_sk_ tokens. The app detects the 401, signs you out, and asks you to sign in again — the device flow issues a fresh token. Your plan is unaffected (it lives in the cloud).

424 "Audio needs a voice provider" / "Video needs a provider"

Image generation uses your Gemini subscription and needs no extra key. Voice needs a connected ElevenLabs account; video needs Luma or HeyGen. Open AI-Proxy → AI Accounts → Connect the provider, then retry — no restart needed, provider keys are re-read per request.

Is using my Claude subscription through AI-Proxy allowed?

It's a gray area under Anthropic's ToS — AI-Proxy uses your subscription via OAuth exactly as the official clients do, but routing it through a gateway isn't explicitly sanctioned. Billing safety: AI-Proxy is OAuth-only (no per-token API keys) and fails loudly on quota — it never silently falls back to paid API billing. Keep Anthropic API credits disabled on your account so nothing can bill even by accident.

Where are the logs?

macOS: ~/Library/Application Support/AI-Proxy/logs
Linux: ~/.config/AI-Proxy/logs
Windows: %APPDATA%\AI-Proxy\logs

Log size is capped (rotation, 200 MB total by default). Include the newest file when reporting issues.