Gateway

The same router the chat uses, with an OpenAI-compatible API in front of it. Existing clients work unmodified — the only line that changes is the base URL.

Base URL and key

https://api.horai.sh/v1

Create a key in Settings → Gateway keys. It is shown once and stored only as a hash, so it cannot be recovered — if you lose one, revoke it and make another.

Keys look like horai_…. Pass one as a bearer token:

Authorization: Bearer horai_your_key_here

Cookies do not authenticate here. The key is the only credential /v1 accepts. A key carries your balance, so treat it like a password: keep it server-side, and never ship it to a browser.

Listing models

GET /v1/models returns everything you can call right now, with its price per million tokens.

curl https://api.horai.sh/v1/models \
  -H "Authorization: Bearer $HORAI_API_KEY"

The list is live and it is yours: add an Anthropic key and Claude models appear in it. A model with no published price is not in the list and will not run.

Chat completions

POST /v1/chat/completions takes messages, model (defaults to auto), stream, max_tokens and temperature.

curl https://api.horai.sh/v1/chat/completions \
  -H "Authorization: Bearer $HORAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Say hello"}]
  }'

With the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    api_key="horai_...",                  # your Horai key
    base_url="https://api.horai.sh/v1",   # the only line that changes
)

resp = client.chat.completions.create(
    model="auto",                         # or name one exactly, as provider/model
    messages=[{"role": "user", "content": "Say hello"}],
)
print(resp.choices[0].message.content)

With the Node SDK:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.HORAI_API_KEY,
  baseURL: "https://api.horai.sh/v1",
});

const resp = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Say hello" }],
});

The response

OpenAI's shape, plus what the call actually cost:

{
  "id": "chatcmpl-7c6vznib48",
  "object": "chat.completion",
  "created": 1784231135,
  "model": "groq/llama-3.3-70b-versatile",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello!" },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 40,
    "completion_tokens": 2,
    "total_tokens": 42
  },
  "horai": {
    "cost_micro_usd": 24,
    "byok": false
  }
}

horai.cost_micro_usd is the exact amount debited, in micro-USD — 1,000,000 is one dollar. horai.byok is true when the call ran on your own provider key, in which case the cost is zero here and your provider bills you directly.

Streaming

Set stream: true for server-sent events. Chunks arrive as chat.completion.chunk, the last one carries usage, and the stream ends with data: [DONE].

curl -N https://api.horai.sh/v1/chat/completions \
  -H "Authorization: Bearer $HORAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Say hello"}],
    "stream": true
  }'

Hang up and the generation is stopped upstream immediately. You are not billed for tokens you did not receive.

Errors

Errors use OpenAI's envelope, so existing retry logic reads them unchanged.

{
  "error": {
    "message": "Balance is $0.0040; at least $0.01 is needed to start a call.",
    "type": "insufficient_quota",
    "code": "insufficient_balance"
  }
}

Billing

• At cost. You pay tokens times the published price per million, and nothing on top.
• From your balance. The same balance as builds. A call needs at least $0.01 to start.
• Charged after the call. A completion's price is not knowable until it exists, so nothing is held up front and you are never charged for more than you used.
• Your own key costs nothing here. Calls that run on a key you added are not debited at all. The usage is still recorded so your history stays complete.
• Rate limit. 60 requests per minute per key.

Every debit shows up in your usage dashboard as gateway.inference.

Next

Usage and billing covers deposits and where the numbers come from.