OpenAI-compatible API

The base URL is the server's address with /v1:

http://127.0.0.1:5823/v1

With the default configuration there is no authentication on loopback, and clients that insist on an API key can be given any string. With an authentication mode configured, send an API key as a bearer token.

#With an SDK

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:5823/v1", api_key="unused")

reply = client.chat.completions.create(
    model="Qwen/Qwen2.5-1.5B-Instruct",
    messages=[{"role": "user", "content": "Say hello in Norwegian."}],
)
print(reply.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://127.0.0.1:5823/v1", apiKey: "unused" });

const stream = await client.chat.completions.create({
  model: "Qwen/Qwen2.5-1.5B-Instruct",
  messages: [{ role: "user", content: "Say hello in Norwegian." }],
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");

#Chat completions

POST /v1/chat/completions

curl http://127.0.0.1:5823/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model": "Qwen/Qwen2.5-1.5B-Instruct",
    "messages": [
      {"role": "system", "content": "You are terse."},
      {"role": "user", "content": "Why is the sky blue?"}
    ],
    "stream": true
  }'

The request takes messages, stream, temperature, top_p, max_tokens (or max_completion_tokens), seed, tools and tool_choice.

stream: true returns text/event-stream in OpenAI's chunk format, ending with [DONE]. Otherwise the answer is one JSON object. The numbers for each generation, prefill and decode rates and cached tokens, are in an extension field.

#Naming a model

model is a repository, or repo@backend for a particular backend:

{ "model": "Qwen/Qwen3-14B@gpu-q4" }

#Reasoning

A reasoning model's working arrives as reasoning_content, the field DeepSeek's API established, and content is the answer alone. Both stream.

#Tools

tools are passed to the model's own chat template and its calls come back as tool_calls. See Tool calls and agents.

#Models

GET /v1/models lists every model on the machine in OpenAI's shape, with a kvad object on each:

{
  "id": "Qwen/Qwen2.5-1.5B-Instruct",
  "object": "model",
  "owned_by": "huggingface",
  "kvad": { "tools": true, "resident": false, "kind": "chat" }
}

kind is chat, image or video. tools says whether the model's template has a place for tool definitions. resident says whether it is in memory now.

#Images

POST /v1/images/generations

OpenAI's fields are prompt, model, n (at most 4), size as WIDTHxHEIGHT, and response_format of b64_json (the default) or url. Beside them are the settings OpenAI has no field for:

Each image in the answer carries a kvad object with its id, its seed, its settings and the time each stage took.

When streaming, the events are image_generation.step, image_generation.partial_image and image_generation.completed.

#Videos

POST /v1/videos starts a job and answers at once.

The request takes prompt, model, size, and seconds or frames, plus fps, seed, audio, input_reference, steps, guidance_scale, negative_prompt, decoder, pipeline and loras. Video explains what each one does.

#Everything else

The server's own API is under /api: models, conversations, jobs, datasets, evals, benchmarks, metrics, accounts. It is documented by the server itself.

• http://127.0.0.1:5823/api is the reference, as a page.
• /api/openapi.json is the same thing as OpenAPI. It is generated from a table that a test compares against the router, so a route that exists is a route that is documented.
• kvad api lists every route, and kvad api GET /api/health calls one.

#What is not there

• tool_choice accepts auto and none. required and a named function are refused, because forcing a call would need a constrained sampler.
• There is no embeddings endpoint and no legacy /v1/completions. Raw completion without a chat template is POST /api/playground/complete.
• Requests are answered one at a time.