API reference

Overview

The SeaWhale AI API is closely compatible with the OpenAI Chat API, with additions for multi-model scenarios. We normalize the interfaces of different models and providers so you can call any of them the same way, which greatly reduces the learning curve and integration effort.

Requests

Chat completions request

Send a POST request to the /v2/chat/completions endpoint to call a model.

The complete TypeScript type definition below shows every available request parameter:

TypeScript

// Request parameter types (subtypes defined below)
type Request = {
  // Provide either messages or prompt
  messages?: Message[];
  prompt?: string;

  // Model name; falls back to your default model when omitted
  model?: string;

  // Force the model to output a specific format (e.g. JSON)
  response_format?: { type: 'json_object' };

  stop?: string | string[]; // Stop sequences; generation halts on a match
  stream?: boolean; // Enable streaming output

  // Core LLM parameters
  max_tokens?: number; // Maximum tokens to generate, range: [1, context_length)
  temperature?: number; // Sampling temperature, controls randomness, range: [0, 2]

  // Tool calling
  // Passed straight through for providers with OpenAI-compatible interfaces
  // Mapped and converted for providers with custom interfaces
  // Otherwise tools are converted to a YAML template and the model replies with an assistant message
  tools?: Tool[];
  tool_choice?: ToolChoice;

  // Advanced optional parameters
  seed?: number; // Random seed (integer) for reproducible output
  top_p?: number; // Nucleus sampling, range: (0, 1]
  top_k?: number; // Top-K sampling, range: [1, Infinity). Note: not supported by OpenAI models
  frequency_penalty?: number; // Reduces repetition, range: [-2, 2]
  presence_penalty?: number; // Encourages new topics, range: [-2, 2]
  repetition_penalty?: number; // Repetition penalty, range: (0, 2]
  logit_bias?: { [key: number]: number }; // Token bias; adjusts the likelihood of specific tokens
  top_logprobs: number; // Return log probabilities for the top N tokens (integer)
  min_p?: number; // Minimum probability threshold, range: [0, 1]
  top_a?: number; // Top-A sampling, range: [0, 1]

  // Predicted outputs (reduces latency)
  prediction?: { type: 'content'; content: string };

  // SeaWhale AI specific parameters
  transforms?: string[]; // Prompt transforms
  models?: string[]; // Fallback model list
  route?: 'fallback'; // Routing strategy: fall back automatically on failure
  provider?: ProviderPreferences; // Provider preferences
  user?: string; // End-user identifier, used for abuse prevention
};

// Subtype definitions

type TextContent = {
  type: 'text';
  text: string;
};

type ImageContentPart = {
  type: 'image_url';
  image_url: {
    url: string; // Image URL or base64-encoded data
    detail?: string; // Optional, defaults to "auto"
  };
};

type ContentPart = TextContent | ImageContentPart;

type Message =
  | {
      role: 'user' | 'assistant' | 'system';
      content: string | ContentPart[]; // ContentPart[] is only valid for the "user" role
      name?: string; // Optional name; some models prepend it to the message
    }
  | {
      role: 'tool';
      content: string;
      tool_call_id: string;
      name?: string;
    };

type FunctionDescription = {
  description?: string;
  name: string;
  parameters: object; // JSON Schema object
};

type Tool = {
  type: 'function';
  function: FunctionDescription;
};

type ToolChoice =
  | 'none'   // Do not use tools
  | 'auto'   // Choose automatically
  | {
      type: 'function';
      function: {
        name: string; // Name of the function to call
      };
    };

About the response_format parameter

Use response_format to make the model return structured JSON.

Support: this parameter is currently supported by only some models, including OpenAI models and Nitro models. Check the model list to confirm your chosen model supports it before using it.

Tip: to force routing to a provider that supports the parameter, set require_parameters to true in the provider preferences.

Responses

Chat completions response

SeaWhale AI normalizes responses from every model and provider to the OpenAI Chat API specification.

What normalization means:

• choices is always an array, even when the model returns a single result
• For streaming requests, each choice carries a delta property
• For non-streaming requests, each choice carries a message property
• This consistency lets the same code handle responses from any model

The complete TypeScript response types:

TypeScript

// Response types (subtypes defined below)
type Response = {
  id: string;
  // The choice type varies with the stream parameter and input type (messages or prompt)
  choices: (NonStreamingChoice | StreamingChoice | NonChatChoice)[];
  created: number; // Unix timestamp
  model: string; // The model actually used
  object: 'chat.completion' | 'chat.completion.chunk';

  system_fingerprint?: string; // System fingerprint, when the provider supports it

  // Non-streaming requests always return usage
  // Streaming requests return a final usage object, at which point choices is an empty array
  usage?: ResponseUsage;
};

// Token usage (passed through when the provider reports it, otherwise computed with the GPT-4 tokenizer)
type ResponseUsage = {
  prompt_tokens: number; // Tokens consumed by the prompt (including images and tool calls)
  completion_tokens: number; // Tokens consumed by the generated content
  total_tokens: number; // Total tokens
};

// Subtype definitions

type NonChatChoice = {
  finish_reason: string | null;
  text: string;
  error?: ErrorResponse;
};

type NonStreamingChoice = {
  finish_reason: string | null; // Normalized finish reason
  message: {
    content: string | null;
    role: string;
    tool_calls?: ToolCall[];
  };
  error?: ErrorResponse;
};

type StreamingChoice = {
  finish_reason: string | null;
  delta: {
    content: string | null; // Incremental content
    role?: string;
    tool_calls?: ToolCall[];
  };
  error?: ErrorResponse;
};

type ErrorResponse = {
  code: number; // Error code
  message: string; // Error message
  metadata?: Record<string, unknown>; // Extra detail, such as provider info or the original error
};

type ToolCall = {
  id: string;
  type: 'function';
  function: FunctionCall;
};

Example response

{
  "id": "gen-xxxxxxxxxxxxxx",
  "choices": [
    {
      "finish_reason": "stop",
      "native_finish_reason": "stop",
      "message": {  // "delta" for streaming requests
        "role": "assistant",
        "content": "Hello!"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 4,
    "total_tokens": 14
  },
  "model": "openai/gpt-3.5-turbo"
}

Finish reason

SeaWhale AI normalizes finish_reason across all models to these values:

• tool_calls — the model called a tool
• stop — completed normally (the model ended the response itself)
• length — hit the maximum token limit
• content_filter — content filtering was triggered
• error — an error occurred