Models & Providers

ScalyClaw routes LLM calls through a configurable provider stack. You can register multiple models from different providers, assign priorities and weights for load balancing, and define fallback chains so that a single provider outage never stops the system. All model configuration lives in Redis — no config files on disk — and changes take effect immediately without a restart.

Model Configuration

Models are added and managed in the dashboard under Settings → Models. Each entry in the model list describes a single provider/model combination. You can have as many entries as you need; the orchestrator selects the right one at call time based on priority, weight, and the capabilities required by that particular request.

Model Properties

Fallback Chain

When a model call fails — due to a network error, rate-limit, timeout, or provider outage — ScalyClaw automatically tries the next available model. The fallback order is determined entirely by the priority field:

1. All models with the lowest priority value are candidates for the first attempt. If there are multiple, they share load according to their weight.
2. If every model in that priority group fails (or none has the required capabilities), the orchestrator moves to the next priority group.
3. This continues until either a call succeeds or all models are exhausted, in which case an error is returned to the channel.

Supported Providers

Providers are registered under config.models.providers as a map from a provider key to an object with an optional apiKey and optional baseUrl. Each model entry's provider field references one of these keys.

Example Configuration

The following shows a realistic multi-provider setup: Anthropic as the primary model, OpenAI as a same-priority peer with lower weight, and a local Ollama model as a last-resort fallback for basic requests.

{
  "models": {
    "providers": {
      "anthropic": { "apiKey": "sk-ant-..." },
      "openai": { "apiKey": "sk-..." },
      "local": { "baseUrl": "http://localhost:11434/v1" }
    },
    "models": [
      {
        "id": "claude-primary",
        "name": "claude-opus-4-6",
        "provider": "anthropic",
        "enabled": true,
        "priority": 1,
        "weight": 75,
        "temperature": 0.7,
        "maxTokens": 8192,
        "contextWindow": 200000,
        "toolEnabled": true,
        "imageEnabled": true,
        "audioEnabled": false,
        "videoEnabled": false,
        "documentEnabled": true,
        "reasoningEnabled": false,
        "inputPricePerMillion": 15.00,
        "outputPricePerMillion": 75.00
      },
      {
        "id": "gpt4o-secondary",
        "name": "gpt-4o",
        "provider": "openai",
        "enabled": true,
        "priority": 1,
        "weight": 25,
        "temperature": 0.7,
        "maxTokens": 4096,
        "contextWindow": 128000,
        "toolEnabled": true,
        "imageEnabled": true,
        "audioEnabled": false,
        "videoEnabled": false,
        "documentEnabled": false,
        "reasoningEnabled": false,
        "inputPricePerMillion": 2.50,
        "outputPricePerMillion": 10.00
      },
      {
        "id": "local-fallback",
        "name": "llama3.2",
        "provider": "local",
        "enabled": true,
        "priority": 2,
        "weight": 100,
        "temperature": 0.7,
        "maxTokens": 2048,
        "contextWindow": 8192,
        "toolEnabled": false,
        "imageEnabled": false,
        "audioEnabled": false,
        "videoEnabled": false,
        "documentEnabled": false,
        "reasoningEnabled": false,
        "inputPricePerMillion": 0,
        "outputPricePerMillion": 0
      }
    ],
    "embeddingModels": []
  }
}

With this config, 75% of priority-1 requests go to Claude (weight 75 out of 100) and 25% go to GPT-4o (weight 25 out of 100). If both fail, the local Llama 3.2 instance handles the request — but only for requests that do not require tools or multimodal inputs, since those capability flags are false for the fallback.

Embedding Models

ScalyClaw's memory system stores every saved memory entry alongside a high-dimensional vector embedding. When the orchestrator retrieves relevant context before an LLM call, it runs a cosine-similarity search using sqlite-vec against those stored vectors. The accuracy of that search depends entirely on the quality of the embedding model you choose.

How Embeddings Are Generated

When a memory entry is saved — either automatically by the orchestrator or explicitly via the save_memory tool — ScalyClaw calls the configured embedding model to convert the text into a float32 vector. That vector is stored in the SQLite database alongside the entry. At retrieval time, the query text is embedded on the fly using the same model, and sqlite-vec finds the nearest stored vectors by cosine distance.

Recommended Models

Embedding Configuration

Embedding models live in config.models.embeddingModels, the same config section as chat models but in a separate array. You can use a different provider for embeddings than for chat — for example, use Anthropic for chat but OpenAI's cheaper embedding API for memory. Each entry shares the same providers map as chat models.

{
  "models": {
    "providers": {
      "openai": { "apiKey": "sk-..." }
    },
    "models": [ /* ... chat models ... */ ],
    "embeddingModels": [
      {
        "id": "openai-embed",
        "name": "text-embedding-3-small",
        "provider": "openai",
        "enabled": true,
        "priority": 1,
        "weight": 100,
        "dimensions": 1536,
        "inputPricePerMillion": 0.02,
        "outputPricePerMillion": 0
      }
    ]
  }
}

For a fully local setup with Ollama, add the local provider to the providers map and point the embedding model at it:

{
  "models": {
    "providers": {
      "local": { "baseUrl": "http://localhost:11434/v1" }
    },
    "models": [ /* ... chat models ... */ ],
    "embeddingModels": [
      {
        "id": "local-embed",
        "name": "nomic-embed-text",
        "provider": "local",
        "enabled": true,
        "priority": 1,
        "weight": 100,
        "dimensions": 768,
        "inputPricePerMillion": 0,
        "outputPricePerMillion": 0
      }
    ]
  }
}

Budget Control

LLM API calls cost money. ScalyClaw tracks token usage per model and per day, accumulates spending estimates based on the inputPricePerMillion and outputPricePerMillion values you configure on each model, and enforces configurable global daily and monthly limits. Budget is a single global config block — there are no per-model budget caps. You can choose between hard enforcement (block all calls when the limit is reached) or soft enforcement (warn but continue).

Enforcement Modes

Per-Model Cost Tracking

Every LLM call records the number of input tokens, output tokens, and the estimated cost in USD using the pricing table ScalyClaw maintains for each known model. Costs are stored in Redis and aggregated by day and by month. The dashboard usage page displays:

• Daily and monthly spend broken down by model
• Token usage histograms per model per day
• Budget consumption as a percentage of configured limits
• A list of the most expensive individual requests

For models with custom or unknown pricing (e.g., local models or new provider releases), set inputPricePerMillion and outputPricePerMillion directly on the model entry in config.models.models. ScalyClaw uses those figures for all cost tracking and budget accounting for that model. Set both to 0 for free local models.

Budget Configuration

{
  "budget": {
    "monthlyLimit": 150,
    "dailyLimit": 10,
    "hardLimit": true,
    "alertThresholds": [50, 80, 90]
  }
}

Budget is a single global block — there are no per-model caps. The fields are:

• monthlyLimit — maximum USD spend per calendar month. Set to 0 for unlimited.
• dailyLimit — maximum USD spend per day (resets at midnight UTC). Set to 0 for unlimited.
• hardLimit — when true, all LLM calls are blocked once a limit is reached. When false, the system continues but emits warnings.
• alertThresholds — array of percentage values (e.g. [50, 80, 90]). A warning is emitted to dashboard logs and any configured alert channels each time cumulative spend crosses one of these thresholds — giving you advance notice before a hard stop occurs.

Custom Pricing

Set inputPricePerMillion and outputPricePerMillion directly on any model entry. ScalyClaw uses those values for all cost tracking for that model:

{
  "id": "my-azure-deployment",
  "name": "my-gpt4o-deployment",
  "provider": "azure",
  "enabled": true,
  "priority": 1,
  "weight": 100,
  "temperature": 0.7,
  "maxTokens": 4096,
  "contextWindow": 128000,
  "toolEnabled": true,
  "imageEnabled": true,
  "audioEnabled": false,
  "videoEnabled": false,
  "documentEnabled": false,
  "reasoningEnabled": false,
  "inputPricePerMillion": 2.50,
  "outputPricePerMillion": 10.00
}