Overview

This documentation explains how to define custom language model metadata via an environment variable (LANGUAGE_MODEL_INFOS) instead of configuring full model definitions manually in the UI. The environment variable is stored on assistants-core service.

Previously, custom models had to be fully defined directly in the UI each time they were used, which was error-prone and cumbersome. With this update, language model information can be centrally defined once via an environment variable and then referenced in the UI using only the language model enum name.

Step-by-Step Guide

Step 1: Understand the Current Limitation (UI-based Definition)

Until now, custom language models had to be defined directly in the UI.

This required manually entering the full language model configuration every time, including (not complete):

Model name
Provider
Version
Capabilities
Token limits

Step 2: Define Language Model Info via Environment Variable

You can now define custom language model metadata once using the LANGUAGE_MODEL_INFOS environment variable. The environment variable is on the assistants-core service. See here for different parameters.

Example

LANGUAGE_MODEL_INFOS='{
  "CUSTOM_MODEL": {
    "name": "CUSTOM_ENUM",
    "provider": "CUSTOM",
    "version": "CUSTOM",
    "capabilities": ["function_calling", "streaming", "vision"],
    "token_limits": {
      "token_limit_input": 30000,
      "token_limit_output": 1500
    }
  }
}

Each top-level key (e.g. CUSTOM_MODEL) represents a language model enum name that can later be selected in the UI.

Step 3: Add the model to allowed models for Unique AI

You can add the model to the list of allowed models for UniqueAI by adding it to UNIQUE_ALLOWED_MODELS in node-chat, using the name defined previously:

Example

UNIQUE_ALLOWED_MODELS== *:CUSSTOM_ENUM

Afterwards a restart of node-chatservice is needed.

Step 4: Use the Language Model Enum in the UI

Once the environment variable is set:

The defined models are automatically loaded by the system
They behave exactly like predefined models
In the UI, you only need to select or reference the language model enum name

Author	@Fabian Schläpfer