Models and providers

After this page every model dropdown in the project — judge, agent model, playground, generation — offers the models you chose, from a credential stored once and never shown again, and you can rotate or disconnect a vendor knowing exactly what depends on it.

Two levels

A Provider is an LLM vendor credential on the organization (Settings in the sidebar footer → Organization → Providers): stored encrypted, validated with a real call when saved, never returned to the browser, shared by every project. A Model is a project-level pointer (Project settings → Models) to one provider model — provider, model id, display name, default parameters, modalities — that the dropdowns read. Connect once, pick per project. The app calls the credential a Provider and never an “API key”.

Connect a Provider

1. 1

   Pick the vendor

   The picker lists the catalogue’s providers (frontier, cloud, aggregator, self-hosted, voice and media vendors); its required and optional fields drive the form, and a multi-mode provider such as Bedrock shows the fields of the chosen auth mode.

2. 2

   Fill the credential

   A label (“Prod key”), the credential fields in the vendor’s own names (api_key, api_base, aws_access_key_id, vertex_credentials …) and a scope: the whole organization (default) or one project, which then outranks the organization’s connection for that project.

3. 3

   Test & save

   The platform makes a genuine minimal call and stores the connection as valid with the models it discovered, or refuses a bad credential at the form. Skip validation exists for servers the platform cannot reach (a self-hosted Ollama behind your firewall) and stores it as untested.

A Provider card shows its status (valid · invalid · untested), when it was validated, and used by n Models. Test re-probes it; Rotate credentials replaces the whole credential map (a half-rotated credential fails in a way that looks like an outage, so partial updates are not offered); Disconnect soft-deletes it after listing the Models that depend on it — they fail at call time with “re-select a connection” until re-pointed. An invalid Provider shows as a banner on the dependent Model dropdowns and as judge_error rows in runs.

Add Models to a project

Project settings → Models → Add models: choose a Provider, then tick models from the catalogue list (featured first, with modality badges — text · vision · image generation · tts · stt — and prices per million tokens); retired models are refused. Modality decides where a Model appears: text Models in judge, agent and chat pickers, image Models on the Image tab, tts and stt on Voice. Test sends a prompt through the Model and shows the reply and latency. Every member of the project sees every Model. Set default judge stores the Model New run and Online rules pre-select. Any empty Model picker in the app offers this path and returns you where you were.

Remove a Model

The Models table shows used by — runs, online rules, agents, playground grids. Removing a Model that anything references is blocked (409) with the list, because an online rule whose judge disappears would fail at provider-call time with nothing to explain it; Replace with re-points the online rules first, or Remove anyway forces it. Removal is a soft delete: Restore undoes it right after, past runs keep the Model’s name in their history, and pickers stop offering it.