AI providers¶

An AI provider is one endpoint mr-review sends the diff to. Providers are created in the app under Settings → AI providers, not in the environment: there is no AI_PROVIDER, AI_API_KEY or AI_MODEL variable. The form calls POST /api/v1/ai-providers, and the provider is stored in ai_providers.yaml in the data directory, API key included.

Types¶

Three types, two backends:

openai and openai_compat share a client but not their defaults: openai knows OpenAI's models (which take temperature, which reasoning levels) and turns structured output on, while openai_compat assumes nothing about the server behind it.

Fields¶

The settings form covers all of these except max_concurrent, which is set through the API or by editing ai_providers.yaml.

For a corporate CA, install it in the system trust store rather than turning ssl_verify off — the HTTP client reads the system store, not a bundled one.

Pointing at an endpoint¶

Before this version base_url was ignored for claude, and the add form could keep the URL of a type it was switched away from. Settings now marks a Claude provider whose base URL is not Anthropic's with a warning — clear the field unless it is a gateway you meant to use.

localhost inside the container is the container, not your machine — a local model reached from a container needs host.docker.internal (Docker Desktop) or the host's LAN address.

Choosing a model¶

Fetch models from API in the provider form asks the endpoint which models it has, using the values in the form as they are — a key or base URL you have typed but not saved yet included (POST /api/v1/ai-providers/preview/models; a blank key keeps the saved one, but only for the saved base URL and type — a changed endpoint needs the key typed in again, so the saved key is never sent anywhere else). The answer is offered next to the list, to add one model at a time or all at once; it never replaces the list, so a hand-picked list survives an endpoint with hundreds of models. GET /api/v1/ai-providers/{id}/models does the same with the saved settings.

The first model in the list is the default — Make default moves another one to the top. A dispatch can also name any model id, listed or not. There is no built-in fallback: a dispatch that names no model, to a provider with an empty list, is refused with 422.

When the endpoint refuses, the answer says why: 401 when it rejects the key, 504 when it does not answer within timeout, 502 for anything else — an unreachable host, a wrong URL, an error from the endpoint — with the endpoint's own message.

For code review, models with large context windows do best. The diff, pinned lines, commit message and review brief all go into one prompt — large MRs can exceed 20k tokens, and the optional context toggles add more.

Dispatch settings¶

Each dispatch can tune the model call; every setting left empty means "the default".

Models differ in what they accept, and a setting a model rejects would fail the whole request with a 400. So each dispatch is fitted to its model before it is sent, and the app shows only the controls the selected model takes. GET /api/v1/ai-providers/{id}/capabilities?model=… returns what that is, decided from the model id alone — no call to the provider:

Claude ids are recognised in any spelling — first-party, Bedrock (us.anthropic.claude-opus-4-5-20251101-v1:0), Vertex (claude-opus-4-5@20251101) or a gateway alias. An id the table does not know, including a newer version of a known family, is treated as a current-generation model.

How a dispatch is fitted to its model:

• Reasoning. On current Claude models it is adaptive thinking with output_config.effort; budget_tokens goes only to the models before 4.6, which accept nothing else. An effort the model does not have becomes the nearest level below it, a budget becomes an effort on an effort-only model and the other way round, and a model that does not reason gets neither. OpenAI gets reasoning_effort; an openai_compat endpoint gets reasoning_effort or, for a budget, a reasoning_budget field for the servers that read one.
• Temperature is sent only to models that take it and only while reasoning is off — thinking models reject it — and is capped at the model's range (1 on Claude).
• Max output tokens is the limit for the whole answer, thinking included, and is capped at the model's maximum. A thinking budget always leaves at least 4,096 tokens for the answer below it, or thinking is turned off. It goes out as max_tokens to Claude, as max_completion_tokens to OpenAI (which requires it for reasoning models) and as max_tokens to openai_compat endpoints — the one name every compatible server understands.

Structured output¶

With structured output on, the answer is constrained to a JSON schema — {"comments": [{"file", "line", "severity", "body"}]}, file and line nullable, severity one of critical, major, minor, suggestion — through output_config.format on Claude and response_format: json_schema (strict) on OpenAI. The model can then only answer with parseable comments, and they still stream in as they are written. The built-in system prompt describes the object instead of a bare array while it is on.

Servers differ, so it is off by default for openai_compat, for OpenAI ids the table does not know, and for a claude or openai provider whose base_url is not the vendor's own API (a gateway, or another vendor such as DeepSeek or Groq behind the openai type). A refusal — Claude's stop_reason: refusal, or OpenAI's refusal field under strict structured output — ends the dispatch with an error carrying the model's message. When an endpoint rejects it, the dispatch fails with the endpoint's message and a hint to turn Structured output off — it is not silently retried without.

How an answer can end¶

• Truncated. The provider stopped at the output limit (Claude max_tokens, or the context window; OpenAI finish_reason: length). The answer counts as cut off even when its text happens to parse, so it never replaces existing comments; on an iteration without comments the ones completed before the cut are saved. The run is marked truncated — raise Max output tokens or narrow the context.
• Refused. Claude declined to answer (stop_reason: refusal, with its category when it gives one), OpenAI returned a refusal under structured output, or an endpoint's content filter stopped the answer. The dispatch ends with an error saying so and is settled like any failed run: existing comments stay as they were.

Concurrency¶

Each provider has a cap on how many dispatches to it can be in flight at once: max_concurrent, falling back to MR_REVIEW__AI_THROTTLE__DEFAULT_MAX_CONCURRENT (4). It exists to keep a local model from being overwhelmed and to put a coarse ceiling on cloud usage.

max_concurrent has no field in the settings form: set it with POST/PATCH /api/v1/ai-providers, or edit ai_providers.yaml directly.

The cap is fixed at first use either way. The semaphore is created on the first dispatch to a provider and kept for the life of the process, so a changed max_concurrent takes effect on the next restart, not on the next dispatch.