LLM providers: servers, config and model detection

The LLM pass is optional. Without it, reviews use the built-in static analyzers and coding tasks use mechanical fixes only. No model id is hard-coded: the server endpoint comes from configuration, and the model is detected from that server unless you pin one.

Where the LLM server is specified

Two configuration layers, repository first:

Repository config (tools.llm object) names the server per repo:

tools.llm: true also works and means "use the server environment". API keys are rejected in repository config on purpose.

Server environment (the operator default):

``text CJR_LLM_PROVIDER=lmstudio CJR_LLM_BASE_URL=http://127.0.0.1:1234/v1 CJR_LLM_MODEL= # empty or "auto" = detect CJR_LLM_API_KEY= # only for keyed compatible gateways OPENAI_API_KEY / ANTHROPIC_API_KEY / GOOGLE_API_KEY (or GEMINI_API_KEY) OLLAMA_HOST=http://127.0.0.1:11434 ``

Keys live only in the server environment. The CLI never sees them.

Model detection

With model: auto (the default), the server asks the configured endpoint for its model list and picks the best current model:

Provider	Detection endpoint	Pick rule
`openai`	`GET /v1/models`	newest GPT chat model, mini preferred at equal version
`anthropic`	`GET /v1/models`	newest Sonnet, then Opus, then Haiku
`google`	`GET /v1beta/models`	newest Gemini Pro, then Flash, `generateContent` only
`ollama`	`GET /api/tags`	first installed model
`vllm`	`GET /v1/models`	coder models first, then largest
`llamacpp`	`GET /v1/models`	the loaded model
`lmstudio`	`GET /v1/models`	coder models first, then largest
`openai-compatible`	`GET /v1/models`	coder models first, then largest

Embedding, audio, image and vision specialties are skipped. The picked model is cached per provider+base URL for the process lifetime. Pin an exact id with model: / CJR_LLM_MODEL when you want determinism or a small fast model on a machine that also hosts very large ones.

Default base URLs when baseUrl is omitted: OpenAI https://api.openai.com/v1, Anthropic https://api.anthropic.com/v1, Google https://generativelanguage.googleapis.com/v1beta, Ollama http://127.0.0.1:11434, vLLM http://127.0.0.1:8000/v1, llama.cpp http://127.0.0.1:8080/v1, LM Studio http://127.0.0.1:1234/v1. openai-compatible has no default; baseUrl is required.

Provider aliases accepted in config: gemini (google), llama.cpp / llama-cpp (llamacpp), lm-studio (lmstudio).

Local servers

Start the server first, then point the config at it:

Ollama: ollama serve, pull a model with ollama pull <model>.
LM Studio: lms server start (port 1234). Downloaded models are served and loaded on demand.
llama.cpp: llama-server -m <model.gguf> --port 8080.
vLLM: vllm serve <model> (port 8000).

Inside Docker, a model server on the host is reached through http://host.docker.internal:<port>; set CJR_LLM_BASE_URL accordingly.

Behavior and limits

Reviews: the LLM pass runs only when the repository opts in (tools.llm). Its findings merge with static findings.
Coding tasks: a configured provider can generate patches. Every generated patch must pass git apply --check before it is accepted.
Failures are reported in the result note (provider, HTTP status or discovery error). A failed LLM pass never fabricates findings and never blocks the static review.
Timeouts: CJR_LLM_DISCOVERY_TIMEOUT_MS (default 5000) and CJR_LLM_TIMEOUT_MS (default 120000). Large just-in-time loaded local models can need a higher chat timeout on first use.