API Usage

Start runs with a bearer API key, track progress, and download signed results.

1) Start a Run (Bearer API Key)

Send your Chutes API key in the Authorization header. The run will execute using that key, not the system key.

If the service is in maintenance mode, run creation returns HTTP 503 with the maintenance message.

Request fields

model_id: bench runner model UUID, Chutes chute_id, or the model slug/name (e.g. zai-org/GLM-4.7-TEE).
subset_pct: integer 1–100 (common values: 1, 5, 10, 25, 50, 100). Minimum sample size is 1 item.
subset_count: optional fixed item count (takes precedence over subset_pct). Recommended for Affine environments where 1% can be large.
subset_seed: optional deterministic seed. Use the same seed across runs to align samples between models.
selected_benchmarks: optional list of benchmark names (use /api/benchmarks to discover valid values).
provider: optional inference provider. Use chutes (default), rlm, gremium-openai, or gremium-anthropic. For RLM runs, use rlm-gpt-4o orrlm-claude-3-5-sonnet as the model. For Gremium runs, use gremium-consensus(OpenAI) or gremium-consensus-anthropicas the model.

curl -X POST https://chutes-bench-runner-api-v2.onrender.com/api/runs/api \
  -H "Authorization: Bearer <CHUTES_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model_id": "<model-uuid | chute_id | model-slug>",
    "subset_pct": 1,
    "subset_count": 25,
    "subset_seed": "affine-seed-001",
    "selected_benchmarks": ["mmlu_pro", "ifbench"],
    "provider": "chutes"
  }'

2) Track Progress

Use the run ID returned by the create call.

# Poll run status
curl https://chutes-bench-runner-api-v2.onrender.com/api/runs/<run-id>

# Stream events (SSE)
curl -N https://chutes-bench-runner-api-v2.onrender.com/api/runs/<run-id>/events

3) Cancel a Run

Cancel a queued or running run.

curl -X POST https://chutes-bench-runner-api-v2.onrender.com/api/runs/<run-id>/cancel

4) Download Results

Exports are available after the run completes (succeeded or failed).

# CSV
curl -O https://chutes-bench-runner-api-v2.onrender.com/api/runs/<run-id>/export?format=csv

# PDF
curl -O https://chutes-bench-runner-api-v2.onrender.com/api/runs/<run-id>/export?format=pdf

# Signed ZIP (JSON + signature)
curl -O https://chutes-bench-runner-api-v2.onrender.com/api/runs/<run-id>/export?format=zip

5) Verify a Signed ZIP

Upload a signed zip to confirm it was produced by the official bench runner.

curl -X POST https://chutes-bench-runner-api-v2.onrender.com/api/exports/verify \
  -F "file=@benchmark_results.zip"

6) List Available Models

Get a list of all models available on Chutes. Use any model ID or slug from this list with the bench runner or Artificial Analysis endpoints.

# List all available Chutes models (OpenAI-compatible)
curl https://llm.chutes.ai/v1/models

# Example: extract model IDs
curl -s https://llm.chutes.ai/v1/models | jq '.data[].id'

Each model object includes: id (model slug), chute_id, pricing, context length, and supported features.

7) Artificial Analysis Benchmarks

Resolve a Chutes model to its Artificial Analysis entry and fetch benchmark scores. The endpoint accepts a bench runner model UUID, chute_id, model slug, or model name.

# Get AA benchmarks for a model
curl "https://chutes-bench-runner-api-v2.onrender.com/api/benchmarks/artificial-analysis?model_id=deepseek-ai/DeepSeek-V3"

# Using chute_id
curl "https://chutes-bench-runner-api-v2.onrender.com/api/benchmarks/artificial-analysis?model_id=0d7184a2-32a3-53e0-9607-058c37edaab5"

# Optional flags
# include_raw=true -> include full AA payload
# llm_fallback=false -> disable LLM-based fallback mapping
curl "https://chutes-bench-runner-api-v2.onrender.com/api/benchmarks/artificial-analysis?model_id=<model>&include_raw=true&llm_fallback=false"

Returns benchmark scores like coding_index, math_index, intelligence_index, GPQA, LiveCodeBench, etc. The service automatically maps Chutes model names to their Artificial Analysis equivalents.

8) Public Key

Fetch the public key for offline verification.

curl https://chutes-bench-runner-api-v2.onrender.com/api/exports/public-key