Model comparison
All models compared
Pick a family, filter by licence, sort the columns — plus the performance profile of the families and price-performance per model.
Ask Polaris about AI models
Answers from the benchmarks — with sources
Choose a model family
Performance profile of the model families
One star per family across 6 categories (0–100), all overlaid; up to three families in colour. Hover a line or the legend to see the values, click to highlight a family.
Values as a table
| Family | Models | Code | CRM | Docs | Integrate | Marketing | Reason |
|---|---|---|---|---|---|---|---|
| GPT | 34 | 84 | 91 | 89 | 86 | 85 | 65 |
| Claude | 26 | 79 | 86 | 87 | 83 | 75 | 59 |
| Gemini | 24 | 79 | 94 | 90 | 93 | 79 | 68 |
| Gemma | 10 | 51 | 56 | 63 | 59 | 68 | 34 |
| Qwen | 25 | 79 | 88 | 89 | 81 | 79 | 66 |
| Grok | 8 | 89 | 97 | 92 | 94 | 83 | 78 |
| DeepSeek | 14 | 91 | 91 | 94 | 79 | 84 | 77 |
| Llama | 21 | 57 | 60 | 60 | 68 | 69 | 34 |
| Mistral | 26 | 45 | 58 | 59 | 72 | 69 | 26 |
| GLM | 5 | 82 | 96 | 93 | 81 | 81 | 68 |
| Kimi | 2 | 88 | 93 | 91 | 76 | 71 | 72 |
| MiniMax | 2 | 83 | 89 | 92 | 76 | 79 | 55 |
| Nemotron | 3 | 64 | 83 | 73 | 58 | 92 | 54 |
| Ø all models | 71 | 79 | 79 | 78 | 76 | 54 |
Ø over all models of a family in this edition — older generations and small variants pull the average down. “Others” is not a family and therefore has no star.
Model comparison & price-performance
Pick two models — their values side by side, both highlighted in the chart.
| Model | Code | CRM | Docs | Integrate | Marketing | Reason | Final | Cost | Speed |
|---|---|---|---|---|---|---|---|---|---|
| GPT o1 pro (manual) | 100 | 100 | 97 | 100 | 95 | 87 | 97 | – | – |
| Qwen3.7 Max | 100 | 100 | 97 | 100 | 88 | 87 | 95 | €2.94 | 0.1 |
X = estimated cost of the benchmark workload (logarithmic), Y = final score (0–100). Efficiency frontier: no other model in the selection is both cheaper and better. Not a recommendation. Not shown for lack of a value: GPT o1 pro (manual). Click a point: that model becomes model B.
225 of 225 models · Click a column header to sort
1 GPT o1 pro (manual) GPTProprietary (cloud) | 100 | 100 | 97 | 100 | 95 | 87 | 97 | – | – |
|---|---|---|---|---|---|---|---|---|---|
2 GPT-5.6 Sol GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 100 | 97 | 100 | 88 | 90 | 96 | €2.06 | 0.56 rps |
3 Qwen3.7 Max QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 100 | 100 | 97 | 100 | 88 | 87 | 95 | €2.94 | 0.10 rps |
4 Google Gemini 3.7 Flash (reasoning high) GeminiProprietary (cloud) | 100 | 100 | 90 | 100 | 88 | 92 | 95 | €1.68 | 0.27 rps |
5 Fugu Ultra OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 100 | 100 | 94 | 98 | 88 | 90 | 95 | €67.46 | 0.04 rps |
6 GPT-5.5 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 100 | 94 | 100 | 88 | 87 | 95 | €3.59 | 0.43 rps |
7 Google Gemini 3.7 Flash (reasoning medium) GeminiProprietary (cloud) | 100 | 94 | 97 | 100 | 88 | 90 | 95 | €1.10 | 0.32 rps |
8 GPT-5.5 Instant GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 97 | 94 | 100 | 88 | 90 | 95 | €2.39 | 0.67 rps |
9 GPT-5.5 Pro GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 100 | 93 | 100 | 88 | 87 | 95 | €37.10 | 0.09 rps |
10 GPT-5.4 Pro GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 97 | 93 | 100 | 88 | 90 | 94 | €34.69 | 0.06 rps |
11 Google Gemini 3.8 Flash (reasoning high) GeminiProprietary (cloud) | 100 | 100 | 90 | 100 | 88 | 90 | 94 | €2.27 | 0.19 rps |
12 Grok 4.5 GrokLicence not assignedResult outdated or incomplete (flagged by the source) | 100 | 100 | 94 | 98 | 82 | 92 | 94 | €4.20 | 0.07 rps |
13 Grok 4.6 (reasoning medium) GrokLicence not assigned | 100 | 100 | 94 | 98 | 82 | 92 | 94 | €4.69 | 0.12 rps |
14 Claude Fable 5.1 (reasoning max) ClaudeProprietary (cloud) | 100 | 100 | 90 | 100 | 88 | 87 | 94 | €11.44 | 0.16 rps |
15 Google Gemini 3.8 Flash (reasoning medium) GeminiProprietary (cloud) | 100 | 94 | 93 | 100 | 88 | 90 | 94 | €1.28 | 0.27 rps |
16 Grok 4.6 (reasoning high) GrokLicence not assigned | 100 | 100 | 94 | 98 | 82 | 90 | 94 | €5.16 | 0.10 rps |
17 Google Gemini 3.7 Flash (reasoning low) GeminiProprietary (cloud) | 100 | 94 | 90 | 100 | 88 | 92 | 94 | €0.60 | 0.40 rps |
18 GPT-5.3 Chat Latest GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 97 | 93 | 98 | 88 | 85 | 93 | €1.93 | 0.74 rps |
19 Grok 4.6 (reasoning low) GrokLicence not assigned | 100 | 100 | 94 | 97 | 82 | 85 | 93 | €3.06 | 0.18 rps |
20 Google Gemini 3.8 Flash (reasoning low) GeminiProprietary (cloud) | 100 | 86 | 93 | 100 | 88 | 90 | 93 | €0.50 | 0.38 rps |
21 Claude Opus 5 (reasoning low) ClaudeProprietary (cloud) | 100 | 100 | 90 | 100 | 78 | 87 | 92 | €2.80 | 0.36 rps |
22 Claude Opus 5 (reasoning medium) ClaudeProprietary (cloud) | 100 | 100 | 94 | 98 | 75 | 87 | 92 | €3.19 | 0.34 rps |
23 Qwen3.8 2.4T A95B (reasoning xhigh) QwenOpen source | 100 | 93 | 94 | 86 | 88 | 92 | 92 | €3.00 | 0.24 rps |
24 GPT o1-preview v1/2024-09-12 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 95 | 92 | 94 | 95 | 88 | 87 | 92 | €52.32 | 0.08 rps |
25 Grok 4.6 (reasoning xhigh) GrokLicence not assigned | 100 | 100 | 94 | 90 | 82 | 85 | 92 | €5.04 | 0.10 rps |
26 Qwen3.8 27B (reasoning xhigh) QwenOpen source | 100 | 100 | 94 | 86 | 82 | 88 | 92 | €1.42 | 0.10 rps |
27 Qwen3.8 2.4T A95B (reasoning medium) QwenOpen source | 98 | 92 | 94 | 91 | 88 | 87 | 92 | €1.98 | 0.40 rps |
28 Claude Opus 5 (reasoning high) ClaudeProprietary (cloud) | 100 | 100 | 92 | 98 | 74 | 86 | 92 | €3.59 | 0.36 rps |
29 Google Gemini 3.6 Flash GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 95 | 91 | 93 | 100 | 88 | 82 | 91 | €0.39 | 1.21 rps |
30 Qwen3.8 Max (reasoning medium) QwenLicence not assigned | 97 | 92 | 90 | 96 | 88 | 87 | 91 | €2.00 | 0.23 rps |
31 GPT-5.6 Terra GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 89 | 97 | 90 | 98 | 88 | 86 | 91 | €1.18 | 0.61 rps |
32 Qwen3.8 2.4T A95B (reasoning low) QwenOpen source | 98 | 96 | 90 | 98 | 81 | 83 | 91 | €1.66 | 0.49 rps |
33 GPT o1 v1/2024-12-17 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 95 | 94 | 91 | 82 | 83 | 91 | €30.63 | 0.17 rps |
34 Qwen3.8 Max (reasoning minimal) QwenLicence not assigned | 100 | 96 | 90 | 92 | 81 | 86 | 91 | €1.67 | 0.28 rps |
35 Qwen3.8 Max (reasoning high) QwenLicence not assigned | 100 | 93 | 93 | 79 | 88 | 92 | 91 | €3.18 | 0.12 rps |
36 Claude Opus 4.7 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 97 | 97 | 98 | 82 | 70 | 91 | €2.05 | 0.43 rps |
37 DeepSeek V4 Flash 0731 (reasoning high) DeepSeekLicence not assigned | 100 | 96 | 100 | 77 | 82 | 87 | 90 | €0.12 | 0.11 rps |
38 DeepSeek V4 Flash Vision Exp (reasoning low) DeepSeekLicence not assigned | 93 | 91 | 97 | 92 | 82 | 87 | 90 | €0.34 | 0.31 rps |
39 Claude Fable 5 (reasoning max) ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 87 | 100 | 90 | 100 | 78 | 87 | 90 | €17.24 | 0.20 rps |
40 Qwen3.8 Max (reasoning xhigh) QwenLicence not assigned | 100 | 93 | 100 | 79 | 82 | 87 | 90 | €3.00 | 0.13 rps |
41 DeepSeek V4 Flash 0731 (reasoning max) DeepSeekLicence not assigned | 100 | 96 | 100 | 75 | 82 | 87 | 90 | €0.14 | 0.14 rps |
42 GPT-5.6 Luna GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 100 | 90 | 97 | 100 | 88 | 66 | 90 | €0.56 | 0.51 rps |
43 Qwen3.8 Max (reasoning low) QwenLicence not assigned | 98 | 92 | 90 | 94 | 79 | 87 | 90 | €1.75 | 0.26 rps |
44 Qwen3.8 27B (reasoning medium) QwenOpen source | 98 | 100 | 92 | 73 | 88 | 88 | 90 | €0.93 | 0.13 rps |
45 Google Gemini 3.1 Pro Preview GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 96 | 93 | 98 | 82 | 78 | 90 | €0.54 | 0.44 rps |
46 GLM 5.2 GLMOpen sourceResult outdated or incomplete (flagged by the source) | 84 | 96 | 96 | 98 | 88 | 77 | 90 | €0.34 | 0.14 rps |
47 GPT o1-mini v1/2024-09-12 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 93 | 96 | 94 | 83 | 82 | 87 | 89 | €8.15 | 0.16 rps |
48 GPT-5.4 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 86 | 97 | 100 | 100 | 82 | 71 | 89 | €0.74 | 0.81 rps |
49 DeepSeek V4 Flash Vision Exp (reasoning high) DeepSeekLicence not assigned | 100 | 96 | 92 | 78 | 82 | 87 | 89 | €0.38 | 0.26 rps |
50 DeepSeek V4 Flash 0731 (reasoning low) DeepSeekLicence not assigned | 100 | 96 | 97 | 75 | 82 | 85 | 89 | €0.17 | 0.06 rps |
51 DeepSeek V4 Pro 0813 (reasoning low) DeepSeekLicence not assigned | 95 | 95 | 97 | 77 | 82 | 87 | 89 | €1.44 | 0.15 rps |
52 GPT-4o v3/2024-11-20 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 86 | 97 | 94 | 95 | 88 | 72 | 89 | €0.63 | 1.14 rps |
53 Qwen3.8 27B (reasoning low) QwenOpen source | 97 | 96 | 90 | 73 | 88 | 88 | 88 | €0.77 | 0.17 rps |
54 DeepSeek V4 Pro DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source) | 100 | 86 | 97 | 77 | 82 | 88 | 88 | €0.35 | 0.20 rps |
55 GPT-4o v1/2024-05-13 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 96 | 100 | 92 | 78 | 74 | 88 | €1.21 | 1.44 rps |
56 DeepSeek V4 Pro 0813 (reasoning high) DeepSeekLicence not assigned | 95 | 91 | 94 | 77 | 82 | 90 | 88 | €1.78 | 0.12 rps |
57 Qwen3.7 Plus QwenLicence not assigned | 90 | 97 | 100 | 83 | 82 | 77 | 88 | €0.09 | 0.78 rps |
58 DeepSeek V4 Flash Vision Exp (reasoning max) DeepSeekLicence not assigned | 100 | 92 | 97 | 74 | 95 | 71 | 88 | €1.64 | 0.12 rps |
59 OpenAI GPT-OSS 120B GPTProprietary (cloud) | 100 | 89 | 93 | 94 | 70 | 83 | 88 | €0.02 | 1.35 rps |
60 Kimi K3 KimiLicence not assignedResult outdated or incomplete (flagged by the source) | 90 | 100 | 92 | 98 | 66 | 82 | 88 | €4.05 | 0.05 rps |
61 Claude Opus 4.8 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 82 | 100 | 93 | 98 | 82 | 73 | 88 | €1.92 | 0.62 rps |
62 Google Gemini 1.5 Pro v2 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 86 | 97 | 94 | 99 | 78 | 74 | 88 | €1.00 | 1.18 rps |
63 DeepSeek V4 Flash (reasoning high) DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source) | 100 | 89 | 89 | 77 | 88 | 83 | 88 | €0.09 | 0.27 rps |
64 X-AI Grok 2 v2/1212 GrokLicence not assignedResult outdated or incomplete (flagged by the source) | 66 | 95 | 97 | 97 | 88 | 78 | 87 | €0.58 | 0.99 rps |
65 Google Gemini 3 Flash Preview GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 91 | 90 | 100 | 88 | 60 | 86 | €0.13 | 0.51 rps |
66 Claude Sonnet 5 (reasoning low) ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 95 | 89 | 100 | 75 | 88 | 69 | 86 | €1.39 | 0.46 rps |
67 Claude Sonnet 5 (reasoning xhigh) ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 97 | 96 | 100 | 91 | 76 | 55 | 86 | €2.90 | 0.35 rps |
68 Mistral Medium 3.5 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 89 | 100 | 86 | 88 | 59 | 85 | €0.44 | 0.37 rps |
69 NVIDIA Nemotron 3.5 Lightning NemotronOpen source | 93 | 88 | 86 | 75 | 88 | 80 | 85 | €0.28 | 0.13 rps |
70 GPT-4 Turbo v5/2024-04-09 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 86 | 99 | 98 | 96 | 88 | 43 | 85 | €2.45 | 0.84 rps |
71 Claude Sonnet 5 (reasoning high) ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 82 | 93 | 90 | 82 | 69 | 84 | €2.70 | 0.36 rps |
72 Meta Muse Glimmer 30B (reasoning low) OthersOpen source | 100 | 93 | 90 | 73 | 82 | 68 | 84 | €0.28 | 0.44 rps |
73 Meta Muse Glimmer 30B (reasoning high) OthersOpen source | 100 | 93 | 86 | 71 | 80 | 75 | 84 | €0.69 | 0.25 rps |
74 GLM 5.3 Flash (reasoning high) GLMOpen source | 77 | 94 | 93 | 77 | 88 | 75 | 84 | €0.03 | 0.38 rps |
75 Google Gemini 2.0 Flash Exp GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 96 | 100 | 100 | 82 | 62 | 84 | €0.03 | 0.85 rps |
76 Google Gemini Exp 1121 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 70 | 97 | 97 | 95 | 72 | 72 | 84 | €0.89 | 0.49 rps |
77 Google Gemini 3.5 Flash-Lite GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 84 | 92 | 97 | 94 | 82 | 54 | 84 | €0.09 | 1.59 rps |
78 Google Gemini 3.5 Flash GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 86 | 83 | 84 | 100 | 88 | 60 | 83 | €0.41 | 0.63 rps |
79 Google Gemini 3.1 Flash-Lite GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 77 | 92 | 93 | 86 | 82 | 68 | 83 | €0.07 | 0.59 rps |
80 Qwen3.6 27B QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 86 | 97 | 100 | 73 | 82 | 61 | 83 | €0.11 | 0.82 rps |
81 GPT-4o v2/2024-08-06 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 84 | 97 | 86 | 82 | 59 | 83 | €0.63 | 1.49 rps |
82 Google Gemini 1.5 Pro 0801 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 84 | 92 | 79 | 100 | 70 | 74 | 83 | €0.90 | 0.83 rps |
83 GPT-4.1 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 86 | 83 | 97 | 79 | 82 | 69 | 83 | €0.50 | 0.84 rps |
84 Gemma 4 31B IT GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 86 | 91 | 90 | 87 | 88 | 55 | 83 | €0.03 | 0.27 rps |
85 Claude Fable 5 (reasoning max, after) ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 58 | 100 | 90 | 75 | 82 | 90 | 83 | €15.16 | 0.17 rps |
86 Meta Muse Glimmer 30B (reasoning medium) OthersOpen source | 100 | 93 | 86 | 71 | 81 | 62 | 82 | €0.48 | 0.35 rps |
87 Claude Sonnet 5 (reasoning medium) ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 89 | 79 | 94 | 75 | 88 | 69 | 82 | €2.14 | 0.38 rps |
88 GLM 5.3 Flash (reasoning low) GLMOpen source | 87 | 93 | 92 | 79 | 69 | 73 | 82 | €0.02 | 0.43 rps |
89 Qwen 2.5 72B Instruct QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 79 | 92 | 94 | 97 | 71 | 59 | 82 | €0.10 | 0.66 rps |
90 GLM 5.1 GLMOpen sourceResult outdated or incomplete (flagged by the source) | 77 | 97 | 97 | 75 | 82 | 63 | 82 | €0.23 | 1.68 rps |
91 Gemma 4 26B A4B IT GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 86 | 94 | 100 | 73 | 82 | 56 | 82 | €0.02 | 0.55 rps |
92 DeepSeek V4 Flash Vision Exp (reasoning none) DeepSeekLicence not assigned | 86 | 89 | 90 | 80 | 88 | 59 | 82 | €0.05 | 0.86 rps |
93 Nous Llama 3.1 405B Hermes 3 LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 68 | 93 | 89 | 98 | 88 | 53 | 81 | €0.54 | 0.49 rps |
94 MiniMax M2.7 MiniMaxLicence not assignedResult outdated or incomplete (flagged by the source) | 97 | 87 | 89 | 75 | 69 | 69 | 81 | €0.45 | 0.61 rps |
95 Claude 3.5 Sonnet v2 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 82 | 97 | 93 | 84 | 71 | 57 | 81 | €0.95 | 0.09 rps |
96 GLM 5.3 Flash (reasoning max) GLMOpen source | 87 | 100 | 89 | 77 | 76 | 54 | 81 | €0.07 | 0.24 rps |
97 GPT-4 v1/0314 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 88 | 98 | 73 | 88 | 45 | 80 | €7.04 | 1.31 rps |
98 GPT-5.4 Mini GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 94 | 81 | 73 | 82 | 59 | 80 | €0.22 | 1.02 rps |
99 Poolside Laguna S 2.1 OthersOpen sourceResult outdated or incomplete (flagged by the source) | 63 | 93 | 100 | 89 | 95 | 40 | 80 | €0.03 | 1.39 rps |
100 X-AI Grok 2 v1/1012 GrokLicence not assignedResult outdated or incomplete (flagged by the source) | 63 | 93 | 87 | 90 | 88 | 58 | 80 | €1.03 | 0.31 rps |
101 GPT-4 v2/0613 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 83 | 95 | 73 | 88 | 45 | 79 | €7.04 | 2.16 rps |
102 Tencent Hy3 OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 84 | 94 | 97 | 73 | 70 | 55 | 79 | €0.05 | 0.11 rps |
103 Qwen3.6 35B A3B QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 66 | 89 | 95 | 84 | 82 | 55 | 78 | €0.05 | 0.57 rps |
104 DeepSeek v3 671B DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source) | 62 | 95 | 97 | 85 | 75 | 55 | 78 | €0.03 | 0.49 rps |
105 GPT-4o Mini GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 87 | 80 | 73 | 100 | 65 | 78 | €0.04 | 1.46 rps |
106 Claude 3.5 Sonnet v1 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 72 | 83 | 89 | 87 | 80 | 58 | 78 | €0.94 | 0.09 rps |
107 GPT-4.1 Mini GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 66 | 97 | 97 | 75 | 82 | 51 | 78 | €0.10 | 0.79 rps |
108 Nous Llama 3.1 70B Hermes 3 LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 74 | 97 | 87 | 82 | 88 | 39 | 78 | €0.03 | 0.42 rps |
109 Claude Opus 4.6 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 95 | 100 | 100 | 81 | 38 | 52 | 78 | €1.59 | 0.40 rps |
110 DeepSeek V4 Flash 0731 (reasoning none) DeepSeekLicence not assigned | 84 | 84 | 81 | 75 | 82 | 58 | 78 | €0.03 | 0.05 rps |
111 Qwen3.7 Flash QwenLicence not assigned | 59 | 89 | 96 | 84 | 82 | 55 | 77 | €0.01 | 0.62 rps |
112 Qwen3.6 Flash QwenLicence not assigned | 59 | 89 | 96 | 84 | 82 | 55 | 77 | €0.06 | 0.91 rps |
113 Claude 3 Opus ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 69 | 88 | 100 | 74 | 76 | 58 | 77 | €4.69 | 0.41 rps |
114 OpenAI GPT-OSS 20B GPTProprietary (cloud) | 87 | 86 | 75 | 73 | 82 | 61 | 77 | €0.02 | 1.35 rps |
115 Meta Llama 3.1 405B Instruct LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 81 | 93 | 92 | 75 | 75 | 48 | 77 | €2.39 | 1.16 rps |
116 GPT-4 Turbo v4/0125-preview GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 66 | 97 | 100 | 83 | 75 | 43 | 77 | €2.45 | 0.84 rps |
117 Google LearnLM 1.5 Pro Experimental GeminiLicence not assignedResult outdated or incomplete (flagged by the source) | 48 | 97 | 85 | 96 | 64 | 72 | 77 | €0.31 | 0.83 rps |
118 GPT-4 Turbo v3/1106-preview GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 66 | 75 | 98 | 73 | 88 | 60 | 76 | €2.46 | 0.68 rps |
119 MiniMax M3 MiniMaxLicence not assignedResult outdated or incomplete (flagged by the source) | 68 | 91 | 94 | 77 | 88 | 41 | 76 | €0.14 | 0.39 rps |
120 Google Gemini Exp 1206 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 52 | 100 | 85 | 77 | 75 | 69 | 76 | €0.88 | 0.16 rps |
121 Qwen 2.5 32B Coder Instruct QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 43 | 94 | 98 | 98 | 76 | 46 | 76 | €0.05 | 0.82 rps |
122 Grok 4.3 GrokLicence not assignedResult outdated or incomplete (flagged by the source) | 84 | 90 | 82 | 80 | 75 | 43 | 76 | €1.69 | 0.17 rps |
123 Kimi K2.6 KimiLicence not assignedResult outdated or incomplete (flagged by the source) | 86 | 86 | 90 | 54 | 76 | 61 | 76 | €0.21 | 0.94 rps |
124 DeepSeek v2.5 236B DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source) | 57 | 80 | 91 | 80 | 88 | 57 | 75 | €0.03 | 0.42 rps |
125 Meta Llama 3.1 70B Instruct f16 LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 74 | 89 | 90 | 75 | 75 | 48 | 75 | €1.79 | 0.90 rps |
126 Google Gemini 1.5 Flash v2 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 64 | 96 | 89 | 76 | 81 | 44 | 75 | €0.06 | 2.01 rps |
127 Claude Opus 4.5 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 70 | 88 | 92 | 77 | 82 | 40 | 75 | €1.56 | 0.43 rps |
128 Google Gemini 1.5 Pro 0409 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 68 | 97 | 96 | 80 | 75 | 26 | 74 | €0.95 | 0.59 rps |
129 Meta Llama 3 70B Instruct LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 81 | 83 | 84 | 67 | 81 | 45 | 73 | €0.06 | 0.85 rps |
130 GPT-3.5 v2/0613 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 68 | 81 | 73 | 87 | 81 | 50 | 73 | €0.34 | 1.46 rps |
131 Amazon Nova Lite OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 67 | 78 | 74 | 94 | 62 | 62 | 73 | €0.02 | 2.19 rps |
132 Claude Sonnet 4.5 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 72 | 91 | 90 | 73 | 75 | 34 | 72 | €0.97 | 0.42 rps |
133 Mistral Large 123B v2/2407 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 68 | 79 | 68 | 75 | 75 | 70 | 72 | €0.57 | 1.02 rps |
134 Google Gemini Flash 1.5 8B GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 70 | 93 | 78 | 67 | 76 | 48 | 72 | €0.01 | 1.19 rps |
135 Google Gemini 1.5 Pro 0514 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 73 | 96 | 79 | 100 | 25 | 60 | 72 | €1.07 | 0.92 rps |
136 Google Gemini 1.5 Flash 0514 GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 32 | 97 | 100 | 76 | 72 | 52 | 72 | €0.06 | 1.77 rps |
137 Google Gemini 1.0 Pro GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source) | 66 | 86 | 83 | 79 | 88 | 28 | 71 | €0.37 | 1.36 rps |
138 Meta Llama 3.2 90B Vision LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 74 | 84 | 87 | 77 | 71 | 32 | 71 | €0.23 | 1.10 rps |
139 GPT-3.5 v3/1106 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 68 | 70 | 71 | 81 | 78 | 58 | 71 | €0.24 | 2.33 rps |
140 Claude 3.5 Haiku ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 52 | 80 | 72 | 75 | 75 | 68 | 70 | €0.32 | 1.24 rps |
141 Claude Sonnet 4.6 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 90 | 88 | 90 | 75 | 38 | 42 | 70 | €0.95 | 0.59 rps |
142 Meta Llama 3.3 70B Instruct LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 74 | 78 | 74 | 77 | 71 | 46 | 70 | €0.10 | 0.71 rps |
143 Claude Haiku 4.5 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 80 | 91 | 73 | 75 | 37 | 70 | €0.32 | 0.76 rps |
144 GPT-3.5 v4/0125 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 87 | 71 | 77 | 78 | 43 | 70 | €0.12 | 1.43 rps |
145 Cohere Command R+ OthersProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 80 | 76 | 72 | 70 | 58 | 70 | €0.83 | 1.90 rps |
146 Mistral Large 123B v3/2411 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 68 | 75 | 64 | 76 | 82 | 51 | 70 | €0.56 | 0.66 rps |
147 Qwen1.5 32B Chat f16 QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 70 | 90 | 82 | 76 | 78 | 20 | 69 | €0.97 | 1.66 rps |
148 Gemma 2 27B IT GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 61 | 72 | 87 | 74 | 89 | 32 | 69 | €0.07 | 0.90 rps |
149 GPT-4.1 Nano GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 68 | 84 | 77 | 66 | 78 | 41 | 69 | €0.03 | 0.79 rps |
150 Mistral 7B OpenChat-3.5 v3 0106 f16 MistralOpen sourceResult outdated or incomplete (flagged by the source) | 68 | 87 | 67 | 74 | 88 | 25 | 68 | €0.32 | 3.39 rps |
151 Meta Llama 3 8B Instruct f16 LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 79 | 62 | 68 | 70 | 80 | 41 | 67 | €0.32 | 3.33 rps |
152 Gemma 7B OpenChat-3.5 v3 0106 f16 GemmaOpen sourceResult outdated or incomplete (flagged by the source) | 63 | 67 | 84 | 58 | 81 | 46 | 67 | €0.21 | 5.09 rps |
153 NVIDIA Nemotron 3 Super 120B A12B NemotronLicence not assignedResult outdated or incomplete (flagged by the source) | 48 | 84 | 69 | 68 | 88 | 42 | 67 | €0.03 | 0.97 rps |
154 GPT-5.4 Nano GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 59 | 90 | 76 | 71 | 82 | 21 | 67 | €0.06 | 1.06 rps |
155 GPT-3.5-instruct 0914 GPTProprietary (cloud) | 47 | 92 | 69 | 69 | 88 | 33 | 66 | €0.35 | 2.15 rps |
156 Amazon Nova Pro OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 64 | 78 | 82 | 79 | 52 | 41 | 66 | €0.22 | 1.34 rps |
157 GPT-3.5 v1/0301 GPTProprietary (cloud)Result outdated or incomplete (flagged by the source) | 55 | 82 | 69 | 81 | 82 | 26 | 66 | €0.35 | 4.12 rps |
158 Llama 3 8B OpenChat-3.6 20240522 f16 LlamaOpen sourceResult outdated or incomplete (flagged by the source) | 76 | 51 | 76 | 65 | 88 | 38 | 66 | €0.28 | 3.79 rps |
159 Mistral 7B OpenChat-3.5 v1 f16 MistralOpen source | 58 | 72 | 72 | 71 | 88 | 33 | 66 | €0.49 | 2.20 rps |
160 Mistral 7B OpenChat-3.5 v2 1210 f16 MistralOpen sourceResult outdated or incomplete (flagged by the source) | 63 | 73 | 72 | 66 | 88 | 30 | 65 | €0.32 | 3.40 rps |
161 Qwen 2.5 7B Instruct QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 48 | 77 | 80 | 68 | 69 | 47 | 65 | €0.13 | 0.63 rps |
162 Starling 7B-alpha f16 OthersLicence not assigned | 58 | 66 | 67 | 73 | 88 | 34 | 64 | €0.58 | 1.85 rps |
163 Mistral Nemo 12B v1/2407 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 54 | 58 | 51 | 99 | 75 | 49 | 64 | €0.03 | 1.22 rps |
164 Meta Llama 3.2 11B Vision LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 70 | 71 | 65 | 70 | 71 | 36 | 64 | €0.04 | 1.49 rps |
165 Nous Llama 3 8B Hermes 2 Theta LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 61 | 73 | 74 | 74 | 85 | 16 | 64 | €0.05 | 0.55 rps |
166 Claude 3 Haiku ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 64 | 69 | 64 | 75 | 75 | 35 | 64 | €0.08 | 0.52 rps |
167 Yi 1.5 34B Chat f16 OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 47 | 78 | 70 | 74 | 86 | 26 | 64 | €1.18 | 1.37 rps |
168 Gemma 3n E4B IT GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 32 | 73 | 73 | 74 | 71 | 54 | 63 | €0.02 | 0.44 rps |
169 Liquid: LFM 40B MoE OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 72 | 69 | 65 | 63 | 82 | 24 | 63 | €0.00 | 1.45 rps |
170 Meta Llama 3.1 8B Instruct f16 LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 57 | 74 | 62 | 74 | 74 | 32 | 62 | €0.45 | 2.41 rps |
171 Gemma 3 4B IT GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 48 | 84 | 60 | 72 | 69 | 37 | 62 | €0.01 | 0.54 rps |
172 Qwen2 7B Instruct f32 QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 50 | 81 | 81 | 61 | 66 | 31 | 62 | €0.46 | 2.36 rps |
173 Claude 3 Sonnet ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 72 | 41 | 74 | 74 | 78 | 28 | 61 | €0.95 | 0.85 rps |
174 MistralAI Ministral 3B 2512 MistralOpen sourceResult outdated or incomplete (flagged by the source) | 53 | 59 | 69 | 71 | 81 | 32 | 61 | €0.02 | 0.54 rps |
175 Mistral Small v3/2409 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 43 | 75 | 71 | 74 | 75 | 26 | 61 | €0.06 | 0.81 rps |
176 NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning NemotronLicence not assignedResult outdated or incomplete (flagged by the source) | 50 | 77 | 64 | 32 | 100 | 41 | 61 | €0.00 | 1.06 rps |
177 Mistral Pixtral 12B MistralOpen sourceResult outdated or incomplete (flagged by the source) | 53 | 69 | 73 | 63 | 64 | 40 | 60 | €0.03 | 0.83 rps |
178 Mixtral 8x22B API (Instruct) MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 53 | 62 | 62 | 97 | 75 | 7 | 59 | €0.17 | 3.12 rps |
179 Anthropic Claude Instant v1.2 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 58 | 75 | 65 | 77 | 65 | 16 | 59 | €2.10 | 1.49 rps |
180 Codestral Mamba 7B v1 MistralOpen sourceResult outdated or incomplete (flagged by the source) | 53 | 66 | 51 | 97 | 71 | 17 | 59 | €0.30 | 2.82 rps |
181 Inflection 3 Productivity OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 46 | 59 | 39 | 70 | 79 | 61 | 59 | €0.92 | 0.17 rps |
182 Anthropic Claude v2.0 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 52 | 55 | 67 | 84 | 34 | 59 | €2.19 | 0.40 rps |
183 Cohere Command R OthersProprietary (cloud)Result outdated or incomplete (flagged by the source) | 45 | 66 | 57 | 74 | 84 | 27 | 59 | €0.13 | 2.50 rps |
184 Amazon Nova Micro OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 58 | 68 | 64 | 71 | 59 | 31 | 59 | €0.01 | 2.41 rps |
185 Qwen1.5 7B Chat f16 QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 56 | 81 | 60 | 56 | 60 | 36 | 58 | €0.29 | 3.76 rps |
186 Mistral Large v1/2402 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 37 | 49 | 70 | 83 | 84 | 25 | 58 | €0.58 | 2.11 rps |
187 Microsoft WizardLM 2 8x22B OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 48 | 76 | 79 | 59 | 62 | 22 | 58 | €0.13 | 0.70 rps |
188 Qwen1.5 14B Chat f16 QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 50 | 58 | 51 | 72 | 84 | 22 | 56 | €0.36 | 3.03 rps |
189 MistralAI Ministral 8B MistralOpen sourceResult outdated or incomplete (flagged by the source) | 56 | 55 | 41 | 82 | 68 | 30 | 55 | €0.02 | 1.02 rps |
190 Anthropic Claude v2.1 ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source) | 29 | 58 | 59 | 78 | 75 | 32 | 55 | €2.25 | 0.35 rps |
191 Mistral 7B OpenOrca f16 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 54 | 57 | 76 | 36 | 78 | 27 | 55 | €0.41 | 2.65 rps |
192 MistralAI Ministral 3B MistralOpen sourceResult outdated or incomplete (flagged by the source) | 50 | 48 | 39 | 89 | 60 | 41 | 54 | €0.01 | 1.02 rps |
193 Cohere Command R 7B 12/2024 OthersProprietary (cloud)Result outdated or incomplete (flagged by the source) | 48 | 68 | 63 | 65 | 55 | 25 | 54 | €0.01 | 1.45 rps |
194 Llama2 13B Vicuna-1.5 f16 LlamaLlama licence | 50 | 37 | 55 | 62 | 82 | 37 | 54 | €0.99 | 1.09 rps |
195 Mistral 7B Instruct v0.1 f16 MistralProprietary (cloud) | 34 | 71 | 69 | 63 | 62 | 23 | 54 | €0.75 | 1.43 rps |
196 Meta Llama 3.2 3B LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 52 | 71 | 66 | 71 | 44 | 14 | 53 | €0.01 | 1.25 rps |
197 Google Recurrent Gemma 9B IT f16 GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 58 | 27 | 71 | 64 | 56 | 23 | 50 | €0.89 | 1.21 rps |
198 Codestral 22B v1 MistralOpen sourceResult outdated or incomplete (flagged by the source) | 38 | 47 | 44 | 84 | 66 | 13 | 49 | €0.06 | 4.03 rps |
199 Qwen: QwQ 32B Preview QwenLicence not assignedResult outdated or incomplete (flagged by the source) | 43 | 32 | 74 | 52 | 48 | 40 | 48 | €0.05 | 0.63 rps |
200 Nous Llama2 13B Hermes f16 LlamaLlama licence | 50 | 24 | 37 | 75 | 60 | 42 | 48 | €1.00 | 1.07 rps |
201 IBM Granite 34B Code Instruct f16 OthersProprietary (cloud)Result outdated or incomplete (flagged by the source) | 63 | 49 | 34 | 67 | 57 | 7 | 46 | €1.07 | 1.51 rps |
202 Meta Llama 3.2 1B LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 32 | 40 | 33 | 53 | 68 | 51 | 46 | €0.02 | 1.69 rps |
203 Mistral Small v2/2402 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 33 | 42 | 45 | 88 | 56 | 8 | 46 | €0.06 | 3.21 rps |
204 Mistral Small v1/2312 (Mixtral) MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 10 | 67 | 63 | 65 | 56 | 8 | 45 | €0.06 | 2.21 rps |
205 DBRX 132B Instruct OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 43 | 39 | 43 | 74 | 59 | 10 | 45 | €0.26 | 1.31 rps |
206 NVIDIA Llama 3.1 Nemotron 70B Instruct LlamaLlama licenceResult outdated or incomplete (flagged by the source) | 68 | 54 | 25 | 72 | 28 | 21 | 45 | €0.09 | 0.53 rps |
207 Mistral Medium v1/2312 MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 41 | 43 | 44 | 59 | 62 | 12 | 44 | €0.81 | 0.35 rps |
208 Microsoft WizardLM 2 7B OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 53 | 34 | 42 | 66 | 53 | 13 | 43 | €0.02 | 0.89 rps |
209 Llama2 13B Puffin f16 LlamaLlama licence | 37 | 15 | 44 | 67 | 56 | 39 | 43 | €4.70 | 0.23 rps |
210 Mistral Tiny v1/2312 (7B Instruct v0.2) MistralProprietary (cloud)Result outdated or incomplete (flagged by the source) | 22 | 47 | 59 | 53 | 62 | 8 | 42 | €0.05 | 2.39 rps |
211 Gemma 2 9B IT GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 45 | 25 | 47 | 36 | 68 | 13 | 39 | €0.02 | 0.88 rps |
212 Meta Llama2 13B chat f16 LlamaLlama licence | 22 | 38 | 17 | 65 | 75 | 6 | 37 | €0.75 | 1.44 rps |
213 Mistral 7B Zephyr-β f16 MistralOpen sourceResult outdated or incomplete (flagged by the source) | 37 | 34 | 46 | 62 | 29 | 4 | 35 | €0.46 | 2.34 rps |
214 Meta Llama2 7B chat f16 LlamaLlama licence | 22 | 33 | 20 | 62 | 50 | 18 | 34 | €0.56 | 1.93 rps |
215 Mistral 7B Notus-v1 f16 MistralLicence not assignedResult outdated or incomplete (flagged by the source) | 10 | 54 | 25 | 60 | 48 | 4 | 33 | €0.75 | 1.43 rps |
216 Orca 2 13B f16 OthersLicence not assigned | 18 | 22 | 32 | 29 | 67 | 20 | 31 | €0.95 | 1.14 rps |
217 Mistral 7B Instruct v0.2 f16 MistralProprietary (cloud) | 11 | 30 | 54 | 25 | 58 | 8 | 31 | €0.96 | 1.12 rps |
218 Mistral 7B v0.1 f16 MistralProprietary (cloud) | 0 | 9 | 48 | 63 | 52 | 12 | 31 | €0.87 | 1.23 rps |
219 Google Gemma 2B IT f16 GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 33 | 28 | 16 | 47 | 15 | 20 | 27 | €0.30 | 3.54 rps |
220 Microsoft Phi 3 Medium 4K Instruct 14B f16 OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 5 | 34 | 30 | 32 | 47 | 8 | 26 | €0.82 | 1.32 rps |
221 Orca 2 7B f16 OthersLicence not assigned | 22 | 0 | 26 | 26 | 52 | 4 | 22 | €0.78 | 1.38 rps |
222 Microsoft Phi 4 Mini Instruct OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 10 | 2 | 12 | 25 | 22 | 4 | 13 | €0.02 | 0.58 rps |
223 Google Gemma 7B IT f16 GemmaLicence not assignedResult outdated or incomplete (flagged by the source) | 0 | 0 | 0 | 6 | 62 | 0 | 11 | €0.99 | 1.08 rps |
224 Meta Llama2 7B f16 LlamaLlama licence | 0 | 5 | 22 | 3 | 28 | 2 | 10 | €0.95 | 1.13 rps |
225 Yi 1.5 9B Chat f16 OthersLicence not assignedResult outdated or incomplete (flagged by the source) | 0 | 4 | 29 | 17 | 0 | 8 | 10 | €1.41 | 0.76 rps |
Scores 0–100 per category, higher is better; the shading is relative to the best result in each column. Cost and speed do not count towards the score. Result outdated or incomplete (flagged by the source)
The benchmark categories in detail
The TIMETOACT GROUP does not test the models on academic quiz questions but on tasks taken from real products. This is what each category checks:
- Code
- Can the model generate and understand code and help with programming?
- CRM
- How well does the model support work with product catalogues and marketplaces?
- Docs
- How well does the model work with large documents and knowledge bases?
- Integrate
- Can the model interact reliably with external APIs, services and plugins?
- Marketing
- How well does the model support marketing — brainstorming, ideation and copywriting?
- Reason
- How well does the model draw logical conclusions within a given context?
- Final
- Overall result across all categories — the basis of the ranking.
- Cost
- Estimated cost of running the benchmark workload (for local models incl. GPU hardware). Not part of the score.
- Speed
- Estimated requests per second — higher is faster. Not part of the score.
Data source: LLM Benchmarks by the TIMETOACT GROUP
All values come from the LLM Benchmarks of the TIMETOACT GROUP. Every month the team tests new and established language models on real use cases from client projects and publishes the results as a leaderboard. comparison.cloud takes the values over unchanged; ranking, licence labels and flags for outdated results follow the source.
New editions are checked daily and picked up automatically. The data is kept as one YAML file per model, with the edition cited as the source of every value.
Data as of: September 2026 edition, imported on 02/11/2026
Settled on a model?The AI Compass findsthe platform to run it.
Starts with what the AI should do (use cases, models, operating mode, scale) and under which rules (data class, regulation, EU AI Act, processing location, data-protection commitments). A language model scores 14 providers on their documented AI catalogue, data-protection commitments and verified evidence; the engine weights and ranks — result by email, with sources.
1.Describe the project
What the AI should do — use cases, models, operating mode, scale — and under which rules: data class, EU AI Act, processing location.
2.Measure the providers
14 cloud providers on their documented AI catalogue, data-protection commitments and verified evidence.
3.Take the recommendation along
Fit per provider with reasons — by email, with all sources.
