LLM Benchmarks · TIMETOACT GROUP

    LLM Benchmarks: GPT, Claude, Qwen & co. capability by capability.

    Which language model is good for documents, integration or code? The monthly TIMETOACT GROUP LLM benchmarks — filterable by model family, with cost and speed.

    Docs & knowledgeCode & integrationCost & speed
    Source: TIMETOACT GROUP LLM Benchmarks, September 2026

    Top 5 · final score

    September 2026 · 225 models

    1. 1

      GPT o1 pro (manual)

      GPT

      97
    2. 2

      GPT-5.6 Sol

      GPT

      96
    3. 3

      Qwen3.7 Max

      Qwen

      95
    4. 4

      Google Gemini 3.7 Flash (reasoning high)

      Gemini

      95
    5. 5

      Fugu Ultra

      95

    Source: LLM Benchmarks by the TIMETOACT GROUP, scores 0–100.

    Model comparison

    All models compared

    Pick a family, filter by licence, sort the columns — plus the performance profile of the families and price-performance per model.

    Ask Polaris about AI models

    Answers from the benchmarks — with sources

    Choose a model family

    Licence

    Performance profile of the model families

    One star per family across 6 categories (0–100), all overlaid; up to three families in colour. Hover a line or the legend to see the values, click to highlight a family.

    CodeCRMDocsIntegrateMarketingReason
    Ø all models
    Values as a table
    FamilyModelsCodeCRMDocsIntegrateMarketingReason
    GPT34849189868565
    Claude26798687837559
    Gemini24799490937968
    Gemma10515663596834
    Qwen25798889817966
    Grok8899792948378
    DeepSeek14919194798477
    Llama21576060686934
    Mistral26455859726926
    GLM5829693818168
    Kimi2889391767172
    MiniMax2838992767955
    Nemotron3648373589254
    Ø all models717979787654

    Ø over all models of a family in this edition — older generations and small variants pull the average down. “Others” is not a family and therefore has no star.

    Model comparison & price-performance

    Pick two models — their values side by side, both highlighted in the chart.

    Values of model A and model B
    ModelCodeCRMDocsIntegrateMarketingReasonFinalCostSpeed
    GPT o1 pro (manual)10010097100958797––
    Qwen3.7 Max10010097100888795€2.940.1
    €0.01€0.10€1.00€10.00€100.00020406080100GPT-5.6 Sol — €2.06 · Final score 96Google Gemini 3.7 Flash (reasoning high) — €1.68 · Final score 95Fugu Ultra — €67.46 · Final score 95GPT-5.5 — €3.59 · Final score 95Google Gemini 3.7 Flash (reasoning medium) — €1.10 · Final score 95GPT-5.5 Instant — €2.39 · Final score 95GPT-5.5 Pro — €37.10 · Final score 95GPT-5.4 Pro — €34.69 · Final score 94Google Gemini 3.8 Flash (reasoning high) — €2.27 · Final score 94Grok 4.5 — €4.20 · Final score 94Grok 4.6 (reasoning medium) — €4.69 · Final score 94Claude Fable 5.1 (reasoning max) — €11.44 · Final score 94Google Gemini 3.8 Flash (reasoning medium) — €1.28 · Final score 94Grok 4.6 (reasoning high) — €5.16 · Final score 94Google Gemini 3.7 Flash (reasoning low) — €0.60 · Final score 94GPT-5.3 Chat Latest — €1.93 · Final score 93Grok 4.6 (reasoning low) — €3.06 · Final score 93Google Gemini 3.8 Flash (reasoning low) — €0.50 · Final score 93Claude Opus 5 (reasoning low) — €2.80 · Final score 92Claude Opus 5 (reasoning medium) — €3.19 · Final score 92Qwen3.8 2.4T A95B (reasoning xhigh) — €3.00 · Final score 92GPT o1-preview v1/2024-09-12 — €52.32 · Final score 92Grok 4.6 (reasoning xhigh) — €5.04 · Final score 92Qwen3.8 27B (reasoning xhigh) — €1.42 · Final score 92Qwen3.8 2.4T A95B (reasoning medium) — €1.98 · Final score 92Claude Opus 5 (reasoning high) — €3.59 · Final score 92Google Gemini 3.6 Flash — €0.39 · Final score 91Qwen3.8 Max (reasoning medium) — €2.00 · Final score 91GPT-5.6 Terra — €1.18 · Final score 91Qwen3.8 2.4T A95B (reasoning low) — €1.66 · Final score 91GPT o1 v1/2024-12-17 — €30.63 · Final score 91Qwen3.8 Max (reasoning minimal) — €1.67 · Final score 91Qwen3.8 Max (reasoning high) — €3.18 · Final score 91Claude Opus 4.7 — €2.05 · Final score 91DeepSeek V4 Flash 0731 (reasoning high) — €0.12 · Final score 90DeepSeek V4 Flash Vision Exp (reasoning low) — €0.34 · Final score 90Claude Fable 5 (reasoning max) — €17.24 · Final score 90Qwen3.8 Max (reasoning xhigh) — €3.00 · Final score 90DeepSeek V4 Flash 0731 (reasoning max) — €0.14 · Final score 90GPT-5.6 Luna — €0.56 · Final score 90Qwen3.8 Max (reasoning low) — €1.75 · Final score 90Qwen3.8 27B (reasoning medium) — €0.93 · Final score 90Google Gemini 3.1 Pro Preview — €0.54 · Final score 90GLM 5.2 — €0.34 · Final score 90GPT o1-mini v1/2024-09-12 — €8.15 · Final score 89GPT-5.4 — €0.74 · Final score 89DeepSeek V4 Flash Vision Exp (reasoning high) — €0.38 · Final score 89DeepSeek V4 Flash 0731 (reasoning low) — €0.17 · Final score 89DeepSeek V4 Pro 0813 (reasoning low) — €1.44 · Final score 89GPT-4o v3/2024-11-20 — €0.63 · Final score 89Qwen3.8 27B (reasoning low) — €0.77 · Final score 88DeepSeek V4 Pro — €0.35 · Final score 88GPT-4o v1/2024-05-13 — €1.21 · Final score 88DeepSeek V4 Pro 0813 (reasoning high) — €1.78 · Final score 88Qwen3.7 Plus — €0.09 · Final score 88DeepSeek V4 Flash Vision Exp (reasoning max) — €1.64 · Final score 88OpenAI GPT-OSS 120B — €0.02 · Final score 88Kimi K3 — €4.05 · Final score 88Claude Opus 4.8 — €1.92 · Final score 88Google Gemini 1.5 Pro v2 — €1.00 · Final score 88DeepSeek V4 Flash (reasoning high) — €0.09 · Final score 88X-AI Grok 2 v2/1212 — €0.58 · Final score 87Google Gemini 3 Flash Preview — €0.13 · Final score 86Claude Sonnet 5 (reasoning low) — €1.39 · Final score 86Claude Sonnet 5 (reasoning xhigh) — €2.90 · Final score 86Mistral Medium 3.5 — €0.44 · Final score 85NVIDIA Nemotron 3.5 Lightning — €0.28 · Final score 85GPT-4 Turbo v5/2024-04-09 — €2.45 · Final score 85Claude Sonnet 5 (reasoning high) — €2.70 · Final score 84Meta Muse Glimmer 30B (reasoning low) — €0.28 · Final score 84Meta Muse Glimmer 30B (reasoning high) — €0.69 · Final score 84GLM 5.3 Flash (reasoning high) — €0.03 · Final score 84Google Gemini 2.0 Flash Exp — €0.03 · Final score 84Google Gemini Exp 1121 — €0.89 · Final score 84Google Gemini 3.5 Flash-Lite — €0.09 · Final score 84Google Gemini 3.5 Flash — €0.41 · Final score 83Google Gemini 3.1 Flash-Lite — €0.07 · Final score 83Qwen3.6 27B — €0.11 · Final score 83GPT-4o v2/2024-08-06 — €0.63 · Final score 83Google Gemini 1.5 Pro 0801 — €0.90 · Final score 83GPT-4.1 — €0.50 · Final score 83Gemma 4 31B IT — €0.03 · Final score 83Claude Fable 5 (reasoning max, after) — €15.16 · Final score 83Meta Muse Glimmer 30B (reasoning medium) — €0.48 · Final score 82Claude Sonnet 5 (reasoning medium) — €2.14 · Final score 82GLM 5.3 Flash (reasoning low) — €0.02 · Final score 82Qwen 2.5 72B Instruct — €0.10 · Final score 82GLM 5.1 — €0.23 · Final score 82Gemma 4 26B A4B IT — €0.02 · Final score 82DeepSeek V4 Flash Vision Exp (reasoning none) — €0.05 · Final score 82Nous Llama 3.1 405B Hermes 3 — €0.54 · Final score 81MiniMax M2.7 — €0.45 · Final score 81Claude 3.5 Sonnet v2 — €0.95 · Final score 81GLM 5.3 Flash (reasoning max) — €0.07 · Final score 81GPT-4 v1/0314 — €7.04 · Final score 80GPT-5.4 Mini — €0.22 · Final score 80Poolside Laguna S 2.1 — €0.03 · Final score 80X-AI Grok 2 v1/1012 — €1.03 · Final score 80GPT-4 v2/0613 — €7.04 · Final score 79Tencent Hy3 — €0.05 · Final score 79Qwen3.6 35B A3B — €0.05 · Final score 78DeepSeek v3 671B — €0.03 · Final score 78GPT-4o Mini — €0.04 · Final score 78Claude 3.5 Sonnet v1 — €0.94 · Final score 78GPT-4.1 Mini — €0.10 · Final score 78Nous Llama 3.1 70B Hermes 3 — €0.03 · Final score 78Claude Opus 4.6 — €1.59 · Final score 78DeepSeek V4 Flash 0731 (reasoning none) — €0.03 · Final score 78Qwen3.7 Flash — €0.01 · Final score 77Qwen3.6 Flash — €0.06 · Final score 77Claude 3 Opus — €4.69 · Final score 77OpenAI GPT-OSS 20B — €0.02 · Final score 77Meta Llama 3.1 405B Instruct — €2.39 · Final score 77GPT-4 Turbo v4/0125-preview — €2.45 · Final score 77Google LearnLM 1.5 Pro Experimental — €0.31 · Final score 77GPT-4 Turbo v3/1106-preview — €2.46 · Final score 76MiniMax M3 — €0.14 · Final score 76Google Gemini Exp 1206 — €0.88 · Final score 76Qwen 2.5 32B Coder Instruct — €0.05 · Final score 76Grok 4.3 — €1.69 · Final score 76Kimi K2.6 — €0.21 · Final score 76DeepSeek v2.5 236B — €0.03 · Final score 75Meta Llama 3.1 70B Instruct f16 — €1.79 · Final score 75Google Gemini 1.5 Flash v2 — €0.06 · Final score 75Claude Opus 4.5 — €1.56 · Final score 75Google Gemini 1.5 Pro 0409 — €0.95 · Final score 74Meta Llama 3 70B Instruct — €0.06 · Final score 73GPT-3.5 v2/0613 — €0.34 · Final score 73Amazon Nova Lite — €0.02 · Final score 73Claude Sonnet 4.5 — €0.97 · Final score 72Mistral Large 123B v2/2407 — €0.57 · Final score 72Google Gemini Flash 1.5 8B — €0.01 · Final score 72Google Gemini 1.5 Pro 0514 — €1.07 · Final score 72Google Gemini 1.5 Flash 0514 — €0.06 · Final score 72Google Gemini 1.0 Pro — €0.37 · Final score 71Meta Llama 3.2 90B Vision — €0.23 · Final score 71GPT-3.5 v3/1106 — €0.24 · Final score 71Claude 3.5 Haiku — €0.32 · Final score 70Claude Sonnet 4.6 — €0.95 · Final score 70Meta Llama 3.3 70B Instruct — €0.10 · Final score 70Claude Haiku 4.5 — €0.32 · Final score 70GPT-3.5 v4/0125 — €0.12 · Final score 70Cohere Command R+ — €0.83 · Final score 70Mistral Large 123B v3/2411 — €0.56 · Final score 70Qwen1.5 32B Chat f16 — €0.97 · Final score 69Gemma 2 27B IT — €0.07 · Final score 69GPT-4.1 Nano — €0.03 · Final score 69Mistral 7B OpenChat-3.5 v3 0106 f16 — €0.32 · Final score 68Meta Llama 3 8B Instruct f16 — €0.32 · Final score 67Gemma 7B OpenChat-3.5 v3 0106 f16 — €0.21 · Final score 67NVIDIA Nemotron 3 Super 120B A12B — €0.03 · Final score 67GPT-5.4 Nano — €0.06 · Final score 67GPT-3.5-instruct 0914 — €0.35 · Final score 66Amazon Nova Pro — €0.22 · Final score 66GPT-3.5 v1/0301 — €0.35 · Final score 66Llama 3 8B OpenChat-3.6 20240522 f16 — €0.28 · Final score 66Mistral 7B OpenChat-3.5 v1 f16 — €0.49 · Final score 66Mistral 7B OpenChat-3.5 v2 1210 f16 — €0.32 · Final score 65Qwen 2.5 7B Instruct — €0.13 · Final score 65Starling 7B-alpha f16 — €0.58 · Final score 64Mistral Nemo 12B v1/2407 — €0.03 · Final score 64Meta Llama 3.2 11B Vision — €0.04 · Final score 64Nous Llama 3 8B Hermes 2 Theta — €0.05 · Final score 64Claude 3 Haiku — €0.08 · Final score 64Yi 1.5 34B Chat f16 — €1.18 · Final score 64Gemma 3n E4B IT — €0.02 · Final score 63Liquid: LFM 40B MoE — €0.00 · Final score 63Meta Llama 3.1 8B Instruct f16 — €0.45 · Final score 62Gemma 3 4B IT — €0.01 · Final score 62Qwen2 7B Instruct f32 — €0.46 · Final score 62Claude 3 Sonnet — €0.95 · Final score 61MistralAI Ministral 3B 2512 — €0.02 · Final score 61Mistral Small v3/2409 — €0.06 · Final score 61NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning — €0.00 · Final score 61Mistral Pixtral 12B — €0.03 · Final score 60Mixtral 8x22B API (Instruct) — €0.17 · Final score 59Anthropic Claude Instant v1.2 — €2.10 · Final score 59Codestral Mamba 7B v1 — €0.30 · Final score 59Inflection 3 Productivity — €0.92 · Final score 59Anthropic Claude v2.0 — €2.19 · Final score 59Cohere Command R — €0.13 · Final score 59Amazon Nova Micro — €0.01 · Final score 59Qwen1.5 7B Chat f16 — €0.29 · Final score 58Mistral Large v1/2402 — €0.58 · Final score 58Microsoft WizardLM 2 8x22B — €0.13 · Final score 58Qwen1.5 14B Chat f16 — €0.36 · Final score 56MistralAI Ministral 8B — €0.02 · Final score 55Anthropic Claude v2.1 — €2.25 · Final score 55Mistral 7B OpenOrca f16 — €0.41 · Final score 55MistralAI Ministral 3B — €0.01 · Final score 54Cohere Command R 7B 12/2024 — €0.01 · Final score 54Llama2 13B Vicuna-1.5 f16 — €0.99 · Final score 54Mistral 7B Instruct v0.1 f16 — €0.75 · Final score 54Meta Llama 3.2 3B — €0.01 · Final score 53Google Recurrent Gemma 9B IT f16 — €0.89 · Final score 50Codestral 22B v1 — €0.06 · Final score 49Qwen: QwQ 32B Preview — €0.05 · Final score 48Nous Llama2 13B Hermes f16 — €1.00 · Final score 48IBM Granite 34B Code Instruct f16 — €1.07 · Final score 46Meta Llama 3.2 1B — €0.02 · Final score 46Mistral Small v2/2402 — €0.06 · Final score 46Mistral Small v1/2312 (Mixtral) — €0.06 · Final score 45DBRX 132B Instruct — €0.26 · Final score 45NVIDIA Llama 3.1 Nemotron 70B Instruct — €0.09 · Final score 45Mistral Medium v1/2312 — €0.81 · Final score 44Microsoft WizardLM 2 7B — €0.02 · Final score 43Llama2 13B Puffin f16 — €4.70 · Final score 43Mistral Tiny v1/2312 (7B Instruct v0.2) — €0.05 · Final score 42Gemma 2 9B IT — €0.02 · Final score 39Meta Llama2 13B chat f16 — €0.75 · Final score 37Mistral 7B Zephyr-β f16 — €0.46 · Final score 35Meta Llama2 7B chat f16 — €0.56 · Final score 34Mistral 7B Notus-v1 f16 — €0.75 · Final score 33Orca 2 13B f16 — €0.95 · Final score 31Mistral 7B Instruct v0.2 f16 — €0.96 · Final score 31Mistral 7B v0.1 f16 — €0.87 · Final score 31Google Gemma 2B IT f16 — €0.30 · Final score 27Microsoft Phi 3 Medium 4K Instruct 14B f16 — €0.82 · Final score 26Orca 2 7B f16 — €0.78 · Final score 22Microsoft Phi 4 Mini Instruct — €0.02 · Final score 13Google Gemma 7B IT f16 — €0.99 · Final score 11Meta Llama2 7B f16 — €0.95 · Final score 10Yi 1.5 9B Chat f16 — €1.41 · Final score 10Qwen3.7 Max — €2.94 · Final score 95Qwen3.7 MaxCost per benchmark run →Final score →
    GPT o1 pro (manual) · no valueQwen3.7 MaxEfficiency frontierMedian of all models
    Efficiency frontier of the selection: (€0.00, 63) · (€0.01, 77) · (€0.02, 88) · (€0.12, 90) · (€0.39, 91) · (€0.50, 93) · (€0.60, 94) · (€1.10, 95) · (€2.06, 96)

    X = estimated cost of the benchmark workload (logarithmic), Y = final score (0–100). Efficiency frontier: no other model in the selection is both cheaper and better. Not a recommendation. Not shown for lack of a value: GPT o1 pro (manual). Click a point: that model becomes model B.

    225 of 225 models

    LLM benchmark results per model and category (0–100, higher is better)
    1

    GPT o1 pro (manual)

    GPTProprietary (cloud)

    10010097100958797––
    2

    GPT-5.6 Sol

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    10010097100889096€2.060.56 rps
    3

    Qwen3.7 Max

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    10010097100888795€2.940.10 rps
    4

    Google Gemini 3.7 Flash (reasoning high)

    GeminiProprietary (cloud)

    10010090100889295€1.680.27 rps
    5

    Fugu Ultra

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    1001009498889095€67.460.04 rps
    6

    GPT-5.5

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    10010094100888795€3.590.43 rps
    7

    Google Gemini 3.7 Flash (reasoning medium)

    GeminiProprietary (cloud)

    1009497100889095€1.100.32 rps
    8

    GPT-5.5 Instant

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    1009794100889095€2.390.67 rps
    9

    GPT-5.5 Pro

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    10010093100888795€37.100.09 rps
    10

    GPT-5.4 Pro

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    1009793100889094€34.690.06 rps
    11

    Google Gemini 3.8 Flash (reasoning high)

    GeminiProprietary (cloud)

    10010090100889094€2.270.19 rps
    12

    Grok 4.5

    GrokLicence not assignedResult outdated or incomplete (flagged by the source)

    1001009498829294€4.200.07 rps
    13

    Grok 4.6 (reasoning medium)

    GrokLicence not assigned

    1001009498829294€4.690.12 rps
    14

    Claude Fable 5.1 (reasoning max)

    ClaudeProprietary (cloud)

    10010090100888794€11.440.16 rps
    15

    Google Gemini 3.8 Flash (reasoning medium)

    GeminiProprietary (cloud)

    1009493100889094€1.280.27 rps
    16

    Grok 4.6 (reasoning high)

    GrokLicence not assigned

    1001009498829094€5.160.10 rps
    17

    Google Gemini 3.7 Flash (reasoning low)

    GeminiProprietary (cloud)

    1009490100889294€0.600.40 rps
    18

    GPT-5.3 Chat Latest

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    100979398888593€1.930.74 rps
    19

    Grok 4.6 (reasoning low)

    GrokLicence not assigned

    1001009497828593€3.060.18 rps
    20

    Google Gemini 3.8 Flash (reasoning low)

    GeminiProprietary (cloud)

    1008693100889093€0.500.38 rps
    21

    Claude Opus 5 (reasoning low)

    ClaudeProprietary (cloud)

    10010090100788792€2.800.36 rps
    22

    Claude Opus 5 (reasoning medium)

    ClaudeProprietary (cloud)

    1001009498758792€3.190.34 rps
    23

    Qwen3.8 2.4T A95B (reasoning xhigh)

    QwenOpen source

    100939486889292€3.000.24 rps
    24

    GPT o1-preview v1/2024-09-12

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    95929495888792€52.320.08 rps
    25

    Grok 4.6 (reasoning xhigh)

    GrokLicence not assigned

    1001009490828592€5.040.10 rps
    26

    Qwen3.8 27B (reasoning xhigh)

    QwenOpen source

    1001009486828892€1.420.10 rps
    27

    Qwen3.8 2.4T A95B (reasoning medium)

    QwenOpen source

    98929491888792€1.980.40 rps
    28

    Claude Opus 5 (reasoning high)

    ClaudeProprietary (cloud)

    1001009298748692€3.590.36 rps
    29

    Google Gemini 3.6 Flash

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    959193100888291€0.391.21 rps
    30

    Qwen3.8 Max (reasoning medium)

    QwenLicence not assigned

    97929096888791€2.000.23 rps
    31

    GPT-5.6 Terra

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    89979098888691€1.180.61 rps
    32

    Qwen3.8 2.4T A95B (reasoning low)

    QwenOpen source

    98969098818391€1.660.49 rps
    33

    GPT o1 v1/2024-12-17

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    100959491828391€30.630.17 rps
    34

    Qwen3.8 Max (reasoning minimal)

    QwenLicence not assigned

    100969092818691€1.670.28 rps
    35

    Qwen3.8 Max (reasoning high)

    QwenLicence not assigned

    100939379889291€3.180.12 rps
    36

    Claude Opus 4.7

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    100979798827091€2.050.43 rps
    37

    DeepSeek V4 Flash 0731 (reasoning high)

    DeepSeekLicence not assigned

    1009610077828790€0.120.11 rps
    38

    DeepSeek V4 Flash Vision Exp (reasoning low)

    DeepSeekLicence not assigned

    93919792828790€0.340.31 rps
    39

    Claude Fable 5 (reasoning max)

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    8710090100788790€17.240.20 rps
    40

    Qwen3.8 Max (reasoning xhigh)

    QwenLicence not assigned

    1009310079828790€3.000.13 rps
    41

    DeepSeek V4 Flash 0731 (reasoning max)

    DeepSeekLicence not assigned

    1009610075828790€0.140.14 rps
    42

    GPT-5.6 Luna

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    1009097100886690€0.560.51 rps
    43

    Qwen3.8 Max (reasoning low)

    QwenLicence not assigned

    98929094798790€1.750.26 rps
    44

    Qwen3.8 27B (reasoning medium)

    QwenOpen source

    981009273888890€0.930.13 rps
    45

    Google Gemini 3.1 Pro Preview

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90969398827890€0.540.44 rps
    46

    GLM 5.2

    GLMOpen sourceResult outdated or incomplete (flagged by the source)

    84969698887790€0.340.14 rps
    47

    GPT o1-mini v1/2024-09-12

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    93969483828789€8.150.16 rps
    48

    GPT-5.4

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    8697100100827189€0.740.81 rps
    49

    DeepSeek V4 Flash Vision Exp (reasoning high)

    DeepSeekLicence not assigned

    100969278828789€0.380.26 rps
    50

    DeepSeek V4 Flash 0731 (reasoning low)

    DeepSeekLicence not assigned

    100969775828589€0.170.06 rps
    51

    DeepSeek V4 Pro 0813 (reasoning low)

    DeepSeekLicence not assigned

    95959777828789€1.440.15 rps
    52

    GPT-4o v3/2024-11-20

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    86979495887289€0.631.14 rps
    53

    Qwen3.8 27B (reasoning low)

    QwenOpen source

    97969073888888€0.770.17 rps
    54

    DeepSeek V4 Pro

    DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source)

    100869777828888€0.350.20 rps
    55

    GPT-4o v1/2024-05-13

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    909610092787488€1.211.44 rps
    56

    DeepSeek V4 Pro 0813 (reasoning high)

    DeepSeekLicence not assigned

    95919477829088€1.780.12 rps
    57

    Qwen3.7 Plus

    QwenLicence not assigned

    909710083827788€0.090.78 rps
    58

    DeepSeek V4 Flash Vision Exp (reasoning max)

    DeepSeekLicence not assigned

    100929774957188€1.640.12 rps
    59

    OpenAI GPT-OSS 120B

    GPTProprietary (cloud)

    100899394708388€0.021.35 rps
    60

    Kimi K3

    KimiLicence not assignedResult outdated or incomplete (flagged by the source)

    901009298668288€4.050.05 rps
    61

    Claude Opus 4.8

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    821009398827388€1.920.62 rps
    62

    Google Gemini 1.5 Pro v2

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    86979499787488€1.001.18 rps
    63

    DeepSeek V4 Flash (reasoning high)

    DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source)

    100898977888388€0.090.27 rps
    64

    X-AI Grok 2 v2/1212

    GrokLicence not assignedResult outdated or incomplete (flagged by the source)

    66959797887887€0.580.99 rps
    65

    Google Gemini 3 Flash Preview

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    909190100886086€0.130.51 rps
    66

    Claude Sonnet 5 (reasoning low)

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    958910075886986€1.390.46 rps
    67

    Claude Sonnet 5 (reasoning xhigh)

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    979610091765586€2.900.35 rps
    68

    Mistral Medium 3.5

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    908910086885985€0.440.37 rps
    69

    NVIDIA Nemotron 3.5 Lightning

    NemotronOpen source

    93888675888085€0.280.13 rps
    70

    GPT-4 Turbo v5/2024-04-09

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    86999896884385€2.450.84 rps
    71

    Claude Sonnet 5 (reasoning high)

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90829390826984€2.700.36 rps
    72

    Meta Muse Glimmer 30B (reasoning low)

    OthersOpen source

    100939073826884€0.280.44 rps
    73

    Meta Muse Glimmer 30B (reasoning high)

    OthersOpen source

    100938671807584€0.690.25 rps
    74

    GLM 5.3 Flash (reasoning high)

    GLMOpen source

    77949377887584€0.030.38 rps
    75

    Google Gemini 2.0 Flash Exp

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    6396100100826284€0.030.85 rps
    76

    Google Gemini Exp 1121

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    70979795727284€0.890.49 rps
    77

    Google Gemini 3.5 Flash-Lite

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    84929794825484€0.091.59 rps
    78

    Google Gemini 3.5 Flash

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    868384100886083€0.410.63 rps
    79

    Google Gemini 3.1 Flash-Lite

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    77929386826883€0.070.59 rps
    80

    Qwen3.6 27B

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    869710073826183€0.110.82 rps
    81

    GPT-4o v2/2024-08-06

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90849786825983€0.631.49 rps
    82

    Google Gemini 1.5 Pro 0801

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    849279100707483€0.900.83 rps
    83

    GPT-4.1

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    86839779826983€0.500.84 rps
    84

    Gemma 4 31B IT

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    86919087885583€0.030.27 rps
    85

    Claude Fable 5 (reasoning max, after)

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    581009075829083€15.160.17 rps
    86

    Meta Muse Glimmer 30B (reasoning medium)

    OthersOpen source

    100938671816282€0.480.35 rps
    87

    Claude Sonnet 5 (reasoning medium)

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    89799475886982€2.140.38 rps
    88

    GLM 5.3 Flash (reasoning low)

    GLMOpen source

    87939279697382€0.020.43 rps
    89

    Qwen 2.5 72B Instruct

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    79929497715982€0.100.66 rps
    90

    GLM 5.1

    GLMOpen sourceResult outdated or incomplete (flagged by the source)

    77979775826382€0.231.68 rps
    91

    Gemma 4 26B A4B IT

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    869410073825682€0.020.55 rps
    92

    DeepSeek V4 Flash Vision Exp (reasoning none)

    DeepSeekLicence not assigned

    86899080885982€0.050.86 rps
    93

    Nous Llama 3.1 405B Hermes 3

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    68938998885381€0.540.49 rps
    94

    MiniMax M2.7

    MiniMaxLicence not assignedResult outdated or incomplete (flagged by the source)

    97878975696981€0.450.61 rps
    95

    Claude 3.5 Sonnet v2

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    82979384715781€0.950.09 rps
    96

    GLM 5.3 Flash (reasoning max)

    GLMOpen source

    871008977765481€0.070.24 rps
    97

    GPT-4 v1/0314

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90889873884580€7.041.31 rps
    98

    GPT-5.4 Mini

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90948173825980€0.221.02 rps
    99

    Poolside Laguna S 2.1

    OthersOpen sourceResult outdated or incomplete (flagged by the source)

    639310089954080€0.031.39 rps
    100

    X-AI Grok 2 v1/1012

    GrokLicence not assignedResult outdated or incomplete (flagged by the source)

    63938790885880€1.030.31 rps
    101

    GPT-4 v2/0613

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90839573884579€7.042.16 rps
    102

    Tencent Hy3

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    84949773705579€0.050.11 rps
    103

    Qwen3.6 35B A3B

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    66899584825578€0.050.57 rps
    104

    DeepSeek v3 671B

    DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source)

    62959785755578€0.030.49 rps
    105

    GPT-4o Mini

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    638780731006578€0.041.46 rps
    106

    Claude 3.5 Sonnet v1

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    72838987805878€0.940.09 rps
    107

    GPT-4.1 Mini

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    66979775825178€0.100.79 rps
    108

    Nous Llama 3.1 70B Hermes 3

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    74978782883978€0.030.42 rps
    109

    Claude Opus 4.6

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    9510010081385278€1.590.40 rps
    110

    DeepSeek V4 Flash 0731 (reasoning none)

    DeepSeekLicence not assigned

    84848175825878€0.030.05 rps
    111

    Qwen3.7 Flash

    QwenLicence not assigned

    59899684825577€0.010.62 rps
    112

    Qwen3.6 Flash

    QwenLicence not assigned

    59899684825577€0.060.91 rps
    113

    Claude 3 Opus

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    698810074765877€4.690.41 rps
    114

    OpenAI GPT-OSS 20B

    GPTProprietary (cloud)

    87867573826177€0.021.35 rps
    115

    Meta Llama 3.1 405B Instruct

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    81939275754877€2.391.16 rps
    116

    GPT-4 Turbo v4/0125-preview

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    669710083754377€2.450.84 rps
    117

    Google LearnLM 1.5 Pro Experimental

    GeminiLicence not assignedResult outdated or incomplete (flagged by the source)

    48978596647277€0.310.83 rps
    118

    GPT-4 Turbo v3/1106-preview

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    66759873886076€2.460.68 rps
    119

    MiniMax M3

    MiniMaxLicence not assignedResult outdated or incomplete (flagged by the source)

    68919477884176€0.140.39 rps
    120

    Google Gemini Exp 1206

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    521008577756976€0.880.16 rps
    121

    Qwen 2.5 32B Coder Instruct

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    43949898764676€0.050.82 rps
    122

    Grok 4.3

    GrokLicence not assignedResult outdated or incomplete (flagged by the source)

    84908280754376€1.690.17 rps
    123

    Kimi K2.6

    KimiLicence not assignedResult outdated or incomplete (flagged by the source)

    86869054766176€0.210.94 rps
    124

    DeepSeek v2.5 236B

    DeepSeekLicence not assignedResult outdated or incomplete (flagged by the source)

    57809180885775€0.030.42 rps
    125

    Meta Llama 3.1 70B Instruct f16

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    74899075754875€1.790.90 rps
    126

    Google Gemini 1.5 Flash v2

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    64968976814475€0.062.01 rps
    127

    Claude Opus 4.5

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    70889277824075€1.560.43 rps
    128

    Google Gemini 1.5 Pro 0409

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    68979680752674€0.950.59 rps
    129

    Meta Llama 3 70B Instruct

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    81838467814573€0.060.85 rps
    130

    GPT-3.5 v2/0613

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    68817387815073€0.341.46 rps
    131

    Amazon Nova Lite

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    67787494626273€0.022.19 rps
    132

    Claude Sonnet 4.5

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    72919073753472€0.970.42 rps
    133

    Mistral Large 123B v2/2407

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    68796875757072€0.571.02 rps
    134

    Google Gemini Flash 1.5 8B

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    70937867764872€0.011.19 rps
    135

    Google Gemini 1.5 Pro 0514

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    739679100256072€1.070.92 rps
    136

    Google Gemini 1.5 Flash 0514

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    329710076725272€0.061.77 rps
    137

    Google Gemini 1.0 Pro

    GeminiProprietary (cloud)Result outdated or incomplete (flagged by the source)

    66868379882871€0.371.36 rps
    138

    Meta Llama 3.2 90B Vision

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    74848777713271€0.231.10 rps
    139

    GPT-3.5 v3/1106

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    68707181785871€0.242.33 rps
    140

    Claude 3.5 Haiku

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    52807275756870€0.321.24 rps
    141

    Claude Sonnet 4.6

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    90889075384270€0.950.59 rps
    142

    Meta Llama 3.3 70B Instruct

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    74787477714670€0.100.71 rps
    143

    Claude Haiku 4.5

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    63809173753770€0.320.76 rps
    144

    GPT-3.5 v4/0125

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    63877177784370€0.121.43 rps
    145

    Cohere Command R+

    OthersProprietary (cloud)Result outdated or incomplete (flagged by the source)

    63807672705870€0.831.90 rps
    146

    Mistral Large 123B v3/2411

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    68756476825170€0.560.66 rps
    147

    Qwen1.5 32B Chat f16

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    70908276782069€0.971.66 rps
    148

    Gemma 2 27B IT

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    61728774893269€0.070.90 rps
    149

    GPT-4.1 Nano

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    68847766784169€0.030.79 rps
    150

    Mistral 7B OpenChat-3.5 v3 0106 f16

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    68876774882568€0.323.39 rps
    151

    Meta Llama 3 8B Instruct f16

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    79626870804167€0.323.33 rps
    152

    Gemma 7B OpenChat-3.5 v3 0106 f16

    GemmaOpen sourceResult outdated or incomplete (flagged by the source)

    63678458814667€0.215.09 rps
    153

    NVIDIA Nemotron 3 Super 120B A12B

    NemotronLicence not assignedResult outdated or incomplete (flagged by the source)

    48846968884267€0.030.97 rps
    154

    GPT-5.4 Nano

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    59907671822167€0.061.06 rps
    155

    GPT-3.5-instruct 0914

    GPTProprietary (cloud)

    47926969883366€0.352.15 rps
    156

    Amazon Nova Pro

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    64788279524166€0.221.34 rps
    157

    GPT-3.5 v1/0301

    GPTProprietary (cloud)Result outdated or incomplete (flagged by the source)

    55826981822666€0.354.12 rps
    158

    Llama 3 8B OpenChat-3.6 20240522 f16

    LlamaOpen sourceResult outdated or incomplete (flagged by the source)

    76517665883866€0.283.79 rps
    159

    Mistral 7B OpenChat-3.5 v1 f16

    MistralOpen source

    58727271883366€0.492.20 rps
    160

    Mistral 7B OpenChat-3.5 v2 1210 f16

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    63737266883065€0.323.40 rps
    161

    Qwen 2.5 7B Instruct

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    48778068694765€0.130.63 rps
    162

    Starling 7B-alpha f16

    OthersLicence not assigned

    58666773883464€0.581.85 rps
    163

    Mistral Nemo 12B v1/2407

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    54585199754964€0.031.22 rps
    164

    Meta Llama 3.2 11B Vision

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    70716570713664€0.041.49 rps
    165

    Nous Llama 3 8B Hermes 2 Theta

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    61737474851664€0.050.55 rps
    166

    Claude 3 Haiku

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    64696475753564€0.080.52 rps
    167

    Yi 1.5 34B Chat f16

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    47787074862664€1.181.37 rps
    168

    Gemma 3n E4B IT

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    32737374715463€0.020.44 rps
    169

    Liquid: LFM 40B MoE

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    72696563822463€0.001.45 rps
    170

    Meta Llama 3.1 8B Instruct f16

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    57746274743262€0.452.41 rps
    171

    Gemma 3 4B IT

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    48846072693762€0.010.54 rps
    172

    Qwen2 7B Instruct f32

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    50818161663162€0.462.36 rps
    173

    Claude 3 Sonnet

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    72417474782861€0.950.85 rps
    174

    MistralAI Ministral 3B 2512

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    53596971813261€0.020.54 rps
    175

    Mistral Small v3/2409

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    43757174752661€0.060.81 rps
    176

    NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning

    NemotronLicence not assignedResult outdated or incomplete (flagged by the source)

    507764321004161€0.001.06 rps
    177

    Mistral Pixtral 12B

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    53697363644060€0.030.83 rps
    178

    Mixtral 8x22B API (Instruct)

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    5362629775759€0.173.12 rps
    179

    Anthropic Claude Instant v1.2

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    58756577651659€2.101.49 rps
    180

    Codestral Mamba 7B v1

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    53665197711759€0.302.82 rps
    181

    Inflection 3 Productivity

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    46593970796159€0.920.17 rps
    182

    Anthropic Claude v2.0

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    63525567843459€2.190.40 rps
    183

    Cohere Command R

    OthersProprietary (cloud)Result outdated or incomplete (flagged by the source)

    45665774842759€0.132.50 rps
    184

    Amazon Nova Micro

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    58686471593159€0.012.41 rps
    185

    Qwen1.5 7B Chat f16

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    56816056603658€0.293.76 rps
    186

    Mistral Large v1/2402

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    37497083842558€0.582.11 rps
    187

    Microsoft WizardLM 2 8x22B

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    48767959622258€0.130.70 rps
    188

    Qwen1.5 14B Chat f16

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    50585172842256€0.363.03 rps
    189

    MistralAI Ministral 8B

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    56554182683055€0.021.02 rps
    190

    Anthropic Claude v2.1

    ClaudeProprietary (cloud)Result outdated or incomplete (flagged by the source)

    29585978753255€2.250.35 rps
    191

    Mistral 7B OpenOrca f16

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    54577636782755€0.412.65 rps
    192

    MistralAI Ministral 3B

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    50483989604154€0.011.02 rps
    193

    Cohere Command R 7B 12/2024

    OthersProprietary (cloud)Result outdated or incomplete (flagged by the source)

    48686365552554€0.011.45 rps
    194

    Llama2 13B Vicuna-1.5 f16

    LlamaLlama licence

    50375562823754€0.991.09 rps
    195

    Mistral 7B Instruct v0.1 f16

    MistralProprietary (cloud)

    34716963622354€0.751.43 rps
    196

    Meta Llama 3.2 3B

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    52716671441453€0.011.25 rps
    197

    Google Recurrent Gemma 9B IT f16

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    58277164562350€0.891.21 rps
    198

    Codestral 22B v1

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    38474484661349€0.064.03 rps
    199

    Qwen: QwQ 32B Preview

    QwenLicence not assignedResult outdated or incomplete (flagged by the source)

    43327452484048€0.050.63 rps
    200

    Nous Llama2 13B Hermes f16

    LlamaLlama licence

    50243775604248€1.001.07 rps
    201

    IBM Granite 34B Code Instruct f16

    OthersProprietary (cloud)Result outdated or incomplete (flagged by the source)

    6349346757746€1.071.51 rps
    202

    Meta Llama 3.2 1B

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    32403353685146€0.021.69 rps
    203

    Mistral Small v2/2402

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    3342458856846€0.063.21 rps
    204

    Mistral Small v1/2312 (Mixtral)

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    1067636556845€0.062.21 rps
    205

    DBRX 132B Instruct

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    43394374591045€0.261.31 rps
    206

    NVIDIA Llama 3.1 Nemotron 70B Instruct

    LlamaLlama licenceResult outdated or incomplete (flagged by the source)

    68542572282145€0.090.53 rps
    207

    Mistral Medium v1/2312

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    41434459621244€0.810.35 rps
    208

    Microsoft WizardLM 2 7B

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    53344266531343€0.020.89 rps
    209

    Llama2 13B Puffin f16

    LlamaLlama licence

    37154467563943€4.700.23 rps
    210

    Mistral Tiny v1/2312 (7B Instruct v0.2)

    MistralProprietary (cloud)Result outdated or incomplete (flagged by the source)

    2247595362842€0.052.39 rps
    211

    Gemma 2 9B IT

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    45254736681339€0.020.88 rps
    212

    Meta Llama2 13B chat f16

    LlamaLlama licence

    2238176575637€0.751.44 rps
    213

    Mistral 7B Zephyr-β f16

    MistralOpen sourceResult outdated or incomplete (flagged by the source)

    3734466229435€0.462.34 rps
    214

    Meta Llama2 7B chat f16

    LlamaLlama licence

    22332062501834€0.561.93 rps
    215

    Mistral 7B Notus-v1 f16

    MistralLicence not assignedResult outdated or incomplete (flagged by the source)

    1054256048433€0.751.43 rps
    216

    Orca 2 13B f16

    OthersLicence not assigned

    18223229672031€0.951.14 rps
    217

    Mistral 7B Instruct v0.2 f16

    MistralProprietary (cloud)

    1130542558831€0.961.12 rps
    218

    Mistral 7B v0.1 f16

    MistralProprietary (cloud)

    094863521231€0.871.23 rps
    219

    Google Gemma 2B IT f16

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    33281647152027€0.303.54 rps
    220

    Microsoft Phi 3 Medium 4K Instruct 14B f16

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    534303247826€0.821.32 rps
    221

    Orca 2 7B f16

    OthersLicence not assigned

    220262652422€0.781.38 rps
    222

    Microsoft Phi 4 Mini Instruct

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    102122522413€0.020.58 rps
    223

    Google Gemma 7B IT f16

    GemmaLicence not assignedResult outdated or incomplete (flagged by the source)

    000662011€0.991.08 rps
    224

    Meta Llama2 7B f16

    LlamaLlama licence

    0522328210€0.951.13 rps
    225

    Yi 1.5 9B Chat f16

    OthersLicence not assignedResult outdated or incomplete (flagged by the source)

    0429170810€1.410.76 rps

    Scores 0–100 per category, higher is better; the shading is relative to the best result in each column. Cost and speed do not count towards the score. Result outdated or incomplete (flagged by the source)

    The benchmark categories in detail

    The TIMETOACT GROUP does not test the models on academic quiz questions but on tasks taken from real products. This is what each category checks:

    Code
    Can the model generate and understand code and help with programming?
    CRM
    How well does the model support work with product catalogues and marketplaces?
    Docs
    How well does the model work with large documents and knowledge bases?
    Integrate
    Can the model interact reliably with external APIs, services and plugins?
    Marketing
    How well does the model support marketing — brainstorming, ideation and copywriting?
    Reason
    How well does the model draw logical conclusions within a given context?
    Final
    Overall result across all categories — the basis of the ranking.
    Cost
    Estimated cost of running the benchmark workload (for local models incl. GPU hardware). Not part of the score.
    Speed
    Estimated requests per second — higher is faster. Not part of the score.

    Data source: LLM Benchmarks by the TIMETOACT GROUP

    All values come from the LLM Benchmarks of the TIMETOACT GROUP. Every month the team tests new and established language models on real use cases from client projects and publishes the results as a leaderboard. comparison.cloud takes the values over unchanged; ranking, licence labels and flags for outdated results follow the source.

    New editions are checked daily and picked up automatically. The data is kept as one YAML file per model, with the edition cited as the source of every value.

    Data as of: September 2026 edition, imported on 02/11/2026

    AI Compass · find the AI platform

    Settled on a model?The AI Compass findsthe platform to run it.

    Starts with what the AI should do (use cases, models, operating mode, scale) and under which rules (data class, regulation, EU AI Act, processing location, data-protection commitments). A language model scores 14 providers on their documented AI catalogue, data-protection commitments and verified evidence; the engine weights and ranks — result by email, with sources.

    1. 1.Describe the project

      What the AI should do — use cases, models, operating mode, scale — and under which rules: data class, EU AI Act, processing location.

    2. 2.Measure the providers

      14 cloud providers on their documented AI catalogue, data-protection commitments and verified evidence.

    3. 3.Take the recommendation along

      Fit per provider with reasons — by email, with all sources.

    14 providers5 dimensionsEU AI Act in the profile