OPEN MODEL BENCHMARKING

Put your AI
to the test.

Measure inference speed, response time and reliability on your own hardware. Every result starts with a repeatable workload and ends with evidence you can inspect.

CPU + GPULocal executionOpen evidence
YOUR BENCHMARKLOCAL FIRST
ONE FIXED WORKLOADYour setup, measured.
Throughput Latency Reliability

The core loop

One result, independently inspectable.

A clear path from your own machine to a result others can reproduce.

  1. 01

    Run

    Use a versioned profile against a local OpenAI-compatible endpoint.

  2. 02

    Package

    Keep the summary, raw runner output, configuration, and reproducibility notes together.

  3. 03

    Validate

    GitHub checks schema, required metadata, and accidental duplicates before merge.

  4. 04

    Compare

    Filter the published registry by model, runtime, hardware, and profile.

Benchmark contract v1

Choose the workload that matters.

Their YAML definitions are versioned alongside every submitted result. A later methodology change creates a new profile ID instead of rewriting history.

chat-small-v1

Interactive assistant latency

Input
1,024 tokens
Output
256 tokens
Load
1 stream
chat-small-extended-v1

More time for slower hardware

Input
1,024 tokens
Output
256 tokens
Load
1 stream
chat-long-v1

Long-context assistant latency

Input
8,192 tokens
Output
512 tokens
Load
1 stream
throughput-v1

Sustained serving throughput

Input
1,024 tokens
Output
256 tokens
Load
16 streams

Benchmark registry

Performance with the evidence to back it up.

Start from the hardware: choose a processor, graphics card or memory size to see how models run on it, or choose a model to see it across machines. Open a result to inspect its configuration, measurements and source artifacts.

5 registry results

Serving hardware

Model and workload

Showing 5 of 5 registry results.

These runs differ in model, context length, tokenizer. Rows side by side are not the same measurement unless model, profile and settings match; the comparison page says exactly what differs.

CPUInference GPURAMVRAMModelQuantizationRuntimeProfileOutput tok/sTTFT p50Record

AMD Ryzen 7 5800X3D 8-Core Processor

8 cores · 16 threads

AMD Radeon RX 9070

GPU only
32 GBinstalled15.92 GB eachdeepseek-ai/DeepSeek-R1-Distill-Qwen-14BQ4_K_MLM Studio 0.4.28 (build 2)chat-small-v253643.7 msView detailsCompare ↗

AMD Ryzen 7 5800X3D 8-Core Processor

Cores not recorded

AMD Radeon RX 9070

GPU only
31.92 GBusable, as reported16 GB eachgemma-4-e4b-uncensored-hauhaucs-aggressiveQ4_K_MLM Studio 0.4.28 (build 2)chat-small-v291.6420.6 msView detailsCompare ↗

AMD Ryzen 7 5800X3D 8-Core Processor

Cores not recorded

AMD Radeon RX 9070

GPU only
31.92 GBusable, as reported16 GB eachqwen2.5-coder-14b-instructQ4_K_MLM Studio 0.4.28 (build 2)chat-small-v242.7612 msView detailsCompare ↗

AMD Ryzen 7 5800X3D 8-Core Processor

Cores not recorded

AMD Radeon RX 9070

GPU only
31.92 GBusable, as reported16 GB eachqwen/qwen3-vl-4bQ4_K_MLM Studio 0.4.28 (build 2)chat-small-v2143.4242.9 msView detailsCompare ↗

AMD Ryzen 7 5800X3D 8-Core Processor

Cores not recorded

AMD Radeon RX 9070

GPU only
31.92 GBusable, as reported16 GB eachdeepseek-ai/DeepSeek-R1-Distill-Qwen-14BQ4_K_MLM Studio 0.4.28 (build 2)chat-small-v253.3619.5 msView detailsCompare ↗