Interactive assistant latency
- Input
- 1,024 tokens
- Output
- 256 tokens
- Load
- 1 stream
OPEN MODEL BENCHMARKING
Measure inference speed, response time and reliability on your own hardware. Every result starts with a repeatable workload and ends with evidence you can inspect.
The core loop
A clear path from your own machine to a result others can reproduce.
01
Use a versioned profile against a local OpenAI-compatible endpoint.
02
Keep the summary, raw runner output, configuration, and reproducibility notes together.
03
GitHub checks schema, required metadata, and accidental duplicates before merge.
04
Filter the published registry by model, runtime, hardware, and profile.
Benchmark contract v1
Their YAML definitions are versioned alongside every submitted result. A later methodology change creates a new profile ID instead of rewriting history.
Interactive assistant latency
More time for slower hardware
Long-context assistant latency
Sustained serving throughput
Benchmark registry
Start from the hardware: choose a processor, graphics card or memory size to see how models run on it, or choose a model to see it across machines. Open a result to inspect its configuration, measurements and source artifacts.
Serving hardware
Model and workload
Showing 5 of 5 registry results.
These runs differ in model, context length, tokenizer. Rows side by side are not the same measurement unless model, profile and settings match; the comparison page says exactly what differs.
| CPU | Inference GPU | RAM | VRAM | Model | Quantization | Runtime | Profile | Output tok/s | TTFT p50 | Record |
|---|---|---|---|---|---|---|---|---|---|---|
AMD Ryzen 7 5800X3D 8-Core Processor 8 cores · 16 threads | AMD Radeon RX 9070 GPU only | 32 GBinstalled | 15.92 GB each | deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | Q4_K_M | LM Studio 0.4.28 (build 2) | chat-small-v2 | 53 | 643.7 ms | View detailsCompare ↗ |
AMD Ryzen 7 5800X3D 8-Core Processor Cores not recorded | AMD Radeon RX 9070 GPU only | 31.92 GBusable, as reported | 16 GB each | gemma-4-e4b-uncensored-hauhaucs-aggressive | Q4_K_M | LM Studio 0.4.28 (build 2) | chat-small-v2 | 91.6 | 420.6 ms | View detailsCompare ↗ |
AMD Ryzen 7 5800X3D 8-Core Processor Cores not recorded | AMD Radeon RX 9070 GPU only | 31.92 GBusable, as reported | 16 GB each | qwen2.5-coder-14b-instruct | Q4_K_M | LM Studio 0.4.28 (build 2) | chat-small-v2 | 42.7 | 612 ms | View detailsCompare ↗ |
AMD Ryzen 7 5800X3D 8-Core Processor Cores not recorded | AMD Radeon RX 9070 GPU only | 31.92 GBusable, as reported | 16 GB each | qwen/qwen3-vl-4b | Q4_K_M | LM Studio 0.4.28 (build 2) | chat-small-v2 | 143.4 | 242.9 ms | View detailsCompare ↗ |
AMD Ryzen 7 5800X3D 8-Core Processor Cores not recorded | AMD Radeon RX 9070 GPU only | 31.92 GBusable, as reported | 16 GB each | deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | Q4_K_M | LM Studio 0.4.28 (build 2) | chat-small-v2 | 53.3 | 619.5 ms | View detailsCompare ↗ |