LLM Benchmarks
(18 articles)Why Frontier AI Benchmarks Are Broken in 2026
A new book by Moritz Hardt argues that benchmark rankings — not scores — are what actually matter. We tested his thesis against every major 2026 AI benchmark.
Krasis vs llama.cpp: Is 10x Faster LLM Inference Real?
Krasis LLM Runtime claims dramatically faster inference than llama.cpp for large MoE models on a single NVIDIA GPU. We break down the real numbers, the...
A $500 GPU Just Beat Claude Sonnet at Coding Tasks
ATLAS, a source-available AI system built by a Virginia Tech student, scores 74.6% on LiveCodeBench using a single $500 consumer GPU — outperforming Claude...
ROCm vs Vulkan Performance: Mi50 Benchmark (4 Models)
New benchmarks pit ROCm 7 nightly against Vulkan on an AMD Mi50 32GB running llama.cpp. Vulkan wins short-context dense inference, but ROCm dominates...
CRYSTAL Benchmark Exposes How AI Models Fake Reasoning
A new benchmark tested 20 multimodal AI models and found 19 of them cherry-pick reasoning steps while skipping actual thinking. The gap between accuracy and...
Qwen3.5-9B vs GPT: Document OCR Benchmarks & Weak Spot
Benchmark data shows Qwen3.5-9B beats frontier models on OCR and field extraction, yet stumbles badly on tables. Here's the honest breakdown.