Back to browse
GitHub Repository

Community-driven benchmark suite for MLX inference engines on Apple Silicon

15 starsPython

mlx-chronos - benchmark MLX inference engines on Apple Silicon

by igurss·Jun 25, 2026·2 points·0 comments

AI Analysis

●●SolidNiche GemBig Brain

Standardized MLX benchmarking when everyone's currently comparing engines manually.

Strengths
  • Sealed JSON results enable reproducible cross-Mac comparisons without cherry-picking
  • Thermal state tracking captures heat throttling that other benchmarks ignore
  • Supports four different MLX engines with consistent measurement protocol
Weaknesses
  • Apple Silicon only excludes the majority of ML deployment targets
  • Tool calling benchmarks marked as planned but not yet implemented
Category
Target Audience

Apple Silicon ML developers comparing inference engines

Similar To

MLPerf · lm-eval-harness · Perplexity Benchmarks

Similar Projects

Data●●Solid

Benchmarking Apple Silicon unified mem for GPU-accelerated SQL analysis

The repo does one practical thing well: quantify the real-world impact of Apple Silicon's unified memory on analytics by running six TPC-H queries plus a GPU-favorable QX and shipping the raw charts and code. It's specific and empirical — you get MLX vs NumPy vs DuckDB numbers and PNGs, not just hand-wavy claims — but it's narrowly scoped to M4 hardware and small-ish scales, so its conclusions are useful for experimentation rather than sweeping generalization.

WizardryNiche Gem
sadopc
314mo ago
AI/ML●●●Banger

iPhone ANE holds LLM tok/s while MLX and LiteRT thermal-throttle

LiteRT beats MLX on Gemma memory while CoreML sips power on the Neural Engine.

Dark HorseBig BrainSolve My Problem
mlboy
1022d ago