Every model sidekick has graded, on every compute path it was graded on. Accuracy compares sidekick's output with the original fp32 model on the parity suite's 51 test inputs (embedders) or its classification cases.
Loading results…A to F, the parity suite's letter. F is a failed hard gate. Compare letters within a task.
−log₁₀ of the worst error: 1 − cosine for embedders (4 nines is a cosine of 0.9999), the largest probability change Δp for classifiers and rerankers (3 nines is a Δp of 0.001). Higher is better. The grade, not the nines, is comparable across tasks' scales.
Median per input from the grading run, one input at a time. Dotted values come from a machine that wasn't idle, or whose load wasn't recorded. Fixed-length benchmark timings, where they exist, are in the value's tooltip.
The process's own footprint once loaded, where benchmarked; hover or tap a value for how it was measured. On Apple silicon a process's count misses Neural Engine memory held by the system, so those rows also give the whole machine's change where it stood clear of noise.
How good the model is at a task, beside how faithfully each path reproduces it. Pick a benchmark and category to add the column and keep only models with that score. Plain scores are MTEB runs, from the MTEB results repository at a pinned commit; italic ones are self-reported by the model's authors. Compare scores within one benchmark only. † marks tasks in the model's training data. Hover or tap a score for its source.
A benchmark may have run on longer inputs than a build takes: each score's tooltip gives the run's context, and each row its build's max tokens. MTEB prompts many models per task, and sidekick adds each model's own query or document prompt by default. To reproduce a run's prompts on any task, send them in the text with input_type: "none", which adds nothing (for e5-small-v2 outside retrieval and reranking, send no prompt at all), or choose a model's own task prompt by name with prompt_name, such as EmbeddingGemma's "Classification". Without the run's prompts, expect lower classification, clustering and STS. A grade-A path reproduces the model closely enough for its score to carry over (embedders: worst cosine ≥ 0.9999; classifiers: worst Δp ≤ 0.001 or within 1.25× the fp16 ceiling); C, D or int8 may not. A few hundred labelled examples from your own data decide. More in docs/MODELS.md.
The licence is the model's own, linked to its text or to the model card that states it. Training data has terms of its own, and whether they reach the trained weights is legally unsettled, so the training data column shows what sidekick's licence audit found. That decides how sidekick publishes a build: openly, with a notice on the model card, when training data is non-commercial, copyleft or undisclosed; gated, with terms you accept first, when the terms must be passed on to everyone who gets the files (Gemma); or not at all when the licence limits commercial use. Hover or tap a licence or a flag for details, and use Leave out to hide what doesn't suit your use. The audit records terms, not legal conclusions. More in docs/PUBLISHING.md and docs/MODELS.md.