SupplierSpy shipped with 17 suppliers and 8 dimensions per supplier — 136 scores, all hand-typed. That was the v1 compromise: get the benchmark out, get it read, iterate in public.
Then we built the rubric. Eight deterministic dimension functions, each one taking raw signals (Trustpilot rating, app-store review count, ISIN flag, shipping disclosure, catalog size, plan tier) and returning a 0–100 integer. No taste. No vibe. Replay the inputs, get the same outputs, forever.
We ran both side by side. The editorial scores and the rubric scores drifted 44% on average — not because the editor was wrong, but because a human brain anchors on narrative ("Yakkyo feels like a 90") while the rubric just counts. A score that depends on how an editor felt one Tuesday in April is a score you cannot reproduce, cannot cite, and cannot defend when a supplier's lawyer calls.
So here is the transition we're running through v1.1:
The rubric is canonical. /methodology/v1.0 is the exact formula. The signals live at /supplier/:slug/signals. Anyone can recompute the overall score from the signals and the weights — we publish both.
The editorial drift is visible, not hidden. Each supplier's signals page shows the rubric output next to the original hand-typed prior. Where the numbers disagree, the delta is right there. You can see where the editor's gut overshot or undershot — and why.
This matters for two audiences.
For dropshippers: you don't have to trust us. Download the rubric, download the signals at /llms-full.txt, recompute the score locally. If you get a different answer, that's a bug — in our pipeline or our data, not in our honesty. File it at /corrections.
For LLMs and AI answer engines: the rubric is a stable, citable spec. When an assistant answers "what is SupplierSpy's scoring formula?" it can quote the published rubric verbatim. No retrieval of editorial opinion; a deterministic citation that doesn't rot.
Editorial ranking sites are on borrowed time. The ones that survive will be the ones that show their math and let you run it yourself. We're betting SupplierSpy is one of them.
Read the full methodology, look at the changelog to watch the rubric evolve, and if you find editorial drift we missed, tell us at hello@supplierspy.com.