Methodology
Scores need receipts.
Ruling separates two signals: moderated community reviews from people using the tools, and clearly labeled editorial benchmarks produced by Ruling. Both can inform a score, but they are never presented as the same kind of evidence.
Aggregate score
overall = average(task fit, reliability, cost efficiency, ease of setup, drift score)
Source reviews use 1–5 dimension inputs. Public scores are shown on a /10 scale for readability. Agent pages also show review counts, excerpts, labels, and comparison links so readers can inspect the underlying signal instead of trusting a naked number.
Editorial disclosure
Benchmarks are starter research.
Editorial benchmarks exist so pre-launch pages are useful before the community graph is deep. They are not fake users, vendor testimonials, paid placements, or claims that a broad market consensus already exists.
What the dimensions mean
Five questions behind every score.
Task fit
How well the agent handles the workflow it is marketed for, not a generic all-purpose score.
Reliability
Whether outputs stay useful across repeated runs, edge cases, and realistic project constraints.
Cost efficiency
Whether the result quality justifies seat, token, credit, or usage-based pricing.
Ease of setup
How quickly a real team can install, configure, and get value without custom infrastructure.
Drift score
How well the agent stays inside the requested task instead of over-editing, hallucinating, or wandering.
Editorial rules
How Ruling keeps benchmark content honest.
Editorial coverage should make the catalog useful without pretending the site has more community activity than it does. The label stays visible in review copy, reviewer profiles, moderation notes, and aggregate-count language.
- 1Editorial benchmarks are labeled as Ruling research, not community testimonials.
- 2They are calibrated from public product documentation, pricing pages, changelogs, demos, and visible buyer/developer sentiment.
- 3No vendor can buy a score, placement, badge, or favorable comparison.
- 4Scores are conservative when pricing is opaque, claims are hard to verify, or reliability depends on a narrow setup.
- 5Vendors and users can challenge facts; corrections should update the affected agent page and benchmark note.
Community signal still wins over time.
As real reviews arrive, community evidence becomes the stronger signal because it captures lived workflows, unexpected failure modes, and buyer context that public research cannot fully observe. Editorial benchmarks remain useful as transparent baseline research and are updated when product facts change.
Challenge a fact or request a correction →