Production quality / v2
Grade production runs on the traffic you already have.
Evaluations is a capability of Signals, not a fourth product—and it is not a pre-production eval harness. It is designed to score production quality on live traffic (output_quality_score plus a PromptQualityScorer) and join that score against cost, so a cheaper run that got worse is never mistaken for a win.
ScopeLive traffic
JoinQuality × cost
FeedsSignals recommendations
StatusPositioning · v2
A quality signal, not a separate product.
A positioning page, not a shipped harness: every line below describes an accepted Signals capability planned for v2 — production-quality scoring on the traffic you already serve.
01 / score
Score the run you already shipped
An output_quality_score and a PromptQualityScorer grade production output on the traffic already flowing through Signals — not a separate offline test suite.
Planned: v2 capability02 / join
Join quality against cost
The quality score sits next to the cost and latency Signals already tracks, so a run that got cheaper but worse surfaces as a regression, not a win.
Planned: quality × cost join03 / recommend
Feed the recommendation, not a dashboard
The joined signal feeds Signals’ recommendations rather than opening a new surface to watch — Evaluations stays a capability, not a destination.
Planned: feeds Signals