Full research-grade evaluation is rarely available to shipping teams. What you need is a repeatable habit that catches regressions before customers do.
Start with twenty to fifty representative tasks labeled by the people who know the domain. Score with a mix of exact checks where possible and structured human review where judgment is required.
Run the suite on every meaningful prompt or retrieval change. Track a small dashboard: pass rate, latency budget adherence, and escalation rate to human review.
When scores drop, resist swapping models as the first move. Inspect failing clusters; many issues are context assembly bugs or unclear product rules rather than raw model quality.