Data Scientist

• Design AI evaluations by defining ground truth, metrics, and scoring methods for models and agents
• Build repeatable evaluation loops to track quality over time and catch regressions before release
• Partner with engineering and analytics engineering to operationalize evaluation harnesses
• Translate evaluation results into actionable recommendations for system improvements
• Establish evaluation guidelines, documentation, and review practices
• Mentor and align teams on evaluation best practices and measurable AI quality
• Partner with Product, Engineering, Analytics Engineering, and business stakeholders to define quality standards and drive iteration
• Own the quality bar independently of the teams that build and optimize the systems

Explore Senior Data Scientist – AI Evaluation at Alpaca. Review requirements and application details.