Senior Applied AI/ML Scientist

Sprout Social, Inc.

. Own end-to-end agent and agentic feature development, including requirements, success definitions, evaluations, guardrails, and tools .

Posted 9/29/2026full-timeRemote • United StatesSenior💰 $168,000 - $210,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in AI/ML evaluation and quality assurance, with a strong focus on model selection, A/B testing, and statistical analysis. Proficient in collaborating with cross-functional teams to translate technical insights into actionable strategies for product improvement.

Highest-signal resume keywords

Applied AI/ML ExperienceEvaluation Framework DesignA/B Testing and Statistical SkillsPython and ML Frameworks ProficiencySQL and Large Dataset Management

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI/ML EvaluationModel SelectionA/B TestingStatistical AnalysisPythonML FrameworksTransformersEmbeddingsSQLMeasurement Design

Soft Skills

CommunicationCollaborationJudgmentRelationship Building

Tools & Technologies

Eval ToolingMLOps PracticesObservability Tools

Industry Keywords

LLM SystemsAgentic SystemsQuality Bar DefinitionBias MitigationResponsible AI Practices

Tech Stack

Tools & technologies

PythonSQL

About the role

Key responsibilities & impact

Own end-to-end agent and agentic feature development, including requirements, success definitions, evaluations, guardrails, and tools
Lead model selection, development, and validation
Design and run A/B tests and other rigorous experiments to measure customer impact
Set and evolve the quality bar across agents
Partner with engineers building and operating agents and evaluation infrastructure
Identify improvements to agent-platform capabilities, including prompt iteration, reusable judges, context-window management, tool use, and adoption of state-of-the-art models
Communicate AI capabilities, limitations, and quality to product, design, engineering, and leadership
Own evaluation and quality for agents and drive evidence-based model-selection and improvement decisions
Contribute to the long-term AI quality and evaluation roadmap and guide evaluation best practices
Build relationships with business-unit leaders, stakeholders, and executives
Elevate technical excellence and stay current with state-of-the-art research

Requirements

What you’ll need

5+ years of applied AI/ML experience with a strong track record in evaluation, measurement, and quality of shipped AI/ML or LLM systems
Experience designing evaluation frameworks and building and calibrating reliable judges or automated evaluators for ML or LLM/agentic systems
Strong judgment in model selection and validation
Rigorous experimentation and statistical skills, including A/B testing, power analysis, and sound measurement design
Foundational ML and modeling knowledge, including Python, common ML frameworks, transformers, and embeddings
SQL proficiency and comfort with large datasets
Ability to work closely with product, design, and engineering and translate technical quality questions into actionable terms
Direct experience evaluating LLM-driven or agentic products in production preferred
Experience defining quality bars or evaluation methodologies preferred
Rapid prototyping with new AI/ML techniques preferred
Familiarity with eval tooling, observability, and MLOps practices preferred
Experience influencing product roadmaps with evidence about AI quality and feasibility preferred
Awareness of model security, bias mitigation, and responsible-AI practices preferred

Benefits

Comp & perks

Premium BCBSIL medical, dental, and vision insurance for you and eligible dependents
Full, free access to Modern Health, including coaching, therapy sessions, and digital wellness resources
401(k) plan with a 50% company match on the first 6% of contributions
100% employer-paid Life and Disability insurance
Flexible PTO policy and company-wide Rest & Recharge days
Up to 16 weeks of paid parental leave
$1,000 USD annual Lifestyle Spending Account
One-time $550 USD home office setup stipend
Monthly $50 USD internet stipend
16 hours of paid volunteer time annually
$100 annual charitable donation match
Pre-tax commuter benefits
Subsidized child/eldercare through Care.com
Discounted pet insurance through Figo
No-cost personalized financial wellness support through Your Money Line