Senior Applied AI/ML Scientist

Sprout Social, Inc.

. Develop agents and agentic features with engineering teams .

Posted 9/29/2026full-timeRemote • CanadaSenior💰 CA$140,700 - CA$177,900 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates extensive experience in AI/ML model evaluation, development, and validation, with a strong focus on designing evaluation frameworks and conducting rigorous experiments. Proficient in Python, SQL, and common ML frameworks, with a commitment to improving AI quality and responsible practices.

Highest-signal resume keywords

Applied AI/ML ExperienceModel Selection and ValidationA/B Testing and Statistical SkillsEvaluation Framework DesignSQL Proficiency

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI/ML Model EvaluationModeling KnowledgePythonSQLA/B TestingStatistical AnalysisTransformersEmbeddingsEvaluation FrameworksAutomated Evaluators

Soft Skills

Strong JudgmentCommunicationCollaboration

Tools & Technologies

DatadogPhoenixMLOps

Industry Keywords

LLM SystemsAgentic ProductsQuality StandardsBias MitigationResponsible AI

Tech Stack

Tools & technologies

PythonSQL

About the role

Key responsibilities & impact

Develop agents and agentic features with engineering teams
Define agent quality standards, evaluations, judges, guardrails, and required tools
Lead model selection, development, and validation
Design and run A/B tests and other rigorous experiments to measure customer impact
Set and evolve quality standards across agents
Partner with engineers building and operating agents and evaluation infrastructure
Improve agent-platform capabilities through prompt iteration, reusable judges, context-window management, tool use, and adoption of state-of-the-art models
Communicate AI capabilities, limitations, and quality to product, design, engineering, and leadership
Own evaluation and quality for agents, drive evidence-based model-selection and improvement decisions, and contribute to AI quality roadmaps and methodologies

Requirements

What you’ll need

5+ years of applied AI/ML experience with a strong track record evaluating, measuring, and improving shipped AI/ML or LLM systems with measurable impact
Experience designing evaluation frameworks, building and calibrating judges or automated evaluators, and establishing reliability for ML or LLM/agentic systems
Strong judgment in model selection and validation
Rigorous experimentation and statistical skills, including A/B testing and power analysis
Foundational ML and modeling knowledge, including Python, common ML frameworks, transformers, and embeddings
SQL proficiency and comfort with large datasets
Ability to work closely with product, design, and engineering and translate technical quality questions for stakeholders
Preferred: experience evaluating production LLM-driven or agentic products
Preferred: experience defining quality bars or evaluation methodologies
Preferred: rapid prototyping with new AI/ML techniques
Preferred: familiarity with eval tooling, observability (e.g. Datadog, Phoenix), and MLOps
Preferred: experience influencing product roadmaps with evidence about AI quality and feasibility
Preferred: awareness of model security, bias mitigation, and responsible-AI practices
Must be based in Alberta, British Columbia, or Ontario, Canada

Benefits

Comp & perks

100% employer-paid health benefits, including medical, dental, vision, life insurance, and disability for the employee and eligible dependents
25 vacation days annually
5 paid sick days
All public holidays
Company-wide Rest & Recharge days
Free access to Modern Health, including coaching, therapy sessions, and digital wellness resources
$950 CAD annual Lifestyle Spending Account for physical, mental, and financial well-being
One-time $550 USD-equivalent home office setup stipend
Monthly $50 USD-equivalent internet stipend
No-cost confidential access to financial experts through Your Money Line
Subsidized childcare and eldercare options through Care.com
Company match for donations to eligible organizations