FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in analyzing Voice AI Agents and conversational AI workflows, with strong capabilities in SQL and Python for data analysis. Proficient in building monitoring systems and evaluation frameworks to assess AI quality and operational impact.
Highest-signal resume keywords
5+ Years Experience In Product AnalyticsStrong SQL SkillsPython For Analytical InvestigationsExperience With ML PipelinesFluent Russian
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Product AnalyticsData AnalyticsOperational AnalyticsAI Quality MetricsLLM EvaluationMonitoring SystemsAnalytical FrameworksOperational DashboardsPrompt IterationAI Evaluation Frameworks
Soft Skills
Strong Communication SkillsAbility To Work In Ambiguous Environments
Industry Keywords
Conversational AIAI AgentsAI Performance VisibilityQuality DegradationOperational EfficiencyPatient ExperienceCall Transcripts AnalysisData PatternsFailure ModesAI Initiatives
Tech Stack
Tools & technologiesPythonSQL
About the role
Key responsibilities & impact- Analyze production behavior of Voice AI Agents and conversational AI workflows
- Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI-driven conversations
- Analyze call transcripts, logs, generated outputs, and operational datasets to uncover patterns and actionable insights
- Design evaluation frameworks for measuring AI quality, reliability, and business impact
- Build monitoring systems and dashboards for AI performance visibility
- Measure AI impact on operational efficiency, business outcomes, and patient experience
- Detect quality shifts, anomalies, and emerging failure modes in production environments
- Partner with Product, Engineering, QA, and Operations teams on AI initiatives
- Support rapid decision-making in fast-changing and ambiguous environments
Requirements
What you’ll need- 5+ years of experience in Product Analytics, Data Analytics, Operational Analytics, or analytics roles within complex production systems
- Strong SQL skills
- Python for analytical investigations and working with large-scale text datasets
- Experience working with ML pipelines or AI-powered products
- Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, and AI failure modes
- Experience building monitoring systems, analytical frameworks, or operational dashboards
- Strong communication skills and ability to translate analysis into product decisions
- Ability to work effectively in ambiguous and fast-changing environments
- Fluent Russian
- English level B2 or higher
- Ability to work CET-overlapping hours
- Experience with conversational AI or AI agents would be a plus
- Experience evaluating LLM outputs would be a plus
- Experience with AI quality metrics and evaluation methodologies would be a plus
- Experience with prompt iteration and AI evaluation frameworks would be a plus
Benefits
Comp & perks- Competitive compensation and clear growth opportunities
- Health insurance after the probation period
- Sports & wellness compensation
- Career growth opportunities
- Personalized English lessons via Preply
- 19 working days of vacation per year
- 4 additional wellness days
- Paid sick leave for the first 5 working days
- Thoughtful gifts for important life events
- Fully remote, long-term B2B collaboration