FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates strong analytical and critical-thinking skills to evaluate AI models effectively, with a focus on attention to detail and clear written communication of observations and preferences.
Highest-signal resume keywords
Bachelor's DegreeAttention To DetailStrong Written Communication SkillsCritical-Thinking SkillsFamiliarity With AI Assistants
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Soft Skills
Excellent Listening SkillsComprehension SkillsReliabilityConsistency
Industry Keywords
Voice-Based AIConversational SystemsEvaluation GuidelinesError ClustersConversational Quality
About the role
Key responsibilities & impact- Review an assigned scenario and roleplay a natural conversation with two different AI models
- Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible
- Maintain approximately the same number of conversational turns with each model to support a fair comparison
- Record your voice during each interaction and assess the quality and behavior of the models' responses
- Evaluate each model across five defined evaluation dimensions
- Identify and categorize relevant error clusters in the audio or conversation
- Compare Model A and Model B based on the evaluation criteria and observations
- Select the model demonstrating stronger overall performance
- Write a detailed rationale explaining the final preference with specific examples and observations
- Apply evaluation guidelines consistently across scenarios and model interactions
Requirements
What you’ll need- Bachelor's degree
- Strong attention to detail and ability to notice subtle differences in conversational quality
- Excellent listening and comprehension skills
- Strong written communication skills, particularly the ability to explain observations clearly and objectively
- Ability to consistently follow detailed evaluation guidelines
- Comfort speaking naturally and role-playing different conversational scenarios
- Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation
- Strong critical-thinking and analytical skills
- Reliability and consistency when completing structured evaluation tasks
- Familiarity with AI assistants, voice-based AI, or conversational systems
Benefits
Comp & perks- W2 employment
- Full-time contract for 3 months
- 40 hours per week