FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Tech Stack
Tools & technologiesSwitching
About the role
Key responsibilities & impact- Interact with different AI models and tools across a wide range of prompts and topics
- Compare two or more AI-generated responses side by side
- Assess accuracy, helpfulness, instruction-following, clarity, and overall response quality
- Follow detailed rating guidelines and multi-step evaluation workflows
- Write clear, well-reasoned explanations supporting evaluation decisions
- Apply evaluation criteria consistently across tasks
- Help improve the accuracy, helpfulness, and reliability of AI systems
Requirements
What you’ll need- Fluency in English, with strong reading comprehension and writing skills
- Extensive hands-on experience using LLMs and critically assessing their outputs
- Strong analytical judgment and ability to weigh multiple dimensions of response quality
- Excellent attention to detail and ability to follow complex instructions carefully
- Ability to explain evaluation decisions clearly and convincingly
- Comfort switching between AI models, tools, and multi-step workflows
- Previous experience in AI response evaluation, data annotation, or quality assurance (nice to have)
- Experience in editing, research, fact-checking, or analytical writing (nice to have)
- Familiarity with evaluation rubrics and side-by-side comparison tasks (nice to have)
Benefits
Comp & perks- Remote work
- Part-time independent contractor arrangement
- Commitment of up to 3 months