FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Tech Stack
Tools & technologiesJavaScriptPythonTypeScript
About the role
Key responsibilities & impact- Evaluate the quality of interactions with modern coding agents such as OpenAI Codex and Claude Code
- Assess AI-generated coding interactions end to end
- Judge whether outputs are useful, correct at a high level, and aligned with strong engineering judgment
- Assess explanations and reasoning, not just code
- Distinguish between levels of response quality
- Provide clear, opinionated written feedback on what worked, what did not, and what felt misleading
- Help define what excellent interaction looks like with Cursor, Codex, and Claude Code
- Make subjective but rigorous judgments about AI coding-agent behavior
Requirements
What you’ll need- Senior, Staff, or Principal-level engineer (or equivalent experience)
- Strong background in TypeScript/JavaScript or Python
- Hands-on experience with at least one of OpenAI Codex, Claude Code, or Cursor
- Deep familiarity with modern AI-assisted development workflows
- Able to evaluate code without executing it or reviewing every line
- Strong written and spoken English (B2 or above); required for detailed written feedback and short video explanations
- Comfortable giving direct, opinionated feedback
- High standards for engineering quality
- Prior exposure to prompt design or evaluation workflows (nice to have)
- Experience mentoring senior engineers or defining engineering standards (nice to have)
- Must complete a take-home evaluation exercise with a recorded Loom walkthrough
- Must agree to a simple background check for the project
Benefits
Comp & perks- Flexible hours, 10–20 hours/week
- Ongoing project opportunities for evaluators who do well
- Start as soon as the take-home is cleared and a project has an open seat