FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in managing large-scale multilingual data collection and evaluation projects, with a strong focus on AI/ML methodologies and technical project management. Proficient in SQL and experienced in optimizing project delivery and unit economics while effectively communicating with diverse stakeholders.
Highest-signal resume keywords
Technical Project ManagementLLM Training and EvaluationSQL ProficiencyAgile MethodologiesData Pipeline Management
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
SQLData Pipeline ManagementLLM TrainingEvaluation MethodologiesProject Unit EconomicsData AnnotationTechnical ScopingWorkflow AutomationData AnalysisRoot-Cause Analysis
Soft Skills
Strong Communication SkillsStakeholder ManagementProblem-Solving
Tools & Technologies
Business Intelligence ToolsAnnotation PlatformsAutomated Evaluation ToolsJiraAPIs
Industry Keywords
AI/MLMultilingual Data ProjectsHuman-in-the-Loop WorkflowsSFTRLHFLLM-as-a-JudgeTechnical RequirementsComplex Workflows
Tech Stack
Tools & technologiesPythonSQL
About the role
Key responsibilities & impact- Lead large-scale multilingual data collection, human-in-the-loop workflows, and LLM evaluation initiatives
- Manage Applied AI data collection and evaluation projects from technical scoping through implementation, validation, delivery, and retrospective review
- Translate technical AI requirements into structured data specifications, annotation requirements, evaluation plans, and acceptance criteria
- Manage timelines, throughput velocity, dependencies, delivery risks, and technical blockers
- Coordinate data collection and annotation for SFT, RLHF preference data, and model evaluation
- Partner with engineering teams to define and validate ingestion, annotation tooling, integrations, quality checks, data generation pipelines, and delivery workflows
- Operationalize hybrid evaluation methodologies combining human review, automated evaluation tooling, and LLM-as-a-judge frameworks
- Investigate data, pipeline, and tooling issues and coordinate resolution with technical owners
- Analyze throughput velocity, quality, unit economics, and supplier/delivery performance using SQL and business intelligence tools
- Define and monitor quality standards, lead root-cause analysis, and establish delivery validation checks
- Coordinate stakeholder acceptance of completed datasets, delivery targets, and evaluation results
- Translate complex technical requirements into actionable plans for global contributors, vendors, and specialized SMEs
- Communicate project status, risks, tradeoffs, unit economics, and operational risks to technical and non-technical stakeholders
- Use contributor feedback and evaluation findings to improve internal tooling, guidelines, and dataset pipelines
Requirements
What you’ll need- 3–5+ years of technical project management experience in AI/ML, data platforms, or technical data operations
- Strong understanding of LLM training and evaluation concepts, including SFT, RLHF, human evaluation, automated evaluations, LLM-as-a-judge, and red teaming
- Proficiency in SQL
- Working knowledge of data pipelines, structured data formats, APIs, and validation processes
- Experience partnering with engineers to manage technical requirements, dependencies, and issue resolution
- Proven ability to track and optimize project unit economics and delivery velocity
- Strong communication skills, including translating technical requirements for multilingual audiences and diverse global contributor networks
- Experience managing complex workflows using Agile, Scrum, or Kanban
- Preferred: experience using Python or scripting tools for data analysis, workflow automation, or evaluation scripting
- Preferred: experience managing specialized SMEs or contributor networks
- Preferred: experience with annotation platforms, automated evaluation tools, business intelligence tools, and Jira
- Preferred: experience delivering complex multilingual or multimodal data projects
- Preferred: background in computer science, data science, engineering, or equivalent practical experience
- Preferred: fluency in an additional language
- Must be authorized to work in the United States on a full-time basis without restriction (application question)
Benefits
Comp & perks- Global collaboration
- Growth opportunities
- Leading tools
- Human-in-the-loop reviews via LILT's global network of professional linguists
- Equal opportunity and inclusive hiring process
