FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
AI Research Scientist – Scientific Evaluation, ML/AI
Lightly. Critically evaluate AI-generated ML/AI research papers for scientific contributions, technical correctness, methodology, experimental design, and conclusions .
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in evaluating AI-generated ML/AI research through rigorous assessment of scientific contributions, technical correctness, and experimental design. Possesses a strong background in machine learning methodologies and a proven publication record in top-tier conferences.
Highest-signal resume keywords
PhD In Machine LearningPeer-Reviewed Publications At NeurIPSExperimental Design And AnalysisScientific Methodology And RigorResearch Paper Review Experience
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Machine LearningArtificial IntelligenceStatistical ValidityBenchmark DesignAblation StudiesReproducibilityTechnical Soundness AssessmentAnalytical SkillsScientific WritingEvaluation Metrics
Soft Skills
Attention To DetailIndependent JudgmentCritical Thinking
Industry Keywords
NeurIPSICMLICLRScientific ContributionsExperimental EvidenceMethodological FlawsNovelty ClaimsEvaluation RubricsResearch MethodologyPeer Review
About the role
Key responsibilities & impact- Critically evaluate AI-generated ML/AI research papers for scientific contributions, technical correctness, methodology, experimental design, and conclusions
- Assess scientific novelty and significance against existing state-of-the-art research
- Evaluate experimental rigor and validity, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies
- Identify methodological flaws, unsupported claims, inconsistencies, or misleading results
- Assess reproducibility and technical soundness using experimental evidence, code, and implementation details where available
- Review and verify relevant academic literature, prior work, citations, and novelty claims
- Evaluate whether AI-generated research meets the standards of top-tier ML/AI conferences such as NeurIPS, ICML, and ICLR
- Provide structured, evidence-based assessments covering strengths, weaknesses, critical technical concerns, and overall scientific merit
- Apply detailed evaluation rubrics consistently with clear, well-reasoned justifications
- Provide expert human judgment on whether research is scientifically valid, genuinely novel, and comparable to leading ML/AI conference work
Requirements
What you’ll need- PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical field, or equivalent demonstrated research expertise
- At least 3 peer-reviewed research papers at NeurIPS, ICML, and/or ICLR (hard requirement), ideally including first-author or significant research contributions
- Substantial hands-on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original scientific work
- Strong understanding of scientific methodology and experimental rigor, including appropriate baselines, statistical validity, benchmark design, ablation studies, and reproducibility
- Ability to independently assess the novelty, significance, and technical soundness of advanced ML/AI research
- Experience reviewing research papers for top-tier ML/AI conferences or journals, ideally NeurIPS, ICML, or ICLR — strongly preferred
- Familiarity with state-of-the-art ML/AI research and established benchmarks
- Ability to verify scientific claims, including reviewing technical arguments, experimental evidence, citations, and code implementations where relevant
- Strong analytical and scientific writing skills
- Ability to apply detailed evaluation guidelines and scoring rubrics while exercising independent expert judgment
- Exceptional attention to detail
- CV and brief note describing research background, areas of expertise, and publication record
- Links to at least 3 peer-reviewed publications at NeurIPS, ICML, and/or ICLR
- Relevant academic peer-review experience, if applicable
Benefits
Comp & perks- Fully remote and flexible — work from anywhere
- Part-time contractor role with flexible hours
- Work on the evaluation of cutting-edge autonomous AI systems capable of conducting end-to-end scientific research
- Apply your research expertise to help assess and advance the scientific capabilities of frontier AI agents