Research Engineer

Gauntlet

. Build evaluation frameworks for AI agents, including datasets, sandbox environments and metrics measuring performance on real-world tasks .

Posted 11/6/2026full-timeRemote • United States, CanadaMid-LevelSenior💰 $175,000 - $195,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and evaluating AI agents, particularly LLM models, with strong Python programming skills and a solid foundation in software engineering principles. Capable of designing experiments, analyzing data, and collaborating across teams to enhance agent performance and reliability.

Highest-signal resume keywords

LLM Model DevelopmentPython ProgrammingApplied Research in Machine LearningExperiment Design and AnalysisAgent Performance Evaluation

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

LLM ModelsPythonSoftware Engineering FundamentalsExperiment DesignData AnalysisPerformance MetricsTesting and ReliabilityTool DevelopmentAgent HarnessesContext Management

Soft Skills

Independent Problem SolvingCollaborationAdaptabilityGenuine Interest in AI

Tools & Technologies

Sandbox EnvironmentsInternal ToolsIntegrationsAria

Industry Keywords

AI AgentsMachine LearningData ScienceEvaluation FrameworksResearch Questions

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

Build evaluation frameworks for AI agents, including datasets, sandbox environments and metrics measuring performance on real-world tasks
Design experiments comparing models, agent harnesses, tools and execution environments, identifying regressions and trade-offs between performance, reliability and cost
Turn qualitative feedback about agent behaviour into research questions, experiments and measurable outcomes
Investigate agent failures and determine whether issues arise from the model, harness, tooling or prompting
Develop ways to improve agent performance
Build and improve agent harnesses, internal tools and integrations, including contributing to Aria and other business agents
Develop shared knowledge and reusable skills across agents
Work with Infrastructure and Security to ensure reliable and safe production operation
Ship engineering improvements and integrations, then take on open-ended research into evaluation, agent behaviour and infrastructure

Requirements

What you’ll need

Hands-on experience building, evaluating or improving LLM models or agents, through professional work, applied research, open-source contributions or substantial personal projects
Strong Python skills and good software engineering fundamentals
Production-quality coding skills and understanding of testing, maintainability and reliability
Experience with applied research, machine learning or data science, including designing experiments, working with datasets and drawing conclusions from results
Understanding of LLM agents beyond prompting, including tool use, harnesses, context management and common failure modes
Ability to break open-ended problems into testable questions and work independently
Experience building tools or systems used by others and evaluating their effectiveness beyond technical benchmarks
Comfort working across research and engineering, collaborating with different teams and adapting as priorities change
Genuine interest in AI agents
Candidates able to work within the contiguous United States and Canada
Advanced degree or postgraduate research experience in computer science, mathematics, statistics, machine learning or a related field is nice to have, not required

Benefits

Comp & perks

Remote first — work from anywhere in the United States
Regular in-person company retreats and cross-country "office visit" perk
100% paid medical, dental, and vision premiums for employees
$1,000 WFH stipend
Monthly reimbursement for home internet, phone, and cellular data
Unlimited vacation
100% paid parental leave of 12 weeks
Fertility benefits
Opportunity for incentive compensation
Equity in the company