Software Engineer, Agent Optimization – Evaluation

Reddit, Inc.

. Lead Reddit's agent optimization and evaluation platform .

Posted 10/19/2026full-timeRemote • United StatesMid-LevelSenior💰 $109,800 - $325,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates extensive experience in building and shipping large-scale production software systems, with a strong focus on designing evaluation and benchmarking systems for AI agents. Capable of driving technical direction and collaborating across engineering and infrastructure teams to optimize performance and productivity.

Highest-signal resume keywords

Large-Scale Production Software SystemsAI Agent Evaluation SystemsCross-Functional CollaborationModern LLMs UnderstandingQuality and Latency Optimization

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Software Engineering FundamentalsBenchmarking Systems DesignExperimentation FrameworksModel Routing StrategiesSynthetic Task GenerationRegression DetectionAutomated EvaluationAgent Performance MeasurementDeveloper Productivity MetricsEngineering Velocity Measurement

Soft Skills

Independent OperationAmbiguous Environment NavigationTechnical Direction Driving

Industry Keywords

AI AgentsEngineering WorkflowsPlatform EngineeringInfrastructureDeveloper Productivity

Tech Stack

Tools & technologies

Distributed Systems

About the role

Key responsibilities & impact

Lead Reddit's agent optimization and evaluation platform
Design and build an agent optimization harness for evaluating, benchmarking, and improving AI agents across software engineering workflows
Develop Reddit-specific evaluation environments for measuring agent performance against meaningful engineering tasks
Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback
Design model routing and orchestration strategies optimizing quality, latency, and cost
Define metrics and experimentation frameworks measuring developer productivity and engineering velocity
Build tooling to analyze agent behavior, investigate failures, and accelerate iteration
Partner with Platform Engineering, Infrastructure, and Developer Productivity teams
Shape Reddit's long-term strategy for a flexible, model-agnostic agent platform

Requirements

What you’ll need

5-12+ years of engineering experience
Extensive experience building and shipping large-scale production software systems
Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems
Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively
Experience building systems that optimize across quality, latency, reliability, and cost
Excellent software engineering fundamentals
Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives
Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations

Benefits

Comp & perks

Comprehensive Healthcare Benefits and Income Replacement Programs
401k with Employer Match
Global Benefit programs that fit your lifestyle, from workspace to professional development to caregiving support
Family Planning Support
Gender-Affirming Care
Mental Health & Coaching Benefits
Flexible Vacation & Paid Volunteer Time Off
Generous Paid Parental Leave
Equity in the form of restricted stock units
Medical, dental, and vision insurance
Flexible vacation and generous time off for vacation
Parental leave