Senior Software Engineer, AI Evaluation Infrastructure
Turing
Posted: 10/19/2026DjangoDockerKubernetesPostgresPythonReactTypeScript
AI Engineer 5 – AI Foundations
Capital One
Posted: 10/19/2026AWSCloudJavaPythonPyTorchScalaC++Go
Senior Full Stack Software Engineer, AI & Cloud Platforms
LPL Financial
Posted: 10/19/2026AngularAWSCloudDjangoFlaskJavaJavaScriptMicroservicesNoSQL+4 more
AI Engineer
Capital One
Posted: 10/19/2026AWSCloudDistributed SystemsJavaPythonPyTorchScalaC++Go
Senior Lead, AI Enablement
Wells Fargo
Posted: 10/19/2026AI Engineer 5, MLX, Agentic AI, Gen AI Platform Services
Capital One
Posted: 10/19/2026AWSCloudJavaPythonPyTorchScalaGo
Principal AI Engineer – AI Acceleration, Enterprise Enablement
U.S. Bank
Posted: 10/19/2026AzureCloudPython
IT Director, Data & AI Architecture
Abbott
Posted: 10/19/2026AzureCloudKafka
Senior Forward Deployed Engineer, AI Enablement
Reddit, Inc.
Posted: 10/19/2026Cloud
Lead Forward Deployed Engineer, AI Enablement
Reddit, Inc.
Posted: 10/19/2026
Senior Agentic AI Engineer – Core Platforms
American Express
Posted: 10/19/2026CloudDistributed SystemsJavaKubernetesPythonC++
AI Platform Administrator
Sedgwick
Posted: 10/19/2026ITSMLinuxMacOSPythonServiceNow
Senior AI Engineer
Sedgwick
Posted: 10/19/2026AWSPythonReactSQLTerraform
AI Engineer
Sedgwick
Posted: 10/19/2026AirflowAWSAzureCloudDockerETLPySparkPythonSQL
IT Director – Data & AI Architecture
Abbott
Posted: 10/19/2026AirflowApacheAzureCloudKafkaKubernetes
Senior AI Engineer
SchoolsFirst Federal Credit Union
Posted: 10/19/2026AWSAzureCloudPythonRPASQL
Lead AI Engineer
Eastern Bank
Posted: 10/19/2026AzureCloud
Lead AI Engineer
Eastern Bank
Posted: 10/19/2026AzureCloud
FP&A AI Architect
Gusto
Posted: 10/19/2026ERPFluxPythonSQL
Functional Lead, AI & Agentforce Delivery
NeuraFlash
Posted: 10/19/2026AWSCloud
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in building and operating production backend systems with a strong focus on Python, Django, and PostgreSQL. Capable of integrating models and improving system performance while ensuring high-quality evaluations and clear technical documentation.
Highest-signal resume keywords
Strong PythonDjango FrameworkPostgreSQLLLM EvaluationTechnical Writing
Tech Stack
Tools & technologiesAbout the role
Key responsibilities & impact- Design and ship end-to-end features across the data model, API, background workers, and UI
- Build and improve agent and model evaluations in sandboxed environments
- Score evaluation results and turn thousands of runs into trustworthy results
- Improve benchmark task quality checks using verifiers, rubric-based grading, and LLM-as-a-judge grading
- Integrate new models, agents, and benchmarks through generalizable designs
- Improve system performance and reliability through efficient queries, safe migrations, and robust concurrency
- Review teammates’ pull requests against engineering guidelines
- Deliver tested, maintainable changes and independently own system areas
Requirements
What you’ll need- Typically 5+ years building and operating production backend systems
- Ownership of at least one service from design through on-call
- Strong Python
- Experience with Django or a similar framework, PostgreSQL, and a task queue such as Celery
- Hands-on experience in LLM evaluation, including benchmark task construction, agent trial scoring, and result aggregation/comparison
- Judgment about task quality, including ambiguous instructions, weak or gameable verifiers, leaked answers, and invalid checks
- Ability to make sound tradeoffs and choose the smallest effective design
- Ownership of migrations, tests, and edge cases
- Ability to quickly adopt team conventions for naming, structure, tests, and error handling
- Comfort with TypeScript and React frontend work
- Clear technical writing in pull request descriptions and design notes
- Nice to have: Docker, Kubernetes, Modal, Daytona, LLM-as-a-judge, rubric-based grading, Claude Code, Codex, OpenHands, MCP, distributed job scheduling, concurrency limits, database locking at scale, and open-source evaluation or infrastructure contributions
Benefits
Comp & perks- Work at the frontier of AI, helping leading AI labs improve advanced models by building expert datasets, RL environments, and benchmarks
- Contribute to leading-edge AI research and showcase work at conferences such as ICLR, ICML, and NeurIPS
- Bring frontier AI innovation to enterprise challenges
- Collaborate with and learn from colleagues with deep AI experience from Google, Meta, Amazon, and other leading technology companies
- Startup-speed environment with speed, ownership, and impact
- Equal opportunity, diverse, inclusive, and authentic workplace
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Backend Systems DevelopmentBenchmark Task ConstructionResult AggregationTask Queue ManagementConcurrency Management
Soft Skills
Judgment About Task QualityAbility to Make TradeoffsAdaptability to Team Conventions
Tools & Technologies
CeleryTypeScriptReactDockerKubernetes
Industry Keywords
LLM-as-a-JudgeRubric-Based GradingDistributed Job SchedulingDatabase Locking at ScaleOpen-Source Contributions