Software Engineer L5/L6 – Model Evaluations, Data Curation

Netflix

. Design and build shared data curation infrastructure, including reusable components, libraries, and workflows .

Posted 10/7/2026full-timeRemote • United StatesMid-LevelSenior💰 $600,000 - $1,066,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building scalable LLM-driven data transformation pipelines and reusable infrastructure in Python, with a strong focus on data quality methods and collaboration with cross-functional teams. Proficient in dataset versioning, artifact management, and optimizing large-scale inference processes.

Highest-signal resume keywords

Python Software EngineeringLLM-Driven Data TransformationData Quality MethodsDistributed Data ProcessingTechnical Leadership

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonData Transformation PipelinesData Quality MethodsSampling StrategiesFilteringDeduplicationLLM Evaluation SystemsDataset VersioningLarge-Scale Inference OptimizationHuman Annotation Workflows

Soft Skills

Collaboration Skills

Tools & Technologies

SparkRayMetaflowAirflow

Industry Keywords

Recommendation SystemsPersonalizationSearchContent CatalogMetadata

Tech Stack

Tools & technologies

AirflowPythonRaySpark

About the role

Key responsibilities & impact

Design and build shared data curation infrastructure, including reusable components, libraries, and workflows
Build scalable LLM-driven data transformation pipelines using raw sources such as the Netflix catalog and metadata
Use large-scale batch inference with attention to quality and token cost
Develop sampling strategies covering coverage, diversity, difficulty, and balance across content and member segments
Develop filtering and quality-control methods, including LLM-as-judge, evaluation-model-based scoring, deduplication, and validation
Partner with researchers to measure how curation choices affect model performance
Make curated datasets discoverable artifacts with versioning, explicit lineage, and reproducibility
Drive adoption of shared curation practices across MEDC and partner modeling teams

Requirements

What you’ll need

Strong software engineering in Python, with experience building reusable infrastructure, libraries, or frameworks used by other engineers and researchers
Experience building LLM-driven data generation or transformation pipelines, including synthetic data, structured outputs, or batch inference at scale
Hands-on experience with data quality methods: sampling strategies, filtering, deduplication, and model-based quality scoring such as LLM-as-judge
Understanding of how data choices affect model behavior and ability to design experiments that measure it
Experience with distributed data processing such as Spark, Ray, or similar
Excellent collaboration skills, particularly with researchers, data scientists, and platform teams
Experience with LLM evaluation systems (must-have for L6)
Technical leadership across data and evaluation infrastructure; experience setting technical direction for a multi-engineer effort (must-have for L6)
Experience with dataset versioning, lineage, and artifact management
Experience optimizing cost and throughput for large-scale LLM inference
Experience with human annotation workflows and calibrating LLM judges against human raters
Experience with pipeline orchestration frameworks such as Metaflow, Airflow, or similar
Background in recommendation systems, personalization, search, or working with content catalog and metadata

Benefits

Comp & perks

Health Plans
Mental Health support
401(k) Retirement Plan with employer match
Stock Option Program
Disability Programs
Health Savings and Flexible Spending Accounts
Family-forming benefits
Life and Serious Injury Benefits
Paid leave of absence programs
Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off
Full-time salaried employees are immediately entitled to flexible time off