Senior SWE – Python, TypeScript

Weekday (YC W21)

. Design, refine, and iterate on rubrics, evaluation frameworks, and quality criteria for preference data.

Posted 10/20/2026contractRemote • United States, CanadaSenior💰 $100 - $150 per hourWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in Python and software engineering principles, with a strong focus on building and maintaining data pipelines and infrastructure. Capable of synthesizing data analysis findings into actionable insights while collaborating effectively with multidisciplinary teams.

Highest-signal resume keywords

Python ProgrammingData Pipeline DevelopmentSoftware Engineering PrinciplesAnalytical SkillsCommunication Skills

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

TypeScriptJavaScriptGoRubric CreationEvaluation Criteria DevelopmentCode QualityDebuggingMaintainable System DesignData AnnotationQuality Assessment

Soft Skills

Written CommunicationVerbal CommunicationIndependent WorkExecution PrioritizationProblem-Solving

Tools & Technologies

VS CodeOpen-Source CodebasesAutomated Evaluation SystemsDeveloper InfrastructureAI Model Evaluation

Industry Keywords

Preference DataQuality CriteriaData GenerationData ProcessingEvaluation Frameworks

Tech Stack

Tools & technologies

JavaScriptPythonTypeScriptGo

About the role

Key responsibilities & impact

Design, refine, and iterate on rubrics, evaluation frameworks, and quality criteria for preference data.
Review, assess, and label data while identifying subtle inconsistencies, quality issues, and opportunities for improvement.
Build, maintain, and optimize pipelines and infrastructure supporting data generation, collection, processing, and evaluation workflows.
Synthesize findings from data analysis and evaluation activities into concise reports, updates, recommendations, and actionable summaries.
Collaborate with researchers and engineers to translate qualitative judgments into structured, repeatable, and scalable processes.
Understand, contribute to, and maintain complex software repositories while ensuring code quality, reliability, and performance.
Identify workflow inefficiencies and implement practical improvements to data quality, evaluation consistency, and engineering processes.
Complete application screening, an AI-based interview, a coding challenge, a technical and cultural interview, and final candidate evaluation.

Requirements

What you’ll need

At least 5 years of professional software engineering experience with Python, particularly in building and maintaining pipelines and infrastructure, or with TypeScript, JavaScript, or Go.
Hands-on experience with preference data collection, rubric creation, or developing evaluation criteria.
Strong understanding of software engineering principles, code quality, debugging, and maintainable system design.
Excellent written and verbal communication skills, with the ability to explain complex technical concepts clearly and concisely.
Strong analytical skills and the ability to make consistent, well-reasoned quality judgments.
Demonstrated ability to explain why a particular solution, example, or implementation meets a high quality standard.
Comfort working independently in ambiguous environments while prioritizing execution and timely delivery.
Experience contributing to or navigating large-scale, complex open-source codebases, such as VS Code (preferred).
Experience building data pipelines, developer infrastructure, or automated evaluation systems (preferred).
Background in AI model evaluation, preference data workflows, data annotation, or quality assessment (preferred).
Experience collaborating with AI researchers, machine learning engineers, or multidisciplinary technical teams (preferred).
Education from a leading university or professional experience at a highly regarded technology company (preferred).
LinkedIn profile mandatory and must be included in the candidate's resume.
Must not currently be employed by Microsoft.
Must be able to commit to the required weekly hours and PST overlap for the full contract duration.