Staff Machine Learning Engineer

Plenful

. Define the architecture for production agents, including tool use, retrieval, state and context management, routing, orchestration, tracing, and failure recovery .

Posted 10/15/2026full-timeSan Francisco • California • United StatesLeadWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in defining architecture for production agents, leading the development and evaluation of LLMs and vision-language models, and implementing MLOps practices. Proven ability to influence design decisions and mentor engineering teams while ensuring compliance with privacy standards in healthcare.

Highest-signal resume keywords

LLM Production ExperienceMLOps with Docker and KubernetesPython and PyTorch ProficiencyEvaluation Strategy DesignHealthcare Data Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Machine LearningModel TrainingSupervised Fine-TuningReinforcement LearningProduction Agent ArchitectureEvaluation StrategiesDataset VersioningDistributed TrainingExperiment TrackingPerformance Metrics Improvement

Soft Skills

MentoringInfluencing Without AuthorityTechnical Review Leadership

Tools & Technologies

LangGraphPineconeVLLMFSDPDeepSpeedMLflowCI/CD

Industry Keywords

Healthcare DataEHRFHIRClaimsPrior Authorization

Tech Stack

Tools & technologies

DockerKubernetesPythonPyTorch

About the role

Key responsibilities & impact

Define the architecture for production agents, including tool use, retrieval, state and context management, routing, orchestration, tracing, and failure recovery
Turn effective approaches into reusable engineering patterns
Make build-versus-buy decisions for models and platform components
Set the company’s post-training strategy
Train and ship LLMs and vision-language models for healthcare tasks
Build in-house capability for continued model training
Establish evaluation that demonstrates production gains
Set the AI quality strategy, including automated graders, calibrated LLM judges, regression suites, human review, failure taxonomies, and offline-to-online validation
Use evaluation results as the basis for release decisions
Design systems that turn production failures and user feedback into improved datasets, evaluations, and models
Set standards for dataset versioning, observability, and MLOps
Co-own the 12-month ML roadmap with engineering and product leaders
Lead technical reviews, mentor Senior engineers, and support hiring
Build privacy by design using minimum necessary data, de-identified or synthetic data where appropriate, and HIPAA-aware practices
Review agents, evaluations, and the training pipeline during the first 30 days and identify the three biggest gaps
Ship a post-training or evaluation improvement that moves a production metric by day 90
Publish the team plan for the following two quarters by day 90

Requirements

What you’ll need

8+ years in ML or software engineering, including 3+ years leading LLM or agentic systems in production
End-to-end applied post-training experience, including supervised fine-tuning, preference optimization, distillation, or reinforcement learning
Experience shipping a model that improved a production metric
Experience designing evaluation strategies for LLM or agent systems that drove release decisions
Experience architecting production agent systems
Experience with LangGraph, RAG, and vector search such as Pinecone
Strong Python and PyTorch
Comfort with distributed training and serving, such as vLLM, FSDP, or DeepSpeed
MLOps experience with Docker, Kubernetes, MLflow or equivalent, CI/CD, and experiment and dataset tracking
Ability to influence without authority through design documents, reviews, and mentoring
Current knowledge of new models and coding agents
Ability to turn papers and releases into experiments and report results with numbers
Evidence such as open-source work, write-ups, talks, or evaluations used by others
Nice to have: healthcare data experience with EHR, FHIR, claims, or prior authorization
Nice to have: publications or open-source work in LLM evaluation, post-training, or agents
Nice to have: experience interviewing and hiring ML engineers
Nice to have: work in another regulated, high-stakes domain

Benefits

Comp & perks

Flexible Hybrid Work Environment — remote-first with meaningful office presence in San Francisco and New York
Healthcare Coverage — full medical, dental, and vision insurance for you and participation for your family
401(k) with Company Match — Plenful matches 50% of your first 3% contributed
Equity — every full-time employee shares in the company’s success
Unlimited PTO
Daily Lunch Stipend — $100/week
Wellness Stipend — $100/month
Commuter Benefits — $100/month for San Francisco and New York-based employees
Paid Parental Leave
Opportunities for Growth
Mission-driven, world-class team
Professional collaboration with experienced, high-performing leaders