Staff/Principal Software Engineer – AI Platform Model Serving

Netflix

. Rebuild Netflix’s AI/ML model serving platform and infrastructure for real-time, asynchronous, and agentic workloads .

Posted 10/27/2026full-timeRemote • United StatesLead💰 $890,000 - $1,690,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates extensive expertise in AI/ML infrastructure, focusing on scalable Foundation Model serving solutions and distributed systems. Proven ability to lead architectural decisions, optimize GPU/CPU inference, and communicate complex technical concepts effectively.

Highest-signal resume keywords

AI/ML Infrastructure ExpertiseFoundation Model Serving SolutionsNVIDIA Triton Inference ServerKubernetes Scaling ExperienceDistributed Systems Architecture

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI Model Optimization TechniquesInference Latency ReductionGPU/CPU Inference OptimizationHigh-Traffic Distributed ServicesArchitecting Systems at Extreme ScaleModel Gateways and Intelligent RoutingEvaluation and Benchmarking Tools for LLMsNVIDIA CUDA ProgrammingPyTorchTensorFlow

Soft Skills

Influencing Architectural DecisionsCommunication of Complex Technical ConceptsCollaboration with Product Managers and Stakeholders

Tools & Technologies

AWSAzureGCPNVIDIA NsightHugging Face TransformersDeepSpeed-InferenceTensorRT-LLMSGLangVLLMNCCL

Industry Keywords

AI/ML Model Inference InfrastructureHigh AvailabilitySecurityPerformanceLong-Term Technical Roadmaps

Tech Stack

Tools & technologies

AWSAzureDistributed SystemsGoogle Cloud PlatformKubernetesPyTorchTensorflow

About the role

Key responsibilities & impact

Rebuild Netflix’s AI/ML model serving platform and infrastructure for real-time, asynchronous, and agentic workloads
Unify in-house inference infrastructure, third-party model hosting, and external LLM providers under a serving, routing, and experimentation layer
Enable consistent deployment, routing, and A/B testing of models regardless of hosting location
Re-engineer GPU capacity provisioning and utilization, including repurposing off-peak production capacity for internal coding and developer productivity use cases
Lead technical architecture and direction for Netflix’s GenAI and Foundation Model serving strategy
Address disaggregated serving, intelligent routing, speculative decoding, prompt caching, and multi-GPU inference
Embed AI capabilities into member-facing products at global scale
Build durable, resumable execution for long-running agentic workloads
Set and own technical direction for a growing engineering organization
Guide architectural decisions and alignment across multiple teams
Shape long-term technical roadmaps and ensure scalability, maintainability, and business alignment
Partner with product managers, operations, and executive stakeholders
Produce architecture decision records and technical proposals

Requirements

What you’ll need

BS/MS/PhD in Computer Science, Applied Math, Engineering, or a related field
12+ years of relevant experience in AI/ML Infrastructure and distributed systems
Experience with high-traffic distributed services and AI/ML model inference infrastructure
Experience building scalable Foundation Model serving solutions
Practical skills reducing inference latency and costs
Familiarity with NVIDIA Triton Inference Server, vLLM, or SGLang
Experience scaling AI/ML models via Kubernetes
Experience with AWS, Azure, or GCP
Deep expertise in distributed systems and high-performance RPC
GPU/CPU inference optimization, control planes, and experimentation infrastructure
Fluency with modern LLM serving stacks
Track record architecting systems at extreme scale
Experience with high availability, security, and performance
Ability to influence architectural decisions across multiple teams
Ability to shape long-term technical roadmaps
Experience partnering with product managers, operations, and executive stakeholders
Ability to communicate complex technical concepts and produce ADRs and proposals
Open-source contributions in AI/ML, distributed systems, or infrastructure (preferred)
Hands-on AI model building and optimization techniques such as quantization, model compression, or model pruning (preferred)
Proficiency with AI coding tools (preferred)
Experience with model gateways and intelligent routing (preferred)
Exposure to multi-adapter model serving, disaggregated prefill/decode, or large-scale KV cache management (preferred)
Experience with evaluation and benchmarking tools for LLMs and AI agents (preferred)
Knowledge of GPU topology, NVIDIA CUDA programming, and machine learning compilers (preferred)
Knowledge of PyTorch, TensorFlow, JAX, Hugging Face Transformers, vLLM, SGLang, TensorRT-LLM, TGI, DeepSpeed-Inference, NVIDIA Dynamo, CUDA, NCCL, and NVIDIA Nsight

Benefits

Comp & perks

Health Plans
Mental Health support
401(k) Retirement Plan with employer match
Stock Option Program
Disability Programs
Health Savings and Flexible Spending Accounts
Family-forming benefits
Life and Serious Injury Benefits
Paid leave of absence programs
Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off
Full-time salaried employees are immediately entitled to flexible time off