Product Manager, Inference Platform

Baseten

. Work with inference engineering and research teams to bring new performance capabilities to customers .

Posted 11/11/2026full-timeUnited StatesMid-LevelSenior💰 $200,000 - $400,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and shipping AI/ML products with a focus on performance optimization, scalability, and production behavior. Proficient in benchmarking, experimentation, and translating technical capabilities into customer value.

Highest-signal resume keywords

AI/ML Product DevelopmentModel Serving ArchitecturesPerformance BenchmarkingPyTorchNVIDIA Dynamo

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Performance OptimizationLatency MeasurementThroughput MeasurementQuality AssessmentCost AnalysisModel OptimizationExperimentationTechnical Translation

Soft Skills

Product JudgmentCustomer Understanding

Tools & Technologies

VLLMTensorRT-LLMSGLang

Industry Keywords

Inference EngineeringResearch CollaborationOpen-Source Inference ProjectsGPU Economics

Tech Stack

Tools & technologies

PyTorch

About the role

Key responsibilities & impact

Work with inference engineering and research teams to bring new performance capabilities to customers
Translate advances in batching, caching, quantization, speculative decoding, and serving architectures into useful product features
Define benchmarks and evaluations demonstrating improvements in latency, throughput, quality, and cost
Understand customer workloads and prioritize high-impact optimizations
Make performance features easier to configure, evaluate, and adopt in production
Help customers understand tradeoffs across hardware, model configurations, autoscaling, and routing
Partner with engineering on release readiness, compatibility, and regression testing
Enable customers to adopt Baseten’s latest inference optimizations without becoming experts in the underlying runtime

Requirements

What you’ll need

Experience building and shipping AI/ML products or infrastructure, particularly where performance, scalability, or production behavior mattered
A technical background in software engineering, ML engineering, research, or equivalent hands-on experience
Strong understanding of model serving architectures, GPU economics, and latency, throughput, quality, and cost tradeoffs
Experience working closely with researchers or infrastructure engineers to bring technically complex capabilities into production
A rigorous approach to benchmarking, experimentation, and performance measurement
Strong product judgment and the ability to translate deep technical work into clear customer value
Hands-on experience with PyTorch, vLLM, TensorRT-LLM, SGLang, NVIDIA Dynamo, or similar technologies (nice to have)
Experience with open-source inference projects, model optimization, or ML systems research (nice to have)
Unrestricted work authorization in the United States
Must not require employment-based visa sponsorship

Benefits

Comp & perks

Competitive compensation, including meaningful equity
(U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
Flexible PTO policy including company wide Winter Break
Paid parental leave
Fertility and family-building stipend through Carrot
(U.S. only) Company-facilitated 401(k)
Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities