Senior Staff Inference Performance Engineer

. Diagnose performance problems across the full serving path, from orchestration through inference internals and accelerators .

Posted 10/15/2026full-timeSanta Clara • California • United StatesSenior💰 $175,000 - $265,000 per yearWebsite

Tools & technologies

KubernetesLinuxNode.jsPythonRustC++

Key responsibilities & impact

Diagnose performance problems across the full serving path, from orchestration through inference internals and accelerators
Isolate problems to specific layers and owners, fixing directly, routing to owning teams, or providing workarounds with evidence
Write minimal, reproducible problem statements for cross-stack findings
Protect throughput, latency, and stability targets by closing performance gaps against bare metal
Assess scaling from single deployments to fleet operation, including routing, placement, and shared-resource contention
Capture workload node requirements, including CPU placement, memory and NUMA locality, and network path, in orchestration configuration
Build lightweight telemetry and benchmarks for prototype stacks
Turn recurring investigations into playbooks and mentor engineers in the profiling-first method

What you’ll need

8+ years of hands-on performance or systems engineering experience
Experience resolving hard regressions across multiple stack layers
Track record of using profiling in unfamiliar, sparsely documented systems
Deep systems fundamentals, including Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior, and network paths
Experience running performance-sensitive workloads on Kubernetes
Working knowledge of LLM inference serving, including batching, KV-cache, prefill/decode, quantization, and speculative decoding
Proficiency in Python and C/C++
Clear technical writing for hardware, firmware, or platform teams
Sound judgment about ownership and problem handoff
Fluency with AI and agentic tools for investigation, code search, log triage, and drafting instrumentation
Experience with LLM serving frameworks such as vLLM, SGLang, or TensorRT-LLM preferred
Hands-on experience with speculative decoding, disaggregated serving, or heterogeneous serving architectures preferred
Experience tuning KV-cache management, request and batch scheduling, or weight prefetch and residency preferred
GPU programming and profiling experience, such as CUDA, Nsight, or NCCL, preferred
Experience with pre-production or bring-up hardware preferred
Rust experience preferred
Startup experience or comfort in environments where priorities shift quickly preferred
Legally authorized to work in the United States

Comp & perks