Inference Performance Engineer, Agent Driven Inference Optimization

NVIDIA

. Distill performance expertise into reusable skills, workflows, and evidence-backed methodologies that AI agents can complete autonomously .

Posted 11/10/2026full-timeSanta Clara • California • United StatesMid-LevelSenior💰 $124,000 - $241,500 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates extensive knowledge in AI model execution optimization, including performance benchmarking and profiling on NVIDIA GPU platforms. Proficient in Python and C++/CUDA, with a strong focus on experimental methodologies and collaboration across inference frameworks.

Highest-signal resume keywords

AI Model Execution OptimizationGPU Workload BenchmarkingPython Engineering SkillsC++/CUDA Codebase ModificationExperimental Methodology

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Performance OptimizationThroughput-Latency TradeoffsKV Cache HandlingParallel Processing TechniquesQuantizationBenchmarkingProfilingKernel-Level Performance InterpretationCUDA Kernel OptimizationAgentic AI Workflows

Soft Skills

Strong Written CommunicationStrong Verbal Communication

Tools & Technologies

Nsight SystemsNsight ComputeCUPTIPyTorch ProfilerTensorRT-LLMVLLMSGLangDynamoNCCLNVSHMEM

Industry Keywords

Disaggregated ServingMoE InferencePublic Inference BenchmarksMLPerf InferenceSemiAnalysis InferenceX

Tech Stack

Tools & technologies

Node.jsPythonPyTorchC++

About the role

Key responsibilities & impact

Distill performance expertise into reusable skills, workflows, and evidence-backed methodologies that AI agents can complete autonomously
Review agent-generated experiments, validate findings, and curate best-known configurations
Improve AI inference workload performance to increase throughput per GPU and user interactivity
Explore configuration options, parallelism techniques, batching, KV cache handling, quantization, and speculative decoding settings
Measure and optimize aggregated and disaggregated serving architectures across TensorRT-LLM, SGLang, vLLM, and Dynamo on NVIDIA GPU platforms
Profile workloads using Nsight Systems, kernel traces, and internal analysis tools
Use roofline and speed-of-light analysis to identify headroom and drive fixes from hypothesis to measured performance gains
Upstream serving framework patches, optimized kernels, and deployment recipes while maintaining model correctness
Collaborate with TensorRT-LLM, SGLang, vLLM, kernel, benchmarking, and GPU architecture teams to deliver performance improvements

Requirements

What you’ll need

BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience
3+ years of relevant engineering experience
Extensive knowledge of efficiency and optimization in AI model execution, including continuous batching, throughput-latency tradeoffs, KV cache and memory limitations, parallel processing techniques, MoE serving, quantization, and serving SLAs
Hands-on experience benchmarking and profiling GPU workloads with Nsight Systems, Nsight Compute, CUPTI, PyTorch profiler, or similar tools
Ability to interpret kernel-level performance data
Strong Python engineering skills
Ability to navigate and modify large C++/CUDA serving codebases
Rigorous experimental methodology using controlled single-variable comparisons and reproducible benchmarks
Strong written and verbal communication skills
Direct contributions to TensorRT-LLM, vLLM, SGLang, FlashInfer, Dynamo, or comparable inference frameworks (standout qualification)
Experience with disaggregated serving, wide expert-parallel MoE inference, KV cache transfer, or NCCL/NIXL/NVSHMEM communication at multi-node scale (standout qualification)
CUDA kernel authorship or optimization experience on Hopper/Blackwell architectures, focusing on Tensor Cores, TMA, and warp specialization (standout qualification)
Proven results on public inference benchmarks such as MLPerf Inference or SemiAnalysis InferenceX (standout qualification)
Experience building or operating agentic AI workflows to automate engineering tasks (standout qualification)

Benefits

Comp & perks

Highly competitive salaries
Comprehensive benefits package
Equity
Benefits for you and your family