Staff Engineer, Inference Optimizations

DigitalOcean

. Lead the technical strategy for benchmarking and performance optimization at inference-engine and GPU-kernel layers .

Posted 10/31/2026full-timeRemote • California • United StatesLead💰 $191,200 - $239,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in performance optimization and benchmarking at inference-engine and GPU-kernel layers, with a strong focus on generative AI inference and attention-layer optimizations. Proven ability to lead technical strategy, integrate open-source projects, and collaborate cross-functionally to enhance product features.

Highest-signal resume keywords

High-Performance ComputingGPU Architecture ExpertiseCUDA ProficiencyAttention-Layer OptimizationOpen-Source Software Contribution

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Performance OptimizationBenchmarking TechniquesQuantization TechniquesKernel TuningMemory ManagementParallel ExecutionFP8/BF16 TechniquesAttention Layer OptimizationCustom CUDA Kernel DevelopmentDistributed GPU Environments

Soft Skills

Technical LeadershipCross-Functional CollaborationSystem Design Skills

Tools & Technologies

AMD AITER LibraryNVIDIA CUDAROCmTriton CompilerMoE Models

Industry Keywords

Generative AILLMVLMLMMGPU Infrastructure

Tech Stack

Tools & technologies

Node.js

About the role

Key responsibilities & impact

Lead the technical strategy for benchmarking and performance optimization at inference-engine and GPU-kernel layers
Solve complex performance issues involving attention layers, memory and precision management, and parallelization across multi-node GPU clusters
Implement optimization techniques for generative AI inference
Improve batch-size performance using AMD's AITER library and tune CK or ASK kernels for FP8/BF16
Identify kernel-fusion opportunities for Transformer-block layers such as FlashAttention and RMS Norm
Tune expert-gateway router kernels for MoE models including Qwen3-235B, DeepSeek V3, and GLM-5
Advise on NVIDIA/AMD hardware procurement and software integration
Develop and deploy quantization techniques including FP8, INT8, and FP4
Lead code and design reviews and elevate the team's technical standards
Partner with Product Management and TPMs to turn hardware limits into shippable product features
Contribute to and integrate open-source AI projects and maintain a presence in GPU infrastructure and model-performance communities

Requirements

What you’ll need

5+ years of experience in high-performance computing or AI infrastructure
Proven track record of solving compute utilization and memory bandwidth bottlenecks
Deep familiarity with the Gen AI landscape, including LLM, VLM, and LMM model families
Hands-on experience with attention-layer optimizations and parallelization across distributed GPU environments
Comprehensive understanding of NVIDIA and AMD GPU architectures and software ecosystems, including CUDA and ROCm
Extensive experience integrating, building with, and contributing to open-source software projects
Excellent system design skills related to low-level GPU programming, optimization, memory access patterns, and parallel execution
Experience acting as a technical lead and driving design and delivery through cross-functional alignment
Deep understanding of GPU architectures, including SMs, warp scheduling, and Tensor Cores
Expert-level Triton or CUDA proficiency
Experience contributing to the Triton compiler or writing custom CUDA kernels for a major LLM is desired

Benefits

Comp & perks

Reimbursement for relevant conferences, training, and education, subject to company policy
Flexible time off
Local team meetups
Mental health and well-being support
Annual bonus plan eligibility for non-sales roles
Eligible employees may receive equity compensation, including a new hire equity grant
Employee Stock Purchase Program in participating countries
Professional development resources
Equal-opportunity employment