Staff Engineer, Inference Optimizations

DigitalOcean

. Lead technical strategy for benchmarking and performance optimization at inference engine and GPU kernel layers .

Posted 10/31/2026full-timeRemote • Massachusetts • United StatesLead💰 $191,200 - $239,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in high-performance computing and AI infrastructure, with a strong focus on GPU optimization, kernel development, and quantization techniques. Proven ability to lead technical strategy and mentor engineers while collaborating with cross-functional teams.

Highest-signal resume keywords

High-Performance ComputingGPU Kernel DevelopmentCUDA ProficiencyAttention Layer OptimizationOpen-Source Software Integration

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

GPU Architecture UnderstandingMemory Bandwidth OptimizationParallel ExecutionQuantization TechniquesCustom CUDA KernelsFP8/BF16 TuningKernel-Fusion OpportunitiesPerformance BenchmarkingMulti-Node GPU ClustersSystem Design Skills

Soft Skills

Technical LeadershipMentoringCross-Functional Collaboration

Tools & Technologies

NVIDIA GPUsAMD GPUsCUDAROCmTensorRTOpenAI TritonAITER LibraryFlashAttentionRMS NormMoE Models

Industry Keywords

Gen AI LandscapeLLMVLMLMMAI Infrastructure

Tech Stack

Tools & technologies

Node.js

About the role

Key responsibilities & impact

Lead technical strategy for benchmarking and performance optimization at inference engine and GPU kernel layers
Optimize attention layers, memory and precision management, and parallelization across multi-node GPU clusters
Improve batch-size performance using AMD’s AITER library for AMD MI355X and tune CK/ASK kernels for FP8/BF16
Identify kernel-fusion opportunities for GLM-5 Transformer-block kernels, including FlashAttention and RMS Norm
Tune expert gateway router kernels for MoE models such as Qwen3-235B, DeepSeek V3, and GLM-5
Serve as subject matter expert on NVIDIA and AMD GPUs and software stacks including CUDA, ROCm, TensorRT, and OpenAI Triton
Develop and deploy quantization techniques including FP8, INT8, and FP4
Lead code and design reviews and mentor engineers through influence
Partner with Product Management and TPMs to translate hardware limits into shippable product features
Contribute to and integrate open-source AI technologies and participate in GPU infrastructure and model performance communities
Guide the technical roadmap for DigitalOcean’s high-performance inference fleet

Requirements

What you’ll need

5+ years of experience in high-performance computing or AI infrastructure
Proven track record of solving compute utilization and memory bandwidth bottlenecks
Deep familiarity with the Gen AI landscape, including LLM, VLM, and LMM model families
Hands-on experience with attention-layer optimizations and parallelization across distributed GPU environments
Comprehensive understanding of NVIDIA and AMD GPU architectures and software ecosystems, including CUDA and ROCm
Extensive experience integrating, building with, and contributing to open-source software projects
Excellent system design skills related to low-level GPU programming, optimization, memory access patterns, and parallel execution
Experience acting as a technical lead and driving design and delivery through cross-functional alignment
Deep understanding of GPU architectures, including SMs, warp scheduling, and Tensor Cores
Expert-level Triton or CUDA proficiency
Expert-level GPU kernel development; custom CUDA kernels or Triton compiler contributions desired
Ability to work remotely

Benefits

Comp & perks

Reimbursement for relevant conferences, training, and education, subject to company policy
Flexible time off
Local team meetups
Mental health and well-being support
Annual bonus plan eligibility for non-sales roles
Possible new hire equity grant, subject to plan terms and local eligibility
Employee Stock Purchase Program in participating countries
Comprehensive total rewards package