Staff Engineer, Inference Optimizations

DigitalOcean

. Lead the technical strategy for benchmarking and performance optimizations at the inference engine and GPU kernel layers .

Posted 10/31/2026full-timeRemote • Texas • United StatesLead💰 $191,200 - $239,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in high-performance computing and AI infrastructure, with a strong focus on GPU optimization techniques, attention-layer enhancements, and parallel execution strategies. Proven ability to lead technical initiatives and collaborate across teams to deliver innovative solutions in generative AI workloads.

Highest-signal resume keywords

High-Performance ComputingGPU Optimization TechniquesAttention-Layer OptimizationsNVIDIA And AMD GPU ArchitecturesExpert-Level Triton Or CUDA Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Attention-Layer OptimizationsParallelization StrategiesMemory ManagementKernel FusionQuantization TechniquesFP8/BF16 TuningLow-Level GPU ProgrammingCustom CUDA KernelsPerformance OptimizationCompute Utilization Solutions

Soft Skills

Technical MentorshipCross-Functional CollaborationSystem Design Skills

Tools & Technologies

AMD AITER LibraryCUDAROCmTriton CompilerOpen-Source AI Projects

Industry Keywords

Generative AILLMVLMLMMGPU Infrastructure

Tech Stack

Tools & technologies

Node.js

About the role

Key responsibilities & impact

Lead the technical strategy for benchmarking and performance optimizations at the inference engine and GPU kernel layers
Engineer solutions for attention-layer optimizations, memory and precision management, and advanced parallelization across multi-node GPU clusters
Implement cutting-edge optimization techniques for generative AI workloads
Improve batch size performance using AMD’s AITER library for AMD MI355X, including tuning CK or ASK for FP8/BF16
Identify kernel fusion opportunities for GLM-5 Transformer-block kernels, including FlashAttention and RMS Norm
Tune expert gateway router kernels for MoE models such as Qwen3-235B, DeepSeek V3, and GLM-5
Advise on modern NVIDIA and AMD GPU families, software stacks, hardware procurement, and software integration
Develop and deploy quantization techniques including FP8, INT8, and experimental FP4
Lead code and design reviews and provide technical mentorship without direct-management responsibilities
Partner with Product Management and TPMs to turn hardware limits into shippable product features
Contribute to and integrate open-source AI projects and maintain a presence in GPU infrastructure and model-performance communities

Requirements

What you’ll need

5+ years of experience in high-performance computing or AI infrastructure
Proven track record of solving compute utilization and memory bandwidth bottlenecks
Deep familiarity with the Gen AI landscape, including LLM, VLM, and LMM model families
Hands-on experience with attention-layer optimizations and parallelization strategies across distributed GPU environments
Comprehensive understanding of NVIDIA and AMD GPU architectures and software ecosystems, including CUDA and ROCm
Extensive experience integrating, building with, and contributing to open-source software projects
Excellent system design skills related to low-level GPU programming, optimization, memory access patterns, and parallel execution
Experience acting as a technical lead and driving design and delivery through cross-functional alignment
Deep understanding of GPU architectures, including SMs, Warp scheduling, and Tensor Cores
Expert-level Triton or CUDA experience
Experience contributing to the Triton compiler or writing custom CUDA kernels for a major LLM is sought

Benefits

Comp & perks

Reimbursement for relevant conferences, training, and education, subject to company policy
Flexible time off
Local team meetups
Mental health and well-being support
Eligible employees may receive equity compensation, including a new hire equity grant
Eligible employees may participate in the Employee Stock Purchase Program in participating countries
Eligibility for DigitalOcean’s annual bonus plan
Equal-opportunity employment