Staff Engineer, Inference Optimizations

DigitalOcean

. Lead technical strategy for benchmarking and performance optimization at inference engine and GPU kernel layers .

Posted 10/31/2026full-timeRemote • Colorado • United StatesLead💰 $191,200 - $239,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in high-performance computing and AI infrastructure, with a strong focus on GPU optimization techniques, attention-layer enhancements, and parallel execution across distributed environments. Proven ability to lead technical strategy and contribute to open-source AI projects while collaborating effectively with cross-functional teams.

Highest-signal resume keywords

High-Performance ComputingGPU Optimization TechniquesAttention-Layer OptimizationNVIDIA And AMD GPU ArchitecturesExpert-Level Triton Or CUDA Proficiency

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Attention-Layer OptimizationParallelization Across Distributed GPU EnvironmentsMemory ManagementPerformance OptimizationKernel Fusion TechniquesQuantization TechniquesLow-Level GPU ProgrammingCustom CUDA KernelsFP8/INT8/FP4 TechniquesCompute Utilization Solutions

Soft Skills

Technical LeadershipCross-Functional CollaborationSystem Design Skills

Tools & Technologies

AMD AITER LibraryCUDAROCmTriton CompilerOpen-Source AI Projects

Industry Keywords

Generative AILLMVLMLMMGPU Infrastructure

Tech Stack

Tools & technologies

Node.js

About the role

Key responsibilities & impact

Lead technical strategy for benchmarking and performance optimization at inference engine and GPU kernel layers
Engineer solutions for attention-layer optimization, memory and precision management, and advanced parallelization across multi-node GPU clusters
Implement cutting-edge optimization techniques for generative AI inference
Improve batch-size performance using AMD’s AITER library for AMD MI355X and tune CK or ASK for FP8/BF16
Identify kernel-fusion opportunities for GLM-5 Transformer-block layers, including FlashAttention and RMS Norm
Tune expert gateway router kernels for MoE models such as Qwen3-235B, DeepSeek V3, and GLM-5
Advise on modern NVIDIA and AMD GPU hardware procurement and software integration
Develop and deploy quantization techniques including FP8, INT8, and experimental FP4
Lead code and design reviews without direct people-management responsibilities
Partner with Product Management and TPMs to turn hardware limits into shippable product features
Contribute to and integrate open-source AI projects and participate in GPU infrastructure and model-performance communities

Requirements

What you’ll need

5+ years of experience in high-performance computing or AI infrastructure
Proven track record of solving compute utilization and memory bandwidth bottlenecks
Deep familiarity with the Gen AI landscape, including LLM, VLM, and LMM model families
Hands-on experience with attention-layer optimizations and parallelization across distributed GPU environments
Comprehensive understanding of NVIDIA and AMD GPU architectures and software ecosystems, including CUDA and ROCm
Extensive experience integrating, building with, and contributing to open-source software projects
Excellent system design skills related to low-level GPU programming, optimization, memory access patterns, and parallel execution
Experience acting as a technical lead and driving design and delivery through cross-functional alignment
Deep understanding of GPU architectures, including SMs, warp scheduling, and Tensor Cores
Expert-level Triton or CUDA proficiency
Experience contributing to the Triton compiler or writing custom CUDA kernels for a major LLM is desired

Benefits

Comp & perks

Reimbursement for relevant conferences, training, and education, subject to company policy
Flexible time off
Local team meetups
Mental health and well-being support
Eligible participation in DigitalOcean’s annual bonus plan
May be eligible for a new hire equity grant
In participating countries, opportunity to participate in the Employee Stock Purchase Program
Equal-opportunity employment