Principal Engineer, Model Performance

Runware

. Own inference performance end to end across kernels, serving engines, model-level optimizations and benchmarks for image, video, audio, 3D and LLM workloads .

Posted 11/6/2026full-timeRemote • United StatesLeadWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates extensive expertise in GPU and deep learning systems engineering, with a strong focus on optimizing inference performance and leading technical teams. Proficient in CUDA, Python, and C++, with a proven track record of deploying state-of-the-art models efficiently.

Highest-signal resume keywords

CUDA ExpertiseGPU Architecture KnowledgeInference Engine Production ExperiencePerformance Team LeadershipBenchmarking and Performance Optimization

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

CUDA Kernel DesignTensorRT-LLMDeep Learning OptimizationQuantization TechniquesParallelism StrategiesPython ProgrammingC++ ProgrammingPyTorch InternalsPerformance Regression SystemsAI for Kernel Tuning

Soft Skills

Team LeadershipMentoringAccountabilityAdaptabilityCollaboration

Tools & Technologies

Nsight SystemsNsight ComputeTritonSGLangVLLMCustom Diffusion PipelinesSonic Inference Engine

Industry Keywords

HPCDeep Learning Systems EngineeringInference PerformanceProduction ModelsLatency Optimization

Tech Stack

Tools & technologies

PythonPyTorchC++

About the role

Key responsibilities & impact

Own inference performance end to end across kernels, serving engines, model-level optimizations and benchmarks for image, video, audio, 3D and LLM workloads
Set the model performance roadmap based on real traffic, cost and upcoming hardware
Write and tune CUDA kernels, fuse attention and MLP paths, introduce FP8, FP4 and INT4 quantization, and implement parallelism and batching strategies on multi-GPU nodes
Build the pipeline to move newly released open models from weights to optimized, production-grade deployments within days
Choose and shape the serving stack, including TensorRT-LLM, vLLM, SGLang, custom diffusion and video pipelines, and the Sonic Inference Engine runtime
Own reproducible, hardware-aware benchmarking and performance regression systems wired into CI
Partner with infrastructure and hardware engineering on GPU memory, storage and network bottlenecks
Build and lead the Model Performance team, including hiring, mentoring and setting technical standards
Use AI agents for automated kernel search and tuning, benchmark generation, profiling triage and documentation

Requirements

What you’ll need

10+ years in GPU, HPC or deep learning systems engineering, including several years optimizing inference performance for production models at scale
Track record of shipping performance improvements and taking state-of-the-art models from reference implementation to the Pareto frontier of latency and cost on real hardware
Deep, current, hands-on CUDA and GPU architecture expertise, including kernel design, memory hierarchy and occupancy, Tensor Cores, fused kernels, CUDA Graphs, NCCL, Nsight Systems and Nsight Compute
Production experience with at least one major inference engine: TensorRT-LLM, vLLM, SGLang or similar
Experience with continuous batching, paged KV cache, speculative decoding, prefill/decode disaggregation, quantization, tensor parallelism, pipeline parallelism and expert parallelism
Fluent in Python and C++
Comfortable with PyTorch internals, torch.compile and Triton
Experience leading engineers, building or running a performance team, hiring, setting technical direction and remaining hands-on
Daily use of AI for kernel search, profiling analysis, benchmarking, documentation and code review
Ability to own outcomes in a fast-moving, ambiguous environment and be accountable for latency, cost and quality across hundreds of thousands of models

Benefits

Comp & perks

Meaningful stock options
Remote-first work arrangement
In-person meetings twice a year
Core hours for collaboration and calls; flexible schedule outside core hours
Generous paid time off, including vacation, sick days and public holidays
Paid maternity, paternity and caregiver leave
Twice-yearly company retreats
GPU budget and company backing for inference performance work