Senior Machine Learning Engineer, Inference

Inworld AI

. Optimize inference using modern serving frameworks and techniques .

Posted 10/14/2026full-timeRemote • SerbiaSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in optimizing inference systems using modern serving frameworks and techniques, with a strong focus on performance profiling and optimization for NVIDIA GPUs. Proven ability to manage the end-to-end process of model deployment from research to production, ensuring stability and reliability in high-performance environments.

Highest-signal resume keywords

Modern Serving FrameworksPerformance Optimization on NVIDIA GPUsKubernetes and RayC++, CUDA, Rust, or PythonPhD in CS, Physics, or Math

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

QuantizationDistillationCaching StrategiesContinuous BatchingPaged AttentionSpeculative DecodingSystems ProgrammingModel ContainerizationProduction DeploymentBenchmark Design

Soft Skills

Ability to Learn QuicklyThrive in AmbiguityWork IndependentlyPrioritize ImpactStable Production Delivery

Tools & Technologies

NVIDIA GPUsKubernetesRayCustom Load BalancingMulti-GPU Infrastructure

Industry Keywords

Inference SystemsHigh-Performance CodeOpen-Source ContributionsTechnical Write-UpsConcurrent Connections

Tech Stack

Tools & technologies

KubernetesNode.jsPythonRayRustC++

About the role

Key responsibilities & impact

Optimize inference using modern serving frameworks and techniques
Accelerate models through quantization, distillation, caching, continuous batching, paged attention, and speculative decoding
Profile and optimize high-performance code for NVIDIA GPUs
Build and scale distributed inference systems using Kubernetes, Ray, custom load balancing, and multi-GPU/multi-node infrastructure
Handle thousands of concurrent connections reliably
Take models from the research team through containerization, serving optimization, and production deployment
Design benchmarks or prototypes to clarify ambiguous technical problems
Challenge architectures to improve latency and throughput
Ensure engineering work is shipped and stable
Collaborate daily with US-based leadership and engineering teams
Contribute to public technical work and open-source projects where appropriate

Requirements

What you’ll need

Deep understanding of modern serving frameworks and techniques such as vLLM or TRT-LLM
Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
Proficiency in C++, CUDA, Rust, or highly optimized Python
Ability to profile code and optimize performance on NVIDIA GPUs
Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and handling thousands of concurrent connections
Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups
Ability to take a model from the research team through containerization and serving optimization to reliable production deployment
PhD in CS, Physics, or Math, or equivalent practical experience building backend or ML systems
Professional fluency in English, written and spoken
Ability to learn quickly, thrive in ambiguity, work independently, and prioritize impact and stable production delivery

Benefits

Comp & perks

Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area in the future, subject to business needs and applicable legal and work authorization requirements
Opportunities to share work and make open-source contributions