Senior / Lead Machine Learning Engineer, Inference

Inworld AI

. Build and optimize realtime inference systems for consumer-facing AI applications .

Posted 10/14/2026full-timeRemote • GermanySeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Expertise in building and optimizing real-time inference systems for AI applications, with a strong focus on model serving, performance optimization, and deployment in high-volume environments. Proficient in leveraging modern frameworks and techniques to enhance GPU workloads and manage concurrent connections effectively.

Highest-signal resume keywords

Deep Understanding of Modern Serving FrameworksProficiency in C++, CUDA, Rust, or Optimized PythonExperience with Kubernetes and RayAbility to Optimize NVIDIA GPU PerformancePhD in CS, Physics, Math, or Equivalent Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Real-Time Inference SystemsModel Serving OptimizationPerformance ProfilingNVIDIA GPU OptimizationQuantization and DistillationContinuous BatchingPaged AttentionSpeculative DecodingSystems ProgrammingOpen-Source Contributions

Soft Skills

Collaboration with Engineering TeamsTechnical Writing

Tools & Technologies

KubernetesRayNVIDIA GPUsVLLMTRT-LLM

Industry Keywords

AI ApplicationsInference EnginesContainerizationProduction DeploymentConcurrent Connections

Tech Stack

Tools & technologies

KubernetesNode.jsPythonRayRustC++

About the role

Key responsibilities & impact

Build and optimize realtime inference systems for consumer-facing AI applications
Work with first-party speech models, LLM serving, and modular high-volume realtime APIs
Optimize model serving and inference performance using modern serving frameworks and acceleration techniques
Profile and optimize code and NVIDIA GPU workloads for latency and throughput
Scale inference across Kubernetes, Ray, multi-GPU, and multi-node environments
Handle thousands of concurrent connections reliably
Take models from research through containerization, serving optimization, and production deployment
Collaborate daily with US-based leadership and engineering teams
Design benchmarks or prototypes to resolve unclear performance and reliability problems
Contribute to impactful engineering work and, where appropriate, open-source projects and technical write-ups

Requirements

What you’ll need

Deep understanding of modern serving frameworks and techniques such as vLLM or TRT-LLM
Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
Proficiency in C++, CUDA, Rust, or highly optimized Python
Ability to profile code and optimize NVIDIA GPU performance
Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and handling thousands of concurrent connections
Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups
Ability to take a model from the research team through containerization, serving optimization, and reliable production deployment
PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems
Professional fluency in English, written and spoken, is required
Ability to work legally in Germany, as indicated by the required legal-right-to-work application question

Benefits

Comp & perks

Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area in the future, subject to business needs and applicable legal and work authorization requirements