Cloud Inference Engineer

Modular

. Build and ship an LLM-focused inference platform using disaggregated inference, multi-node deployment, high-performance networking, distributed KV-cache management, and high-throughput batch processing .

Posted 11/3/2026full-timeLos Altos • California • United StatesJuniorMid-Level💰 $148,000 - $324,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and deploying LLM-focused inference platforms, with a strong emphasis on Kubernetes, high-performance networking, and machine learning technologies. Proven ability to create reusable software tools and collaborate effectively across teams to optimize application performance.

Highest-signal resume keywords

KubernetesBackend EngineeringMachine Learning TechnologiesPythonHigh-Performance Computing

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Backend EngineeringMachine Learning TechnologiesPythonMojoRustC++Durable Software ToolsHigh-Throughput Batch ProcessingDistributed KV-Cache ManagementMulti-Node Deployment

Tools & Technologies

Kubernetes OperatorsHelm ChartsMulti-Cloud DeploymentsAutoscalingCold-Start Optimizations

Industry Keywords

LLM-Focused Inference PlatformObservabilityScalable Enterprise DeploymentDeveloper Model DeploymentHigh-Scale ML Inference Infrastructure

Tech Stack

Tools & technologies

CloudKubernetesNode.jsPythonRustC++Go

About the role

Key responsibilities & impact

Build and ship an LLM-focused inference platform using disaggregated inference, multi-node deployment, high-performance networking, distributed KV-cache management, and high-throughput batch processing
Improve operational excellence through request-to-kernel observability, multi-cloud deployments, autoscaling, and cold-start optimizations
Collaborate with kernels and GenAI teams to integrate kernel, serving, and cluster optimizations for state-of-the-art application performance
Build Helm charts, Kubernetes operators, and maintainable deployment tooling
Work with a distributed AI infrastructure team focused on scalable enterprise and developer model deployment

Requirements

What you’ll need

5+ years of experience working in backend engineering
Experience with Kubernetes and operating your own services
Ability to create durable, reusable software tools and libraries leveraged across teams and functions
Experience in machine learning technologies and use cases
Bachelor's degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 4+ years of Software Engineering or related work experience; OR Master's degree and 3+ years; OR PhD and 2+ years
2+ years of work experience with programming languages such as Python, Mojo, Rust, or C++
Candidates must be based in the US or Canada
Familiarity with Go is helpful but not required
Experience with high performance computing/networking and high-scale ML inference infrastructure is helpful but not required

Benefits

Comp & perks

Comprehensive healthcare coverage
Retirement and savings programs
Employee stock purchase opportunities
Paid time off
Wellbeing resources
Family support programs
Learning and development opportunities
RSU grants
Regular team onsites and local meetups
Travel opportunities 2-4 times a year