Senior/Staff Software Engineer, Inference, Compute Infrastructure Engineering

Together AI

. Build the provisioning state machine modeling the full physical-host lifecycle from discovery and inference bring-up through GPU driver/CUDA installation, health validation, and decommission/RMA.

Posted 11/3/2026full-timeRemote • IndiaSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and managing software control planes and orchestration systems, with a strong focus on reliability, automation, and developer experience. Proficient in programming languages such as Go and Python, and experienced with event-driven architectures and durable workflow orchestration tools.

Highest-signal resume keywords

Go ProgrammingPython ProgrammingKubernetes ControllersEvent-Driven SystemsWorkflow Orchestration

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Software EngineeringDeclarative APIsInfrastructure AutomationCI/CD PracticesAutomated TestingVersion ControlIdempotencyRollback MechanismsHealth ValidationSelf-Healing Systems

Soft Skills

Product MindsetCollaboration

Tools & Technologies

TemporalCadenceKafkaNATSSQSCUDANCCLInfiniBandPXE/iPXERedfish/IPMI

Industry Keywords

GPU InfrastructureDatacenter-Scale InfrastructureHyperscaler

Tech Stack

Tools & technologies

CloudKafkaKubernetesPythonRustGo

About the role

Key responsibilities & impact

Build the provisioning state machine modeling the full physical-host lifecycle from discovery and inference bring-up through GPU driver/CUDA installation, health validation, and decommission/RMA.
Build declarative self-service APIs and a control plane for requesting, scaling, and tearing down inference clusters.
Automate self-healing by detecting degraded or failed nodes, draining them safely, triggering repair or replacement, and restoring healthy capacity.
Own pipeline reliability through idempotency, retries, rollback, and drift detection.
Partner with the inference/ML platform team to encode cluster topology, interconnect, and scheduling constraints as platform abstractions.
Apply strong typing, automated tests, code review, versioning, and CI/CD to infrastructure code.
Build Kubernetes-native controllers, reconciliation loops, API/CLI self-service surfaces, and event-driven health, remediation, and utilization systems for an efficient GPU inference fleet.
Operate and support the software in production.

Requirements

What you’ll need

Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living.
Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution.
Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).
Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts.
A product mindset; experience building internal platforms or APIs consumed by other engineering teams and caring about developer experience.
Nice to have: Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC), networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure.
Nice to have: Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE).
Nice to have: Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.
Nice to have: Systems programming in Rust or Go.