Together AI

Infrastructure Engineer

• Build the provisioning state machine modeling the full physical-host lifecycle from discovery and inference bring-up through GPU driver/CUDA installation, health validation, and decommission/RMA.
• Build declarative self-service APIs and a control plane for requesting, scaling, and tearing down inference clusters.
• Automate self-healing by detecting degraded or failed nodes, draining them safely, triggering repair or replacement, and restoring healthy capacity.
• Own pipeline reliability through idempotency, retries, rollback, and drift detection.
• Partner with the inference/ML platform team to encode cluster topology, interconnect, and scheduling constraints as platform abstractions.
• Apply strong typing, automated tests, code review, versioning, and CI/CD to infrastructure code.
• Build Kubernetes-native controllers, reconciliation loops, API/CLI self-service surfaces, and event-driven health, remediation, and utilization systems for an efficient GPU inference fleet.
• Operate and support the software in production.

Explore Senior/Staff Software Engineer, Inference, Compute… at Together AI. Review requirements and application details.

Senior/Staff Software Engineer, Inference, Compute Infrastructure Engineering

Engineer

• Select, configure, and optimize inference engines based on hardware, model architecture, and workload profile
• Develop configuration updates to win critical POCs and benchmarks and optimize customer deployments
• Tune KV cache, apply speculative decoding, determine optimal tensor and pipeline parallelism, and determine quantization strategies to meet throughput and latency targets
• Drive hands-on RL training runs and optimize system design
• Guide customers through LoRA, SFT, DPO, RLHF, and GRPO pipelines from experimentation through production
• Act as the primary technical point of contact for strategic accounts
• Monitor and optimize endpoint configurations and help customers maximize platform value
• Collaborate with customers to achieve critical milestones
• Establish direct alignment with strategic customers during onboarding
• Ensure appropriate inference and post-training configurations are in place from day one
• Surface field insights to influence the software and model roadmap
• Contribute to the product to support customer requirements and improve the customer experience
• Drive early feature and research adoption with strategic customers

Explore Forward Deployed Engineer – Inference, Post-Training, Mandarin… at Together AI. Review requirements and application details.

Together AI

Open roles

Senior/Staff Software Engineer, Inference, Compute Infrastructure Engineering

Forward Deployed Engineer – Inference, Post-Training, Mandarin Speaking