Salad

AI Engineer

• Own the AI Gateway system end to end, including request routing, streaming, batch/async job handling, and horizontal scaling across multiple gateway servers
• Build and tune fleet-efficiency algorithms for node autoscaling, performance scoring, and eviction of underperformers
• Operate and scale inference nodes running vLLM, llama.cpp, and similar servers
• Design and maintain OpenTelemetry-based observability across the gateway and fleet
• Maintain accurate pay-per-token and subscription billing integrations based on telemetry
• Benchmark new models, quantizations, and GPU hardware for approximately 25% of working time
• Work with product and marketing teams to decide what enters production
• Collaborate on cross-system design and integrations with Salad engineering
• Explain technical trade-offs to technical and non-technical teammates
• Report directly to the CTO
• Participate in shared after-hours support

Staff Software Engineer, AI Inference Gateway