FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Staff Software Engineer, AI Gateway
Docker, Inc. Set technical direction for core AI Gateway systems, including the model router, MCP gateway, and control plane .
Tech Stack
Tools & technologiesAWSCloudDistributed SystemsDockerKubernetesMicroservicesRustGo
About the role
Key responsibilities & impact- Set technical direction for core AI Gateway systems, including the model router, MCP gateway, and control plane
- Ensure architectural decisions support 10x usage growth
- Identify infrastructure, process, and tooling gaps and drive fixes through production
- Define, deliver, and implement the Gateway's technical strategy with engineering leadership
- Solve complex problems involving dynamic model selection, auto-failover, cost-based routing, and end-to-end policy enforcement
- Serve as the expert for secure credential storage, session summarization, caching, and rate limiting
- Build infrastructure for evaluating, replaying, and backtesting agent and LLM behavior
- Mentor senior and mid-level engineers through code, design review, and example
- Partner with product, design, and leadership on control plane UI direction and product strategy
- Define a unified abstraction layer across model and tool providers including OpenAI, Anthropic, Google, and AWS Bedrock
- Architect secure, scalable identity and credential vaulting for enterprise-scale access
- Build real-time and historical analytics infrastructure for AI agent behavior
- Ensure policy enforcement and logging work end-to-end from prompt to tool to response
- Set the standard for developer experience through intuitive APIs and first-class observability
Requirements
What you’ll need- 8+ years of backend engineering experience
- Track record of owning production-grade systems at significant scale
- Deep specialization in distributed systems, cloud-native infrastructure, and API design
- Experience building secure, high-throughput services such as gateways, proxies, load balancers, and policy engines
- Fluency in Go and/or Rust
- Track record of identifying systemic gaps and driving fixes across team boundaries
- Strong product mindset and experience shaping technical direction
- Ability to operate autonomously with wide latitude
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience
- Prior experience with OpenAI, Anthropic, or similar LLM APIs is a bonus
- Familiarity with RAG architectures, vector databases, or agent frameworks is a bonus
- Experience with policy engines, observability frameworks, or API gateways is a bonus
- Understanding of OAuth 2.1, secret management, and cloud IAM systems is a bonus
- Experience with Kubernetes, Docker, and microservices architecture is a bonus
- Sponsorship considered case-by-case based on business needs
Benefits
Comp & perks- Equity for all full-time employees
- Remote-first work from home
- Flexible schedule
- Generous PTO
- Designated quarterly Whaleness Days
- Designated end-of-year Whaleness break
- Home office support
- Technology stipend equivalent to US$100 net per month
- Annual learning and development stipend for conferences, courses, certifications, and continued learning
- 16 weeks of paid parental leave after six months of employment
- Medical benefits
- Retirement benefits
- Paid holidays
- Docker swag