QA Automation Engineer

• Own infrastructure that continuously verifies whether autonomous agents take intended actions before and after production
• Stage, test, and lifecycle-manage bots and agent workflows
• Establish baseline quality metrics and coverage standards for AI initiatives
• Build infrastructure to run automated research chains and agent evaluations at scale
• Create automated evaluation frameworks for LLM outputs covering accuracy, safety, bias, and regression
• Implement RAG evaluation, prompt regression testing, and golden-set benchmarking
• Define and enforce quality thresholds before release
• Partner with AI Researchers and Engineers on evaluation design for new model and agent patterns
• Build release-gating infrastructure, including staged rollouts and rollback authority
• Develop AI automation solutions to maintain software quality across the product portfolio
• Integrate testing into GitHub Actions or equivalent for AI model and agent releases
• Maintain test suites as models, prompts, and tools change
• Collaborate with AI Solutions Architects, AI Engineers, Principal AI Product, and product teams on evaluation, launch readiness, QA, and feature validation
• Document standards, runbooks, and patterns so non-engineers building in Sage can self-serve basic testing
• Set technical standards and grow into leading a small automation team

Director, AI Automation Engineering

🌐
Worldwide
❌ Jobs You've Hidden
⭐️ Saved Jobs
✅ Applied Jobs
✉️ Email Alerts
👤 Account
404 - Page Not Found
🌐
Worldwide
Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com
Search
Remote jobs
Search Jobs by country
Search jobs by city
Search jobs by job title
Search entry-level jobs
Search junior-level jobs
Search senior-level jobs
Search jobs by tech stack
Search jobs by contract type
Search remote internships
Search remote part-time jobs
Remote jobs Anywhere in the World
Companies Hiring Anywhere in the World
Companies Hiring Sales People Anywhere in the World
Companies Hiring Software Engineers Anywhere in the World
Resources
About us
Advice
Tips for finding remote jobs
Interview questions and answers
Resume examples
Cover letter examples
Post a job
Affiliates
Is Remote Rocketship legit?
Privacy policy
Terms of service
Job board SEO course
Remote Job Search MasterClass
Resume Review
AI Apply Copilot
OpenClaw job finder
API docs
Find jobs using your resume
Jobs by Country
Remote jobs anywhere in the world (Worldwide remote jobs)
Remote jobs United States
Remote jobs Australia
Remote jobs Brazil
Remote jobs Canada
Remote jobs France
Remote jobs Ireland
Remote jobs Germany
Remote jobs Netherlands
Remote jobs Spain
Remote jobs UK
Popular Jobs
Remote data analyst jobs
Remote customer support jobs
Remote executive assistant jobs
Remote marketing jobs
Remote product designer jobs
Remote product manager jobs
Remote project manager jobs
Remote recruiter jobs
Remote sales jobs
Remote software engineer jobs
Jobs by Type
Remote full-time jobs
Remote part-time jobs
Remote contract jobs
Remote internship jobs
Remote entry-level jobs
Remote jobs with no experience required
Remote junior jobs (1-3 years of experience)
Digital nomad jobs
Remote jobs with no degree required
Freelance remote jobs
Temporary remote jobs
Remote jobs hiring now
Stay at home mom jobs

Staff DevOps Automation Engineer

• Design, develop, and maintain automation capabilities within the homegrown Assisted Manual Test Tool (AMTT) platform for iOS, Android, and payment solution testing
• Build automated test solutions for functional, integration, regression, stress, and scalability testing
• Develop and enhance in-house testing platforms to improve automation coverage, execution efficiency, and reporting
• Partner with engineering teams to integrate automated testing into CI/CD pipelines and release processes
• Support reliable and repeatable testing across multiple product releases and environments
• Identify, troubleshoot, and resolve software defects, reliability issues, and performance bottlenecks
• Analyze application stability, including crashes, memory leaks, CPU utilization, and resource consumption issues
• Generate quality metrics, root cause analyses, and actionable recommendations to improve system performance, scalability, and product quality