FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in software quality engineering with a focus on automation, test strategy design, and collaboration with cross-functional teams. Proficient in building and maintaining automated test suites for customer-facing AI features, ensuring high-quality releases through effective debugging and test reliability improvements.
Highest-signal resume keywords
Software Quality EngineeringCoded Automation Using PythonAutomated Test Suites for UI and APICI/CD IntegrationExperience with Customer-Facing AI Features
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Functional TestingRegression TestingIntegration TestingDebuggingTest Strategy DesignTest Reliability ImprovementBuilding Golden DatasetsDesigning EvalsAutomation ROIFlaky Test Identification
Soft Skills
CollaborationJudgmentAnalytical Skills
Tools & Technologies
PythonTypeScriptPytestPlaywrightCI/CD Pipelines
Industry Keywords
Customer-Facing AI FeaturesAgentsCopilotsRAGClassification
Tech Stack
Tools & technologiesPythonTypeScript
About the role
Key responsibilities & impact- Own end-to-end quality for assigned product areas and features.
- Design, review, and execute comprehensive functional, regression, and integration test strategies.
- Lead feature-level release sign-offs with engineering and product.
- Debug failures across UI, backend, and APIs to identify root causes.
- Collaborate with Customer Support to reproduce, analyze, and prevent customer-reported issues.
- Build, extend, and maintain automated test suites for UI, API, and backend services.
- Identify and fix flaky tests to improve test reliability.
- Reduce manual regression time through automation ROI.
- Integrate automated tests into CI/CD pipelines and ensure fast feedback cycles.
- Collaborate with developers to shift quality left through testability, better coverage, and early validation.
Requirements
What you’ll need- 4+ years of experience in software quality engineering.
- Hands-on QA experience on customer-facing AI features such as agents, copilots, RAG or classification.
- A track record of building and maintaining golden datasets from real customer data.
- Experience designing evals: rubrics, LLM-as-judge, and regression runs on every prompt or model change.
- Strong coded automation using Python or TypeScript, pytest or Playwright, and CI.
- The judgement to own the release bar for a non-deterministic product.
Benefits
Comp & perks- Experience with eval and observability tools such as Langfuse, LangSmith, Braintrust, Ragas, DeepEval or promptfoo.
- Red-teaming and safety testing: prompt injection, jailbreaks, PII leakage and bias.
- Background in B2B SaaS, customer support or helpdesk products.
- Synthetic test data or conversation generation, and using LLMs to generate test cases.
- Production quality monitoring, including sampling live traffic, human review loops and quality dashboards.
- Working knowledge of how LLM features are built (prompts, RAG, tool calling), so failures can be traced to a cause.
