Senior QA Automation Engineer

Data Wow Co., Ltd.

. Build evaluation suites for LLM and ML features, including curated test sets, reference-answer scoring, justified LLM-as-judge evaluation, and human review where appropriate.

Posted 10/2/2026full-timePathum Wan • ThailandSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in test automation and quality assurance for LLM and ML features, with strong programming skills in Python and experience in designing automated test suites. Proficient in API and data testing, with the ability to communicate effectively in Thai and English.

Highest-signal resume keywords

Test AutomationPython ProgrammingLLM Testing ExperienceAPI TestingAutomation Frameworks

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Test AutomationPython ProgrammingLLM TestingAPI TestingSQLRegression TestingData ValidationSemantic SimilarityStructured-Output ValidationCI/CD

Soft Skills

Clear CommunicationJudgment in Automation

Tools & Technologies

PytestPlaywrightCypressRestAssured

Industry Keywords

Quality AssuranceMachine LearningEvaluation SuitesTest StrategyData Pipelines

Tech Stack

Tools & technologies

CypressJavaPythonSQLTypeScript

About the role

Key responsibilities & impact

Build evaluation suites for LLM and ML features, including curated test sets, reference-answer scoring, justified LLM-as-judge evaluation, and human review where appropriate.
Define measurable quality standards with the delivery team before launch and hold releases to them.
Test retrieval pipelines end to end, including chunking, indexing, retrieval quality, answer grounding, and missing-document behavior.
Write regression tests for non-deterministic output using semantic similarity, structured-output validation, tolerance bands, and repeated sampling.
Test client-relevant failure modes including hallucination, prompt injection, cross-tenant or cross-session data leakage, refusal and over-refusal, and Thai-language degradation.
Track cost and latency alongside accuracy before release.
Detect post-release drift by comparing production samples with the baseline and raise issues early.
Design and maintain automated test suites for APIs, services, data pipelines, and web applications built for client projects.
Run tests in CI and maintain fast, stable suites, treating flakiness as a defect.
Set the test strategy at engagement start, including automation scope, manual testing, exit criteria, and client evidence.

Requirements

What you’ll need

4-5 years in test automation, QA engineering, or software engineering with substantial testing ownership.
Strong programming ability in Python, or in TypeScript or Java with the willingness to work primarily in Python.
Hands-on experience testing an LLM or ML-based product in production or close to it.
Practical command of at least one automation framework such as Pytest, Playwright, Cypress, or RestAssured, and experience running suites in CI.
Working knowledge of API and data testing, including SQL and validating data through a pipeline.
The judgement to decide what deserves automation and what does not, and to say so when a suite is producing noise rather than signal.
Clear written and spoken communication in Thai and English.

Benefits

Comp & perks

Hybrid Working options with Flexible Working Hours.
Your Choice of Gear: Mac or Windows? You pick your preferred work equipment.
Comprehensive Group Health Insurance.
Monthly Grab Transportation credits.
Job-related training courses and certificate fee coverage.
10+ days of annual leave.
Free snacks and drinks available at the office.