QA Engineer, Voice AI Agents

Arbiter

. Investigate reported and detected issues on live campaigns, including dropped calls, wrong agent responses, failed handoffs, incorrect scheduling outcomes, and data mismatches .

Posted 10/8/2026full-timeRemote • United StatesMid-LevelSenior💰 $180,000 - $200,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in quality assurance processes, including bug reporting, test plan development, and ongoing campaign quality analysis. Proficient in SQL, API interactions, and automation of quality checks, with a strong focus on communication and collaboration across teams.

Highest-signal resume keywords

Quality AssuranceSQL QueryingBug ReportingTest Plan DevelopmentVoice AI Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

DebuggingTest PlansBug ReportsAutomationData Analysis

Soft Skills

Clear CommunicationGood JudgmentDocumentation

Tools & Technologies

APIsJSONYAMLObservability ToolsCall Analytics Tools

Industry Keywords

Healthcare ExperienceHIPAAPHI HandlingConversational AIIVR

Tech Stack

Tools & technologies

PythonSQL

About the role

Key responsibilities & impact

Investigate reported and detected issues on live campaigns, including dropped calls, wrong agent responses, failed handoffs, incorrect scheduling outcomes, and data mismatches
Trace problems across call recordings, transcripts, system logs, campaign configuration, and integration data to identify root causes
Write clear, reproducible bug reports with severity, member impact, and supporting evidence
Triage issues and route them to the appropriate owner
Run pre-launch QA on campaigns and workflow changes, testing scripts, edge cases, escalation paths, and handoffs
Perform ongoing conversation QA on live campaigns and flag trends
Verify targeting, sequencing, disposition logic, and outcome capture
Help build and maintain test cases, scoring rubrics, regression suites, and release checklists
Automate repetitive checks such as transcript analysis, configuration validation, and outcome reconciliation
Track quality metrics and report patterns to deployment, product, and engineering teams
Work cross-functionally with engineers, deployment managers, and clinical and nurse operations staff

Requirements

What you’ll need

3 to 5 years in QA, support engineering, technical operations, or a similar role on a production software platform
Strong debugging instinct
Comfort reading logs, querying data with SQL, and working with APIs and JSON or YAML configuration
Experience writing test plans and bug reports
Clear written communication and precise documentation habits
Good judgment on severity and prioritization
Preferred: experience with voice AI, conversational AI, IVR, or contact center platforms
Preferred: scripting ability in Python or similar
Preferred: exposure to LLM evaluation, prompt testing, or conversation quality scoring
Preferred: healthcare experience, especially payer or provider outreach, and familiarity with HIPAA and PHI handling
Preferred: experience with observability tools or call analytics tools

Benefits

Comp & perks

Meaningful equity
Generous paid time off (PTO)
Comprehensive health, vision, and dental insurance
Life and disability insurance
Simple IRA matching
Professional development budget for conferences, courses, and certifications
Wellness programs