Senior QA Automation Engineer

IDT BY INDET GROUP

. Embed fully within a Scrum team and participate in all ceremonies and associated artifacts .

Posted 9/18/2026full-timeRemote • ColombiaSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in QA for complex integrated products, with a focus on testing SIP/VoIP systems and non-deterministic outputs. Proficient in Python for automation framework development and skilled in defining quality metrics to inform release decisions.

Highest-signal resume keywords

5+ Years In QASIP/VoIP Testing ExperiencePython Automation FrameworksQuality Metrics DefinitionFormal TMS Tools Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Test Strategy DevelopmentTest Scenario DesignAutomated TestingTroubleshooting SIP TracesQuality Metrics TrackingAdversarial TestingEvaluation Framework MaintenanceCI/CD IntegrationData AnalysisSIP Device Provisioning

Soft Skills

Analytical SkillsCreativity In Test DesignCommunication Skills

Tools & Technologies

JiraTMS ToolsCI/CD ToolsSIP DevicesSoftphones

Industry Keywords

AI Voice Virtual AgentLLMASRTTSTelephonyCall RoutingGolden DatasetsSemantic AssertionsPacket CapturesLATAM Candidates

Tech Stack

Tools & technologies

PythonVoIP

About the role

Key responsibilities & impact

Embed fully within a Scrum team and participate in all ceremonies and associated artifacts
Own the end-to-end test strategy for the AI voice virtual agent across LLM, speech, and telephony layers
Design test scenarios for multi-turn dialogues, interruptions, ambiguous input, escalations, and failure recovery
Build and maintain an evaluation framework for non-deterministic output using golden datasets and semantic/rubric-based assertions
Adversarially test hallucinations, incorrect grounding, guardrail bypasses, prompt injections, and unauthorized customer-data disclosure
Prepare and provision SIP devices, endpoints, and end-to-end call scenarios
Diagnose failures across LLM, ASR, TTS, media path, and call-routing layers
Extend automated coverage and integrate test suites into CI/CD release gates
Define and track quality metrics such as task completion, containment, intent accuracy, transcription accuracy, latency, and escalation correctness
Maintain documented test coverage in the TMS with traceability to business requirements
Communicate and escalate quality risks using data and evidence

Requirements

What you’ll need

5+ years in QA, with recent hands-on ownership of a complex, integrated product
Practical experience testing SIP/VoIP systems, including call flows, SIP signaling, media handling, softphones, or SIP trunking
Ability to troubleshoot using SIP traces or packet captures
Demonstrated creativity in test design
Proficiency in Python, sufficient to independently build and maintain automation frameworks
Experience testing non-deterministic or probabilistic systems
Strong analytical skills, including defining quality metrics and using data for release decisions
Hands-on experience with formal TMS tools and Jira
Professional fluency in English, spoken and written
Reliable overlap with US Eastern business hours
Advanced spoken English is required
Candidates must be from LATAM

Benefits

Comp & perks

Remote work opportunity
B2B Employment ($, gross)
Stable job with long-term growth perspective
Competitive salary with annual performance review
Really good hardware
Continuous learning and career growth opportunities
Compensation for professional training, seminars, and conferences
Referral program with rewards
Company-supported English classes