Anaplan

SDET

• Design and architect comprehensive test plans, evaluation frameworks, and quality metrics for LLM-powered applications and agentic workflows
• Develop robust end-to-end automated test suites in Python across evaluations, API checks, UI checks, and performance tests
• Create automated prompt testing systems and regression suites to detect unintended changes in model behavior
• Establish evaluation criteria for GenAI quality across accuracy, relevance, safety, consistency, and latency
• Perform adversarial testing (red-teaming) to identify and mitigate hallucinations, biases, and security vulnerabilities
• Develop internal tools and testing frameworks for validating GenAI implementations
• Deploy real-time monitoring and alerting systems to detect quality degradation in production AI features
• Own the quality and testing strategy, maintain high-fidelity test datasets, and define QA best practices

Explore Senior Software Development Engineer in Test – AI at Anaplan. Review requirements and application details.

Senior Software Development Engineer in Test – AI

• Develop and execute end-to-end automated test coverage across evaluations, APIs, UIs, and performance tests using Python
• Implement testing strategies for conversational AI, agentic systems, and LLM-powered workflows
• Run and maintain automated prompt-testing suites, including regression tests for unintended model-behaviour changes
• Support evaluation frameworks measuring GenAI accuracy, relevance, safety, consistency, and latency
• Curate high-quality test datasets and golden examples representing diverse user scenarios and edge cases
• Assist with adversarial testing for model hallucinations and biases, and support production alerting integration
• Use internal tools and testing frameworks to help developers validate GenAI code and assist with user acceptance testing
• Own automated coverage for the workstream and document testing procedures and known issues

Explore QA Automation Engineer – AI at Anaplan. Review requirements and application details.