AI Software Development Engineer in Test – SDET

Newfold Digital

. Design, validate, and optimise testing strategies for AI-powered applications, copilots, chatbots, RAG pipelines, and autonomous AI agents .

Posted 11/10/2026full-timeRemote • Argentina, IndiaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in AI testing strategies, including Prompt Engineering, Context Engineering, and automated validation pipelines. Proficient in Python programming and cloud-based testing across platforms like AWS, Azure, and GCP, with a strong foundation in Software Testing and Quality Engineering principles.

Highest-signal resume keywords

Python ProgrammingAI/LLM TestingCI/CD PipelinesDocker & KubernetesSoftware Testing & Quality Engineering

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI Evaluation FrameworksAutomated Validation PipelinesSQL Data ValidationDebuggingAnalytical SkillsTroubleshootingPerformance TestingSecurity TestingContract TestingChaos Testing

Soft Skills

CollaborationAnalytical ThinkingProblem-Solving

Tools & Technologies

Robot FrameworkPlaywrightLangChainLangGraphLlamaIndexCrewAISemantic KernelCursorGitHub CopilotOpenAI APIs

Industry Keywords

AI GovernanceMLOps WorkflowsAgile/ScrumMicroservices TestingUI AutomationSynthetic MonitoringProduction ObservabilityVector DatabasesEmbeddingsRAG Systems

Tech Stack

Tools & technologies

AWSAzureCloudDistributed SystemsDockerGoogle Cloud PlatformGraphQLKubernetesMicroservicesPythonSQL

About the role

Key responsibilities & impact

Design, validate, and optimise testing strategies for AI-powered applications, copilots, chatbots, RAG pipelines, and autonomous AI agents
Develop AI evaluation frameworks for hallucination detection, groundedness, toxicity and bias, response quality, context relevance, prompt regression, determinism, consistency, safety, and guardrails
Build automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics
Create synthetic and benchmark datasets for AI testing scenarios
Test multi-agent orchestration workflows and agent memory/context retention
Validate MCP integrations and tool interoperability
Work on Context Engineering, Prompt Engineering, and Prompt Optimisation strategies
Evaluate latency, token usage, model cost optimisation, throughput, and scalability
Develop and maintain scalable UI/API automation frameworks using Python and testing tools
Automate testing for REST APIs, GraphQL APIs, event-driven systems, streaming systems, webhooks, microservices, and MFE architectures
Build reusable automation libraries, utilities, and AI-assisted testing workflows
Integrate automation into CI/CD pipelines and quality gates
Validate RAG architectures, vector databases, embeddings, retrieval accuracy, re-ranking, and vector search relevance
Build or validate AI agents and workflows involving tool calling, function calling, agent memory, planning/execution, multimodal systems, and structured outputs
Maintain quality workflows integrated with CI/CD platforms
Work with Docker and Kubernetes environments
Perform cloud-based testing and deployments across AWS, Azure, OCI, and GCP
Validate distributed systems reliability and resiliency
Monitor AI system behaviour using observability, logging, and tracing tools
Contribute to shift-left and quality engineering best practices
Collaborate with Engineering, AI/ML, Platform, Product, and DevOps teams

Requirements

What you’ll need

Strong programming experience in Python
Strong understanding of Software Testing & Quality Engineering principles
Hands-on experience with AI/LLM testing, Prompt Engineering, Context Engineering, AI Evals, RAG systems, and AI Agents
Experience with CI/CD pipelines, Docker & Kubernetes, cloud platforms (AWS/Azure/OCI/GCP), API testing & automation, microservices testing, and UI automation
Experience with SQL and data validation
Strong debugging, analytical, and troubleshooting skills
Experience working in Agile/Scrum environments
Experience with Robot Framework preferred
Experience with Playwright preferred
Knowledge of LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel, and MCP preferred
Hands-on experience with Cursor, GitHub Copilot, Claude, Gemini, and OpenAI APIs preferred
Experience with vector databases and embeddings preferred
Exposure to MLOps workflows and ML lifecycle testing preferred
Knowledge of performance testing and security testing preferred
Experience validating AI governance/compliance requirements preferred
Experience with contract testing, chaos testing, or resilience engineering preferred
Familiarity with accessibility testing and cross-browser/device testing preferred
Exposure to synthetic monitoring and production observability preferred
Bachelor’s/Master’s degree in Computer Science, Engineering, AI/ML, or related field preferred
Equivalent hands-on experience and demonstrable projects highly valued

Benefits

Comp & perks

Work-from-home, hybrid, or work-from-the-office environment scenarios
OSDE 310 Health Care
Monthly Salary Reviews
Tuition Reimbursement
Internet Reimbursement
In-company English Lessons
LinkedIn Learning
Referral Program Bonus
F5 extra days off
Annual Bonus
Opportunity to work on global projects with a multicultural team
Learning and development opportunities
20+ affinity groups for networking and connection
Mentoring and career growth opportunities