Software Development Engineer in Test – Agentic Automation

CrowdStrike

. Design, develop and maintain automated test suites and frameworks for distributed, cloud-based systems .

Posted 11/2/2026full-timeRemote • CanadaMid-LevelSenior💰 CA$120,000 - CA$170,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and maintaining automated test frameworks for distributed cloud-based systems, with strong programming skills in Python, Java, or Node.js. Proven ability to integrate testing processes with CI/CD pipelines and utilize AI technologies to enhance testing and decision-making.

Highest-signal resume keywords

Automated Test Framework DevelopmentCI/CD Pipeline IntegrationPerformance and Load TestingAI Technologies UtilizationTest Automation Development

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Python ProgrammingJava ProgrammingNode.js ProgrammingTest AutomationPerformance TestingLoad TestingAPI TestingDistributed SystemsData StructuresAlgorithms

Soft Skills

Skeptical MindsetVerification-First Approach

Tools & Technologies

JenkinsGitLab CI/CDLocustK6GrafanaPrometheusCloudWatch

Industry Keywords

AI Output EvaluationEnd-to-End TestingTest Data SetupFailure TriageRelease ValidationCybersecurityThreat Intelligence

Tech Stack

Tools & technologies

CloudCyber SecurityDistributed SystemsGrafanaGRPCJavaJavaScriptJenkinsNode.jsPrometheusPython

About the role

Key responsibilities & impact

Design, develop and maintain automated test suites and frameworks for distributed, cloud-based systems
Maintain and extend agentic test generation and analysis tooling built into major testing frameworks
Build AI-powered workflows and agent skill files to automate manual test toil, including test data setup, failure triage and release validation
Maintain eval suites, golden datasets and scoring pipelines for AI agents, scoring correctness, tool use, trajectory, faithfulness, safety, latency and cost
Enforce CI eval gates so changes to prompts, models, tools or orchestration are measured before release
Feed production failures back into evaluation sets
Detect and mitigate unreliable AI output, including hallucinated assertions, flaky or redundant tests, incorrect failure triage and judge bias
Design, run and analyze performance, load and stress tests to find bottlenecks and validate scalability across distributed services
Validate integration of platform systems, services, data and applications, as well as customer-facing features end-to-end
Review product, system and agent designs for testability, evaluability, flaws and improvements
Share knowledge and contribute to the team's quality and evaluation standards

Requirements

What you’ll need

Degree in Computer Science or equivalent experience (OOP, data structures, algorithms, distributed systems)
5+ years in software quality, including 3+ years of hands-on SDET or test automation development
Strong programming skills in at least one of Java, Node.js or Python, with Python preferred for eval and agent work
Proven experience designing, building and maintaining test frameworks integrated with CI/CD pipelines (e.g., Jenkins, GitLab CI/CD)
Experience testing web services and APIs (REST, gRPC, JSON)
Hands-on experience with performance and load testing using tools such as Locust or k6, including designing realistic workloads and interpreting results (latency percentiles, throughput, error rates)
A skeptical, verification-first mindset toward AI output, with a clear sense of when to use deterministic assertions and when to use probabilistic evaluation
Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes
Expected to maintain legal entitlement to work in Canada
Subject to background checks, including criminal record, credit, and/or reference checks
Bonus: End-to-end eval-driven development experience, including release gating, trajectory and tool-use evals, and calibrating LLM-as-judge against human review
Bonus: Prompt engineering and context or tool design for agents
Bonus: Integrating performance tests into CI/CD and using observability tools (e.g., Grafana, Prometheus, CloudWatch)
Bonus: Background in cybersecurity, threat intelligence, malware analysis or sandboxing/detonation environments

Benefits

Comp & perks

Market leader in compensation and equity awards
Comprehensive physical and mental wellness programs
Competitive vacation and holidays for recharge
Paid parental and adoption leaves
Professional development opportunities for all employees regardless of level or role
Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections
Vibrant office culture with world class amenities
Variable/incentive compensation
Equity
Equal pay for equal work