FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates strong capabilities in AI evaluation, quality engineering, and security analysis, with proficiency in Python and Git. Engages in collaborative problem-solving and effective communication while navigating complex AI systems and methodologies.
Highest-signal resume keywords
Python ProgrammingGit and GitHub ProficiencyAI Evaluation TechniquesQuality EngineeringSecurity Analysis
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Software EngineeringComputer ScienceTest Automation FrameworksCI/CD PipelinesData ManipulationData Quality TechniquesFuzz TestingProperty-Based TestingMetamorphic TestingBenchmark Design
Soft Skills
Clear Written CommunicationComfortable with Ambiguity
Tools & Technologies
AI Assurance ToolkitEvaluation InfrastructureAgentic Engineering WorkflowObservability ToolsTrace Analysis Tools
Industry Keywords
AI AgentsSecurity CompliancePolicy TestingRed-Team EvaluationResponsible AI Risk AssessmentLLM ApplicationsSemantic ModelingKnowledge Graphs
Tech Stack
Tools & technologiesPython
About the role
Key responsibilities & impact- Work directly with a senior architect on the machinery and methodology for evaluating, monitoring, and investigating agentic AI systems
- Complete a scoped project within the AI assurance toolkit focused on evaluation infrastructure, quality engineering, security analysis, or tooling for understanding agent behavior
- Extend the evaluation harness and improve configuration, execution, reproducibility, and comparability of evaluation runs
- Investigate scoring, aggregation, run-to-run variance, sample size, and confidence in evaluation results
- Research agent evaluation tooling and techniques and prototype promising ideas
- Investigate AI-agent security and trustworthiness, including sensitive context, policy compliance, unsafe instructions, tool errors, and reviewable evidence
- Build observability and investigation tools for trace analysis, failure categorization, regression detection, and engineering reports
- Use a GitHub-native, agentic engineering workflow with AI-assisted development tools
- Present findings and form opinions about the work
- Collaborate with experienced software engineers across quality engineering, AI evaluation, reliability analysis, and security-minded investigation
Requirements
What you’ll need- Completed 3rd year of, or recently graduated from, a program in Software Engineering, Computer Engineering, Computer Science, or a related field
- Currently enrolled in full-time education or a recent/upcoming graduate within 12 months of the placement end date
- Demonstrated interest in software quality, reliability, security, or investigation-oriented engineering
- Demonstrated interest in AI agents and their evaluation
- Comfortable writing code in Python or a comparable language
- Comfortable with Git and GitHub
- Clear written communication
- Comfortable with ambiguity
- Nice to have: experience with test automation frameworks, CI/CD pipelines, property-based, fuzz, or metamorphic testing
- Nice to have: exposure to application security, threat modeling, policy testing, red-team style evaluation, secure software development, or responsible AI risk assessment
- Nice to have: exposure to LLM applications, RAG, prompt engineering, agentic frameworks, and evaluation tooling
- Nice to have: familiarity with LLM-as-judge, rubric-based scoring, or benchmark design
- Nice to have: experience analyzing logs, traces, audit records, experiment results, or failure reports
- Nice to have: experience with data manipulation, analysis, data quality, or validation techniques
- Nice to have: exposure to knowledge graphs, semantic modeling, or ontologies
- Nice to have: exposure to supply chain, logistics, manufacturing, or enterprise planning systems
- Nice to have: contributions to open-source projects, research, competitions, or hackathons
Benefits
Comp & perks- Flexible vacation and Kinaxis Days (company-wide days off)
- Flexible work options
- Physical and mental well-being programs
- Regularly scheduled virtual fitness classes
- Mentorship programs, training, and career development
- Recognition programs and referral rewards
- Hackathons
- Recruitment accommodations upon request
