FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in AI product strategy, including the development and evaluation of AI agents, while effectively translating technical requirements into actionable specifications. Strong ability to collaborate with engineering teams and communicate complex concepts to diverse stakeholders.
Highest-signal resume keywords
AI Product StrategyLarge Language Models (LLMs)Agent Architecture PatternsProduct Requirement Documents (PRDs)Evaluation Methodologies for AI Systems
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
AI Features DevelopmentPrompt EngineeringContext EngineeringModel EvaluationData AnalysisTechnical Product ManagementQuality Standards EstablishmentSuccess Metrics DefinitionRegression Testing ProcessesAgent Workflows Development
Soft Skills
Excellent Communication SkillsStrong Product JudgmentCollaboration with Engineering Teams
Tools & Technologies
Model Context Protocol (MCP)AI Evaluation MethodologiesData Observability PlatformsData Lineage ToolsMetadata Management PlatformsTeams IntegrationSlack IntegrationServiceNow IntegrationJira Integration
Industry Keywords
AI/ML PlatformsB2B SaaSData PlatformsDeveloper ToolsNon-Deterministic AI Systems
Tech Stack
Tools & technologiesAssemblyServiceNowGo
About the role
Key responsibilities & impact- Own the AI product strategy, roadmap, and prioritisation.
- Define which workflows agents should manage independently, where they should assist users, and where human oversight is necessary.
- Evaluate the feasibility, limitations, failure modes, and long-term maintainability of agentic AI features.
- Define agent autonomy and human-in-the-loop boundaries, gradually expanding autonomous capabilities as reliability and user trust improve.
- Translate customer requirements, technical constraints, and model capabilities into detailed Product Requirement Documents (PRDs) and actionable engineering specifications.
- Partner with engineering teams to design, develop, evaluate, and launch production-ready AI agents.
- Establish quality standards across accuracy, latency, cost, observability, security, and reliability.
- Build and maintain AI evaluation frameworks, including golden datasets, quality rubrics, and regression testing processes.
- Replicate real-world customer environments and incident scenarios to validate agent performance and production readiness.
- Define and monitor AI product success metrics, including resolution accuracy, acceptance rates, time to root cause, deflection rates, and token cost per outcome.
- Analyse production data and customer feedback to identify performance gaps and continuously improve AI capabilities.
- Collaborate with Go-to-Market teams on product launches, technical enablement, and customer-facing documentation.
- Communicate agent capabilities, limitations, context sources, and decision-making processes clearly to customers.
- Stay updated on foundation models, agent frameworks, Model Context Protocol (MCP), tool-calling standards, retrieval techniques, and AI evaluation methodologies.
- Champion a transparent, reliable, and user-friendly experience for data engineers and analysts.
Requirements
What you’ll need- 3+ years of product management experience working on technically complex products, such as AI/ML platforms, data platforms, developer tools, or APIs.
- Hands-on experience building with Large Language Models (LLMs) in a technical role, including shipping production AI features, developing agent workflows, prompt and context engineering, or conducting model evaluations.
- Strong understanding of foundation model capabilities, limitations, and cost, latency, and quality trade-offs.
- Working knowledge of agent architecture patterns, including tool calling, context assembly, retrieval, orchestration, and guardrails.
- Experience with evaluation methodologies for non-deterministic AI systems.
- Strong product judgment and the ability to identify when AI is an appropriate solution and when alternative approaches are more suitable.
- Proven ability to collaborate closely with engineering teams and make informed technical and product trade-offs.
- Excellent communication skills, with the ability to explain complex technical concepts to technical and non-technical stakeholders.
- Bachelor's degree in Computer Science, Data Science, Information Systems, Engineering, or a related technical field, or equivalent practical experience.
- Preferred: Experience with MCP, agent frameworks, or integrating AI capabilities into Teams, Slack, ServiceNow, and Jira.
- Preferred: Previous ownership of an AI or agent-based product within a B2B SaaS or data-focused organisation.
- Preferred: Familiarity with data observability, data lineage, governance, and metadata management platforms.
- Preferred: Background in data engineering or data architecture, or experience building AI solutions on enterprise data systems.
- Preferred: Experience working in a high-growth SaaS or developer-focused product environment.