Flodesk

AI Engineer

• Manage member-facing AI behavior across prompt-to-create, agentic editing and future jobs such as segmentation, scheduling, analytics and recommendations
• Create, test, version and document prompts and agent instructions
• Define system use of context, member data, tools and product state
• Build and prototype agent loops, tools and structured interfaces
• Own AI evaluation practice, including representative datasets, behavioral scenarios, scoring rubrics, regression suites and human review
• Build and operate evaluation harnesses, monitoring and quality dashboards
• Diagnose failures across prompts, context, orchestration, models, tools, data and product code
• Set quality baselines and release gates for prompt, model and agent changes
• Turn production failures, traces and member feedback into durable regression cases
• Partner with product, design, marketing and copy experts to encode Flodesk's point of view into the system

Senior Applied AI Engineer, Agent Quality & Evaluations

• Manage member-facing AI behavior across prompt-to-create, agentic editing, segmentation, scheduling, analytics and recommendations
• Create, test, version and document prompts and agent instructions
• Define required context, member data, tools and product state, translating them into requirements and acceptance criteria
• Own AI evaluation practice, including datasets, behavioral scenarios, scoring rubrics, regression coverage and human review
• Run recurring evaluation and quality monitoring; set baselines and release gates for prompt, model and agent changes
• Review traces and AI interactions to diagnose failures across prompts, context, models, tools, product structure and implementation
• Fix owned issues and provide actionable changes for engineering
• Turn member research, support feedback, analytics, production failures and domain expertise into priorities, experiments and regression cases
• Own day-to-day product direction and learning agenda for the AI experience
• Partner with product, design, marketing, copy and engineering

Senior Applied AI Engineer, Agent Quality & Evaluations

AI Product Lead – AI Experience, Quality