Samsung Food

Artificial Intelligence

• Build the experience-quality evaluation layer for AI-generated coaching in Samsung Health
• Author a failure taxonomy for coaching outputs
• Hand-grade at least 150 traces with open-coded notes, including at least 40 sparse-data synthetic profiles
• Categorize failures into 5–10 categories and quantify their frequency
• Freeze at least 50 traces as a holdout set
• Create binary pass/fail criteria for each taxonomy category
• Write annotation guidelines with pass and fail examples
• Double-code at least 30 traces, resolve disagreements, and maintain a guideline revision log
• Create judge prompts for every criterion
• Validate the LLM judge using true-positive and true-negative rates on development and holdout sets
• Run weekly readouts on helpfulness, relevance, and tone
• Document a revalidation routine triggered by model or prompt changes and quarterly regardless
• Produce a complete playbook covering grading, taxonomy updates, rubric revisions, judge revalidation, and weekly readouts
• Conduct a handover test enabling the internal owner to rerun validation and a weekly readout independently
• Obtain Head of Product sign-off on the taxonomy and rubric
• Collaborate with and hand over the method to an internal owner

AI Quality & Evaluation Lead

Consultant

• Take ownership of currently unratified behavioural parameters inside an AI-powered weight management coaching engine
• Shape engagement mechanics, including nudges and engagement loops, for beta product surfaces
• Review agent behaviour against COM-B and related behaviour-change frameworks
• Complete the ratification register with decisions, evidence, and standing
• Review and edit instructions guiding the engine’s coaching approach, goal-gap analysis, action selection, and user messaging
• Conduct a behavioural-guardrail evidence pass
• Flag and frame items requiring medical sign-off for internal clinical review
• Establish a pass bar and scored golden set for uncalibrated evaluation criteria
• Provide nudge and engagement-loop design recommendations
• Provide behavioural sign-off on intervention contracts from freelance specialists
• Review beta-user system performance and clear review queues
• Complete five defined project milestones over the six-month engagement

AI Quality & Evaluation Lead

Behavioural Science Consultant