AI Conversation Research, Evaluation Specialist

Gramian Consulting

. Review AI conversations involving real-world interpersonal situations and everyday decision-making .

Posted 10/22/2026full-timeRemote • United States, Saudi Arabia, Canada, United Kingdom, Australia, IrelandMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in qualitative research, client assessment, and conversation analysis, with a strong focus on identifying context and communication patterns. Holds relevant social work or counseling credentials and possesses extensive experience in client-facing roles.

Highest-signal resume keywords

Licensed Social Work CredentialQualitative Research ExperienceClient Assessment SkillsCounseling or Therapy LicenseConversation Analysis Expertise

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Qualitative ResearchConversation AnalysisThematic AnalysisClient AssessmentInterviewingCounselingPsychological AssessmentCase ReviewUser ResearchQualitative Coding

Soft Skills

Analytical ThinkingCommunication SkillsCollaborationAttention to DetailProblem-Solving

Certifications & Qualifications

Master's Degree in Social Work (MSW)LCSWLICSWLMSWLMFTLPCLCPCPhD in PsychologyPsyD in Psychology

Industry Keywords

Human-Computer InteractionSocial WorkCounselingTherapyOrganizational ConsultingMediationCoachingStudent ServicesAdvisory FieldsInterpersonal Situations

About the role

Key responsibilities & impact

Review AI conversations involving real-world interpersonal situations and everyday decision-making
Identify missing context and recommend relevant follow-up questions to improve AI responses
Evaluate whether additional information would materially change an AI-generated recommendation
Distinguish load-bearing information affecting a decision from non-essential or descriptive details
Conduct qualitative reviews and document findings using structured evaluation guidelines
Assess whether conversations consider relevant personal, psychological, social, financial, practical, and organizational factors
Identify missed opportunities for clarification, deeper understanding, and information gathering
Review conversations involving relationships, work, education, family, wellbeing, planning, and resource constraints
Collaborate with research teams to ensure consistent, evidence-based assessments
Maintain accuracy, clarity, and consistency while meeting project deadlines

Requirements

What you’ll need

At least 3–5 years of relevant direct professional experience in one or more listed domains
Licensed Social Work pathway: Master's degree in Social Work (MSW) or an equivalent graduate-level qualification
Licensed Social Work pathway: LCSW, LICSW, LMSW, or an equivalent applicable social work license or credential
Experience in client assessment, case review, interviewing, counseling, or related social-work practice
Counseling, Therapy, or Psychology pathway: Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy license or credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology
Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work
Qualitative Research, HCI, or Communication pathway: MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline
Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding
Ability to analyze conversations systematically and identify missing context, communication patterns, and opportunities for clarification
Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields pathway: relevant professional qualification, credential, or demonstrated subject-matter expertise
At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway
Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work
Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations