Senior Legal AI Prompter, Evaluator

Thomson Reuters

. Create, test, and refine prompts, skills, and agent configurations for CoCounsel Legal .

Posted 11/9/2026full-timeUnited StatesSenior💰 $119,700 - $222,300 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in applying Generative AI and prompt engineering within legal contexts, leveraging a Juris Doctor degree and extensive experience in legal practice. Proficient in Python programming and collaborating with cross-functional teams to integrate LLMs into innovative legal products.

Highest-signal resume keywords

Juris Doctor DegreeGenerative AI ExpertisePython ProgrammingLegal Product ExperiencePrompt Engineering

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Prompt EngineeringPython ScriptingLLM EvaluationAutomated TestingData AnalysisRubric DesignFailure Mode AnalysisCode DebuggingExperimentationDataset Creation

Soft Skills

Excellent CommunicationCollaboration SkillsMentoringAdaptabilityStrategic Feedback

Tools & Technologies

GitOpenAIAnthropicGeminiLegal Research ToolsAgentic SystemsRAG Systems

Certifications & Qualifications

Bar Admission

Industry Keywords

Legal TechnologyLLMLegal PracticeProduct DevelopmentCross-Functional Collaboration

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

Create, test, and refine prompts, skills, and agent configurations for CoCounsel Legal
Experiment with Thomson and other LLMs on legal tasks and compare results
Build gold-standard datasets with subject matter experts
Coordinate evaluations and design rubrics, metrics, success criteria, and LLM-as-judge prompts
Write and run scripts, experiments, and automated tests across large datasets and time periods
Monitor production performance and analyze failure modes
Identify whether failures originate in prompting, tooling, retrieval, or content
Provide strategic feedback to software engineers and data scientists
Collaborate with cross-functional teams integrating LLMs into multi-step systems
Evaluate the feasibility and potential impact of new product ideas
Engage legal professionals, industry leaders, and internal stakeholders to gather insights and validate product concepts
Stay informed about legal technology and LLM developments
Mentor colleagues
Draft guidance materials, reusable prompts, and best-practice documentation

Requirements

What you’ll need

Juris Doctor degree or equivalent law degree
Bar admission in a US or Canadian jurisdiction
5–7 years of combined experience, including at least 2–3 years of legal practice
Related experience such as collaborating with software engineers, building or testing products, applying generative AI, or writing code
Proven expertise applying generative AI, including prompt engineering and evaluating model outputs
Understanding of how LLMs work and where they fail
Proficiency working in a Python code base and using Git
Ability to write, modify, and debug Python scripts
Experience calling LLMs through code, such as OpenAI, Anthropic, or Gemini
Experience using coding agents to build and iterate on work
Excellent communication and collaboration skills
Substantial experience with legal products and technology, including online legal research and practice tools
Hands-on experience designing evaluations, such as rubrics, human grading frameworks, or LLM-as-judge prompts
Familiarity with agentic and RAG systems, including retrieval, tool use, and prompting effects on output quality
Comfortable with ambiguity and able to learn new tools and models quickly

Benefits

Comp & perks

Flexible hybrid working environment
Work from anywhere for up to 8 weeks per year
Flexible vacation
Two company-wide Mental Health Days off
Headspace app access
Retirement savings
Tuition reimbursement
Employee incentive programs
Mental, physical, and financial wellbeing resources
Two paid volunteer days off annually
Pro-bono consulting projects and ESG initiatives
Health, dental, vision, disability, and life insurance programs
Competitive 401k plan with company match
Competitive vacation, sick and safe paid time off
Paid holidays, including two company mental health days off
Parental leave
Sabbatical leave
Optional hospital, accident and sickness insurance
Optional life and AD&D insurance
Flexible Spending and Health Savings Accounts
Fitness reimbursement
Employee Assistance Program
Group Legal Identity Theft Protection benefit
529 Plan access
Commuter benefits
Adoption & Surrogacy Assistance
Employee Stock Purchase Plan
Annual Bonus based on enterprise and individual performance