Director, AI Assisted Solutions

The College Board

. Lead end-to-end delivery for assigned College Board assessment-program engagements .

Posted 9/26/2026full-timeRemote • United StatesLead💰 $88,000 - $135,000 per yearWebsite

About the role

Key responsibilities & impact

Lead end-to-end delivery for assigned College Board assessment-program engagements
Define problems and requirements, design and build AI tools, prompts, and workflows
Deliver working solutions with training and adoption plans
Evaluate AI-generated output for construct misalignment, flawed reasoning, bias, and factual errors
Own final quality sign-off on tools delivered to partner teams
Establish quality standards and evaluation criteria, including rubrics, benchmarks, and scoring frameworks
Identify requirements for systems-based solutions requiring additional stakeholder input and resources
Diagnose cultural, staffing, process, and technology gaps affecting scale
Shape engagement priorities and program roadmaps for AI-enabled item development
Define agentic workflow standards and inputs
Inform enterprise-wide agentic-human hybrid architecture, strategy, and roadmaps
Represent supported programs in cross-program tooling and methodology discussions
Design and deliver AI-assisted and agentic workflow training
Develop process documentation for responsible tool adoption
Establish and document impact metrics
Uphold College Board assessment design and development standards
Experiment with emerging AI tools, models, and techniques
Present learnings to teams and internal communities of practice
Attend and represent College Board at relevant external meetings and conferences

Requirements

What you’ll need

7+ years of experience in standardized assessment item development across multiple item types
Expert-level command of item-writing craft and construct alignment
Deep subject-matter expertise in Reading Comprehension, Writing, Math, or Science assessment
3+ years of hands-on experience with AI-assisted or agentic techniques, including AIG, RAG, prompt engineering, and context engineering
Demonstrated experience driving process or technology improvements to adoption with measurable impact
Working knowledge of large language models, hallucinations, context-window constraints, and training-data staleness
Experience with agentic and multi-agent workflows, tool-calling, and chained or orchestrated prompts
Experience with vibe coding or coding and rapid prototyping
Ability to independently build working scripts, tools, or interactive prototypes
Experience designing processes, educating others, and enabling adoption of tools, processes, and systems
Ability to bridge technical and non-technical conversations
Proven cross-functional collaboration skills
Excellent organizational and planning skills
Experience implementing processes and procedures, managing project plans and schedules, and maintaining tracking documentation
Experience with item- and test-development life cycles, digital item authoring and review tools, and content management/item banking systems
Comfort working with ambiguity in a fast-paced environment
Ability to travel 6–8 times per year
Authorization to work in the United States
Passion for expanding educational and career opportunities and mission-driven work
Curiosity and willingness to experiment with emerging technologies and AI-driven solutions
Clear and concise written and verbal communication skills
Learner’s mindset and commitment to growth
Drive for impact and excellence
Collaborative and empathetic approach
Master’s degree or equivalent education and experience strongly preferred

Benefits

Comp & perks

Meaningful career
Supportive team
Comprehensive package designed to help employees thrive
Fair and competitive compensation
Location-adjusted salaries
Transparent, market-data-rooted compensation offers
Fully remote work option
Option for hybrid work for candidates living near College Board offices, with Tuesdays and Wednesdays in office
Travel to relevant external professional meetings and conferences