Director, AI – Tooling

Salesforce

. Lead the team accountable for generative AI capabilities, evaluation tooling, observability, and emerging program initiatives powering Salesforce's Help Agent .

Posted 9/23/2026full-timeUnited StatesLead💰 $164,000 - $261,500 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in generative AI capabilities, including retrieval architecture and LLM configurations, while effectively leading cross-functional teams and managing strategic initiatives. Proficient in observability practices and performance monitoring to ensure production health and quality metrics.

Highest-signal resume keywords

Generative AI CapabilitiesRetrieval ArchitectureObservability and Performance MonitoringEvaluations Tooling StrategyNNAOV Program Management

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Generative AI StrategyLLM ConfigurationsPrompt OptimizationAgentic Readiness ScoringAnswer Quality MetricsError AnalysisSession TracingFailure ClusteringCustom Evaluation FrameworksProduction Health Monitoring

Soft Skills

Excellent Communication SkillsStrong Executive PresenceTeam LeadershipStakeholder ManagementTrusted Advisor

Tools & Technologies

SlackbotClaude CodeTesting CenterAgentforce Testing AgentsCentralized Observability Platform

Industry Keywords

AI & ToolingDigital Success Data & AIHelp AgentBusiness Impact MeasurementPipeline Generation

About the role

Key responsibilities & impact

Lead the team accountable for generative AI capabilities, evaluation tooling, observability, and emerging program initiatives powering Salesforce's Help Agent
Set the AI & Tooling team mission, operating model, and priorities within Digital Success Data & AI
Drive the strategic roadmap for Help Agent generative AI capabilities, including retrieval architecture, LLM configurations, prompt and planner optimization, and agentic readiness scoring
Partner with Tech & Product and technology teams to influence the Help Agent roadmap and direct engineering relationships
Own the end-to-end evaluations tooling strategy, including Testing Center, Agentforce Testing Agents, and custom evaluation frameworks
Define and track answer quality metrics and expand evaluation capabilities toward greater scale and automation
Lead the Help Agent observability program, including real-time performance monitoring, session tracing, error analysis, failure clustering, and business impact measurement
Represent Customer Zero on the Tech & Product Observability product roadmap
Oversee the Net New Annual Order Value program, including program goals, intake, pipeline generation, and business outcome reporting
Support creation and delivery of AI Skills via Slackbot, Claude Code, and other applications
Monitor and manage Help Agent production health and coordinate cross-functional remediation
Define and mature the AI Evaluations Framework from manual processes toward scalable, automated production-scale evaluation
Represent AI & Tooling in leadership forums and communicate progress, risks, and strategic direction to senior stakeholders
Establish a centralized observability platform and standardize tooling across the Data & AI team

Requirements

What you’ll need

Excellent communication and presentation skills
Strong executive presence
Proven ability to lead a team and develop talent
Ability to navigate complex stakeholder groups and drive alignment
Ability to act as a trusted advisor to the business
Experience with generative AI capabilities and strategy
Knowledge of retrieval architecture, LLM configurations, prompt and planner optimization, and agentic readiness scoring
Experience with evaluations tooling and custom evaluation frameworks
Ability to define and track answer quality metrics including completeness, correctness, relevance, faithfulness, clarity, and latency
Experience with observability, performance monitoring, session tracing, error analysis, failure clustering, and business impact measurement
Experience managing cross-functional Technology and Product partnerships
Experience with NNAOV program management and pipeline generation
Experience creating and delivering AI Skills using applications such as Slackbot and Claude Code
Ability to monitor production health metrics including latency, engagement rate, escalation rate, resolution rate, and quality scores

Benefits

Comp & perks

Time off programs
Medical insurance
Dental insurance
Vision insurance
Mental health support
Paid parental leave
Life insurance
Disability insurance
401(k)
Employee stock purchasing program
Certain roles may be eligible for incentive compensation
Certain roles may be eligible for equity
Reasonable accommodation during the application or recruiting process