Lead AI Engineer

McKesson

. Define the technical architecture and roadmap for LLM/agentic systems, including model routing, connector/integration patterns, RAG pipelines, and prompt/evaluation frameworks .

Posted 10/27/2026full-timeColumbus • Ohio • United StatesSenior💰 $155,300 - $258,800 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in defining technical architecture for LLM and agentic systems, ensuring compliance with security and regulatory standards while mentoring engineering teams. Proficient in building production-level AI applications and leading architectural decisions in complex environments.

Highest-signal resume keywords

Technical Leadership in AI/MLProduction LLM Application DevelopmentArchitecture Standards SettingExperience with AI Governance FrameworksHands-on Expertise in Python and Java

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI/ML Systems DevelopmentPrompt EngineeringRAG/GraphRAGTool Integration ProtocolsREST APIsEvent-Driven ArchitectureMicroservicesCloud-Native DesignVector Retrieval SystemsMLOps

Soft Skills

Clear CommunicationMentoring

Tools & Technologies

AzureAWSGCPConfluenceJira

Certifications & Qualifications

Bachelor's Degree in Computer ScienceMaster's or Doctorate Preferred

Industry Keywords

Healthcare ITRegulated Data EnvironmentsAI GovernanceAI Tooling VendorsModel Providers

Tech Stack

Tools & technologies

AWSAzureCloudGoogle Cloud PlatformJavaMicroservicesPython

About the role

Key responsibilities & impact

Define the technical architecture and roadmap for LLM/agentic systems, including model routing, connector/integration patterns, RAG pipelines, and prompt/evaluation frameworks
Lead architecture reviews for new AI capabilities and ensure security, privacy, and data-provenance requirements are met
Drive the technical design of shared AI platform capabilities, including knowledge retrieval, citation/traceability, and permission-aware data access
Architect and build high-complexity components for priority AI delivery initiatives
Set coding, architecture, and evaluation standards for Senior and Associate AI Engineers
Run design reviews and pair on complex agentic workflows
Establish secure, observable, cost-attributed AI operations, including spend attribution, abuse monitoring, model evaluation, and rollback strategies
Partner with engineering managers, product, platform, and Legal/Compliance teams on architecture decisions
Evaluate foundation model providers and AI tooling vendors; make build-vs-buy and model-selection recommendations
Ensure AI systems handling protected health information meet regulatory and enterprise governance requirements
Represent AI architecture in cross-company forums and influence enterprise-wide AI platform direction
Mentor Senior and Associate AI Engineers

Requirements

What you’ll need

Bachelor's degree or equivalent experience; fewer years required with a relevant Master's or Doctorate
Typically 8+ years of software engineering experience, including 3+ years focused on AI/ML, LLM, or NLP systems in production
Demonstrated technical leadership above a Senior AI Engineer level, such as owning architecture for a team or platform area
Deep hands-on expertise building production LLM applications, including prompt engineering, RAG/GraphRAG, agentic workflows, tool use, and evaluation harnesses
Experience with commercial large language model APIs such as Anthropic and OpenAI
Experience with tool-integration protocols such as Model Context Protocol (MCP)
Strong software engineering fundamentals in Python and/or Java, REST APIs, event-driven architecture, microservices, and cloud-native design
Azure preferred; AWS/GCP acceptable
Experience with vector/graph retrieval systems, embeddings, and production-scale knowledge-base architectures
Working knowledge of LLMOps/MLOps, including usage attribution, cost/spend monitoring, abuse detection, and model evaluation pipelines
Ability to set architectural standards and mentor engineers through design review and pairing
Familiarity with regulated/healthcare data environments and AI governance frameworks is a plus
Advanced degree in Computer Science, AI/ML, or a related technical field preferred
Prior experience leading AI/ML platform or applied-AI teams in healthcare IT or another highly regulated industry
Experience modernizing legacy systems toward AI-embedded workflows
Experience with Confluence/Jira-based engineering governance and enterprise architecture review boards
Awareness of the GenAI/agentic landscape and continuous learning
Experience managing relationships with AI tooling vendors and model providers
Clear written and verbal communication for technical and executive audiences
No sponsorship presently or in the future

Benefits

Comp & perks

Competitive compensation package as part of McKesson's Total Rewards
Annual bonus or long-term incentive opportunities may be offered
Equal employment opportunities
Reasonable accommodation for job search or application