Principal AI Engineer

Requirements:

• Bachelors degree or equivalent experience, typically with 12+ years in software engineering, including 6+ years developing production AI/ML, LLM, or NLP systems and 3+ years as a senior individual-contributor technical leader; fewer years may be required with a relevant masters or doctoral degree.
• Demonstrated technical leadership beyond the Lead AI Engineer level, including setting direction and standards across multiple teams.
• Track record of defining technical strategies adopted by multiple teams.
• Hands-on expertise building production LLM applications at scale, including prompt engineering, RAG, agentic workflows, tool use, and evaluation harnesses, with sound judgment about durable technical approaches.
• Experience with commercial LLM APIs, such as Anthropic or OpenAI, and tool-integration protocols such as MCP, including assessment of emerging alternatives.
• Strong software engineering fundamentals, including Python and/or Java, REST APIs, event-driven architecture, microservices, and cloud-native design; Azure is preferred, with AWS or GCP also acceptable.
• Experience with vector and graph retrieval systems, embeddings, and production-scale knowledge-base architectures across multiple products.
• Deep working knowledge of LLMOps/MLOps, including usage attribution, cost monitoring, abuse detection, and model evaluation pipelines, with experience designing these systems from scratch.
• Ability to influence and align senior engineering and executive stakeholders without direct authority.
• Familiarity with regulated or healthcare data environments and AI governance, including model risk review, connector approval tiers, and PHI handling.
• Preferred: Advanced degree in Computer Science, AI/ML, or a related technical field.
• Preferred: Experience as a Principal, Staff, or Distinguished Engineer setting AI/ML strategy across multiple teams in healthcare IT or another highly regulated industry.
• Preferred: Publications, patents, open-source contributions, or industry presentations on applied AI/LLM systems.
• Preferred: Experience developing enterprise AI governance policy with Legal, Compliance, and Privacy leaders.
• Additional strengths: Awareness of the evolving GenAI and agentic landscape and ability to distinguish durable technical opportunities from hype; experience managing enterprise relationships with AI infrastructure vendors and model providers; exceptional written and verbal communication for executive, board-level, and external technical audiences.

Responsibilities:

• Own our multi-year AI platform strategy and roadmap across teams, covering model strategy, retrieval and RAG architecture, connector ecosystems, and evaluation frameworks.
• Serve as the final technical escalation point for our most complex and ambiguous AI platform architecture and reliability challenges across teams and systems.
• Establish and evolve architecture, security, and evaluation standards adopted by multiple Lead AI Engineers and their teams, auditing them as our platform and the threat landscape change.
• Advise VP- and executive-level leaders on AI platform strategy, model risk, and investment trade-offs.
• Lead architecture and prototyping for our most ambiguous and high-risk AI initiatives where existing teams or patterns do not provide a solution.
• Develop Lead and Senior AI Engineers as technical leaders through design reviews, coaching, and structured knowledge-sharing across teams.
• Shape enterprise AI governance with Legal, Compliance, and Privacy leaders, including model risk review, connector and tool approval frameworks, and PHI-handling standards.
• Direct our company-wide evaluation and long-term strategy for foundation model providers and AI infrastructure vendors.
• Represent our AI engineering capabilities in industry forums and technical relationships with partners and vendors, and support recruiting senior technical talent.
• Influence AI platform direction and investment across our broader enterprise, beyond any single business unit.

Technologies:

• AI
• AWS
• Azure
• Cloud
• GCP
• Support
• Java
• LLM
• MCP
• MLOps
• Python
• RAG
• REST
• Security
• microservices

More

We are McKesson, an impact-driven Fortune 10 healthcare company delivering insights, products, and services to help make quality care more accessible and affordable. We foster a culture of growth, meaningful impact, and new ideas, and work together to shape the future of health for patients, communities, and our people. This Principal AI Engineer is our highest-level individual-contributor technical leader for the AI platform, working across the organization in a highly regulated healthcare environment. We prioritize candidates residing in the Columbus, Ohio metropolitan area and are unable to provide sponsorship now or in the future. We offer a competitive compensation package based on factors including performance, experience, skills, equity, and geographic market; additional compensation may include an annual bonus or long-term incentives. The base pay range is $195,000–$325,000. We are an equal opportunity employer and provide reasonable accommodations for applicants with disabilities.

last updated 43 week of 2026