Principal AI Cloud Engineer

Requirements:

• Bachelors, masters, or doctoral degree in computer science, engineering, or a related field
• At least 7 years of experience building large-scale distributed cloud infrastructure
• At least 5 years of hands-on experience with Azure or AWS
• Demonstrated experience with AI/ML infrastructure, including GPU clusters, Kubernetes, CI/CD, and observability
• Strong skills in infrastructure as code (Terraform or Bicep), Kubernetes, networking, and security
• Expertise in cloud-native patterns, including containers, service mesh, and serverless architectures
• Familiarity with MLOps/LLMOps infrastructure, such as model serving, feature stores, and vector databases
• Python programming skills for infrastructure automation and experience with Go or TypeScript for tooling
• Understanding of frontend and backend integration for AI services
• Preferred: GPU optimization experience, including CUDA/NCCL or TensorRT-LLM
• Preferred: Familiarity with Prometheus, Grafana, or OpenTelemetry
• Preferred: Experience with event streaming such as Kafka or Azure Event Hubs, and real-time systems
• Preferred: Experience with AI platform products such as Azure ML, MLflow, KServe, or Hugging Face

Responsibilities:

• Design, build, and operate cloud-native infrastructure for machine learning and generative AI workloads, including compute, networking, storage, databases, and security
• Establish monitoring and reliability practices for AI infrastructure, including metrics, logging, tracing, and service-level objectives
• Build CI/CD and GitOps pipelines for infrastructure as code and AI platform components
• Manage AI infrastructure costs through GPU rightsizing, caching, inference optimization, and cost governance
• Enable AI platform services with secure APIs, microservices, event-driven architectures, and custom model runtimes
• Support retrieval-augmented generation systems, including embeddings, chunking, and retrieval pipelines
• Build scalable serving infrastructure for large language models and machine learning models, including caching and token optimization
• Define and evolve cloud AI infrastructure architecture, including Kubernetes, service mesh, serverless patterns, multi-region availability, disaster recovery, and compliance readiness
• Set standards for containerization, infrastructure as code, and secure networking for AI systems
• Implement defense-in-depth security and Responsible AI controls, including least-privilege access, private networking, encryption, data residency, lineage, and audit readiness
• Lead infrastructure discovery and solution design with stakeholders
• Operate platforms using SRE practices, including error budgets, incident response, and chaos testing
• Mentor engineers and develop reusable infrastructure modules, templates, and paved paths
• Define reference and production-ready cloud solutions for AI and generative AI, and build reusable, secure, observable components such as APIs, SDKs, microservices, and pipelines
• Operationalize large language models and retrieval-augmented generation with appropriate controls and Responsible AI safeguards
• Guide platform roadmaps to improve delivery speed, reduce risk, and achieve measurable business outcomes

Technologies:

• AI
• AWS
• Azure
• Backend
• CI/CD
• Cloud
• CUDA
• DevSecOps
• Frontend
• GitOps
• Grafana
• Support
• Kafka
• Kubernetes
• LLM
• Machine Learning
• MLflow
• MLOps
• Model Serving
• OpenTelemetry
• Prometheus
• Python
• Security
• Serverless
• Terraform
• TypeScript
• microservices
• ASP.NET
• DevOps
• IAM
• Network
• RAG
• vLLM

More

We are BMO, a global financial group guided by our purpose to grow the good in business and life. Our diverse, collaborative team builds cloud-native AI solutions that help teams across the bank innovate securely and responsibly. As Principal AI & Cloud Engineer, you will shape our enterprise AI cloud strategy and technical standards, working across cloud infrastructure, data and feature pipelines, model serving, LLMOps, and DevSecOps. The role is based at 100 King Street West, with an application deadline of November 30, 2026. The salaried pay range is $105,000–$215,000; compensation varies with location, skills, experience, education, and qualifications. Our total rewards may include incentives, bonuses, health insurance, tuition reimbursement, accident and life insurance, and retirement savings plans. We offer training, coaching, management support, and opportunities to build skills and make an impact, and provide accommodations during the selection process upon request.

last updated 43 week of 2026