AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course
Kubernetes serves as a highly scalable infrastructure for deploying, serving, and managing machine learning models within production settings.
This live, instructor-led session—available either online or onsite—targets intermediate professionals seeking to reliably operationalize ML models on Kubernetes.
By the end of this training, participants will be equipped with the capabilities to:
- Containerize and ready ML models for streamlined deployment.
- Deliver model services using contemporary inference frameworks.
- Enhance workload performance through autoscaling, GPU setup, and resource optimization.
- Apply model release methodologies, including A/B testing and canary releases.
Delivery Format
- A mix of lectures, architectural insights, and facilitated discussions.
- Comprehensive practical tasks featuring realistic deployment contexts.
- Practical implementation within a live Kubernetes environment.
Customization Availability
- Should you require this course to align with your specific environment or toolchain, please reach out to explore customization possibilities.
Course Outline
Getting Machine Learning Models Ready for Deployment
- Packaging models using Docker
- Exporting models from TensorFlow and PyTorch
- Strategies for versioning and storage
Serving Models on Kubernetes
- Introduction to inference servers
- Deploying TensorFlow Serving and TorchServe
- Establishing model endpoints
Optimizing Inference Performance
- Implementing batching strategies
- Managing concurrent request handling
- Tuning latency and throughput
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
Provisioning GPUs and Managing Resources
- Setting up GPU nodes
- Overview of the NVIDIA device plugin
- Defining resource requests and limits for ML workloads
Model Rollout and Release Methodologies
- Blue/green deployments
- Canary rollout patterns
- A/B testing for model assessment
Monitoring and Observability for Production ML
- Metrics for inference workloads
- Best practices for logging and tracing
- Dashboards and alerting systems
Security and Reliability Factors
- Protecting model endpoints
- Network policies and access controls
- Maintaining high availability
Conclusion and Future Steps
Requirements
- Knowledge of containerized application workflows
- Proficiency with Python-based machine learning models
- Basic familiarity with Kubernetes concepts
Target Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course - Enquiry
Testimonials (1)
I like the Kubernetes part of the training as I had some tasks with K8s pods and ports and it was nice to get a better understanding now. Thank you!
Laura Miron - Betfair
Course - Docker and Kubernetes
Related Courses
Advanced Platform Engineering: Scaling with Microservices and Kubernetes
35 HoursThis instructor-led, live training in Botswana (online or onsite) is designed for advanced platform engineers and DevOps professionals who aim to master application scaling through microservices and Kubernetes.
Upon completion of this training, participants will be able to:
- Design and implement scalable microservices architectures.
- Deploy and manage applications on Kubernetes clusters.
- Utilize Helm charts for efficient service deployment.
- Monitor and maintain the health of microservices in production environments.
- Apply best practices for security and compliance within a Kubernetes environment.
AI-Driven Kubernetes Operations: Autoscaling, Scheduling & Resource Optimization
21 HoursCertified Kubernetes Administrator (CKA) - exam preparation
21 HoursCertified Kubernetes Application Developer (CKAD) - exam preparation
21 HoursCertified Kubernetes Security Specialist (CKS)
21 HoursThis instructor-led, live training session in Botswana (conducted online or onsite) is designed for Kubernetes professionals aiming to prepare for the CKS examination.
By the end of this programme, participants will possess the knowledge to secure Kubernetes environments and container-based applications throughout the different stages of an application's life cycle: build, deployment, and runtime.
Cloud‑Native DevOps Mastery: Designing, Deploying & Operating Scalable Kubernetes Microservices
49 HoursThis intensive 7-day programme offers a comprehensive, practical exploration into the design, deployment, and operation of cloud-native applications through contemporary DevOps methodologies.
Participants will delve into crafting scalable microservices architectures, refining container environments, and managing production workloads via Kubernetes. The curriculum encompasses advanced deployment strategies, GitOps-driven automation, and observability techniques to guarantee system reliability and optimal performance.
A significant emphasis is placed on navigating real-world operational challenges, including incident response, failure simulation, and root cause analysis. The programme wraps up by leveraging AI-powered tools to facilitate troubleshooting and expedite operational decision-making.
Upon completion of the training, participants will possess a solid grasp of how to build, deploy, monitor, and sustain resilient distributed systems within a Kubernetes-based ecosystem.
Introduction to Containers, Kubernetes & OpenShift
35 HoursDiscover the essential concepts of containers, Kubernetes, and OpenShift through a practical, hands-on course tailored for developers, DevOps engineers, and IT professionals. Learners will master the creation of containerized applications, workload deployment, Kubernetes resource management, and the use of OpenShift to enhance modern application delivery across cloud and hybrid landscapes.
Container Platforms Administration with Docker, Kubernetes and OpenShift
35 HoursLearn how to build, deploy, and administer containerised environments using Docker, Kubernetes, and OpenShift. This five-day, hands-on training covers container images, Kubernetes workloads, cluster networking, storage, security, monitoring, and practical OpenShift administration. Participants gain the skills needed to operate modern container platforms and troubleshoot applications across development and production environments.
Docker and Kubernetes: Building and Scaling a Containerized Application
21 HoursIn this instructor-led live training in Botswana (onsite or remote), participants will learn how to create and manage Docker containers, then deploy a sample application inside a container. Participants will also learn how to automate, scale, and manage their containerized applications within a Kubernetes cluster. Finally, the training goes on to more advanced topics, walking participants through the process of securing, scaling and monitoring a Kubernetes cluster.
By the end of this training, participants will be able to:
- Set up and run a Docker container.
- Deploy a containerized server and web application.
- Build and manage Docker images.
- Set up a Docker and Kubernetes cluster.
- Use Kubernetes to deploy and manage a clustered web application.
- Secure, scale and monitor a Kubernetes cluster.
Docker and Kubernetes
21 HoursTraining Objectives: Acquire theoretical and practical skills in Docker and Kubernetes.
Docker and Kubernetes advanced
35 HoursThe 'Advanced Docker and Kubernetes' course is a comprehensive training covering the Kubernetes platform and its ecosystem. It targets participants with varying levels of experience, ensuring a full range of both basic and advanced Kubernetes features. It includes key concepts such as Containers, Labels, Controllers, Services, Secrets, Persistent Data Volumes, Namespaces, Quotas, Container Networking Model, Service Discovery, Scaling, Load Balancing, Cluster Management, Kubernetes Installation, Cluster Security, Access Control, Controller High Availability, Monitoring and Logging, Application Auto-scaling, Advanced Scheduling, Microservices-based Applications, Application Design Patterns, and Application Deployment on a Kubernetes Cluster.
Edge AI on Kubernetes: Deploying Intelligent Applications at the Edge
21 HoursDeploying Kubernetes Applications with Helm
7 HoursThis instructor-led, live training in Botswana (online or onsite) is aimed at engineers who wish to use Helm to streamline the process of installing and managing Kubernetes applications.
By the end of this training, participants will be able to:
- Install and configure Helm.
- Create reproducible builds of Kubernetes applications.
- Share applications as Helm charts.
- Run third-party applications saved as Helm charts.
- Manage releases of Helm packages.
Introduction to Minikube and Kubernetes
21 HoursThis instructor-led live training in Botswana (online or on-site) is designed for software developers and DevOps professionals at beginner to intermediate levels who wish to learn how to set up and manage a local Kubernetes environment using Minikube.
By the end of this training, participants will be able to:
- Install and configure Minikube on their local machine.
- Grasp the fundamental concepts and architecture of Kubernetes.
- Deploy and manage containers using kubectl and the Minikube dashboard.
- Set up persistent storage and networking solutions for Kubernetes.
- Utilize Minikube for developing, testing, and debugging applications.
Minikube for Developers
14 HoursThis instructor-led, live training in Botswana (online or onsite) is designed for intermediate developers and DevOps engineers who want to utilise Minikube as part of their development workflow.
By the end of this training, participants will be able to:
- Set up and manage a local Kubernetes environment using Minikube.
- Understand how to deploy, manage, and debug applications on Minikube.
- Integrate Minikube into their continuous integration and deployment pipelines.
- Optimize their development process using Minikube's advanced features.
- Apply best practices for local Kubernetes development.