AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course
Kubernetes serves as a robust, scalable platform designed for the deployment, serving, and management of machine learning models within production settings.
This instructor-led live training, available either online or onsite, targets intermediate-level professionals aiming to reliably operationalise ML models on Kubernetes.
Upon successful completion, participants will acquire the ability to:
- Containerise and ready ML models for efficient deployment.
- Deploy models using contemporary inference frameworks.
- Enhance workload performance through autoscaling, GPU configuration, and resource tuning.
- Execute model rollout strategies, including A/B testing and canary deployments.
Course Format
- A blend of lectures, architectural analysis, and guided discussions.
- Comprehensive practical exercises grounded in real-world deployment scenarios.
- Hands-on implementation within a live Kubernetes environment.
Customisation Options
- Should you require this course to be tailored to your specific environment or toolchain, please contact us to explore customisation possibilities.
Course Outline
Preparing Machine Learning Models for Deployment
- Packaging models with Docker
- Exporting models from TensorFlow and PyTorch
- Versioning and storage considerations
Model Serving on Kubernetes
- Overview of inference servers
- Deploying TensorFlow Serving and TorchServe
- Setting up model endpoints
Inference Optimization Techniques
- Batching strategies
- Concurrent request handling
- Latency and throughput tuning
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Configuring GPU nodes
- NVIDIA device plugin overview
- Resource requests and limits for ML workloads
Model Rollout and Release Strategies
- Blue/green deployments
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for ML in Production
- Metrics for inference workloads
- Logging and tracing practices
- Dashboards and alerting
Security and Reliability Considerations
- Securing model endpoints
- Network policies and access control
- Ensuring high availability
Summary and Next Steps
Requirements
- Understanding of containerised application workflows
- Experience with Python-based machine learning models
- Familiarity with Kubernetes fundamentals
Target Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course - Enquiry
Testimonials (2)
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
The training met expectations with its clear explanations, real-world examples, and hands-on labs that made complex topics easy to understand. It provided valuable insights into container orchestration, security, scaling and many other advanced topics.
Anna Wyszomirska-Szmyd - Akamai
Course - Docker and Kubernetes advanced
Related Courses
Advanced Platform Engineering: Scaling with Microservices and Kubernetes
35 HoursThis instructor-led, live training in Botswana (online or onsite) is designed for advanced platform engineers and DevOps professionals who aim to master application scaling through microservices and Kubernetes.
Upon completion of this training, participants will be able to:
- Design and implement scalable microservices architectures.
- Deploy and manage applications on Kubernetes clusters.
- Utilize Helm charts for efficient service deployment.
- Monitor and maintain the health of microservices in production environments.
- Apply best practices for security and compliance within a Kubernetes environment.
AI-Driven Kubernetes Operations: Autoscaling, Scheduling & Resource Optimization
21 HoursKubernetes is a container orchestration platform widely used for managing distributed applications at scale.
This instructor-led, live training (online or onsite) is aimed at advanced-level practitioners who wish to apply AI and machine learning techniques to optimize Kubernetes resource usage, scheduling decisions, and autoscaling strategies.
Upon completion of this programme, participants will be able to:
- Apply AI/ML models to improve workload scheduling decisions in Kubernetes.
- Use predictive analytics to optimize CPU, GPU, and memory allocation.
- Implement intelligent autoscaling using reinforcement learning and metric forecasting.
- Reduce infrastructure cost and latency through automated resource optimization.
Format of the Course
- Instructor-guided technical presentations and deep-dive discussions.
- Hands-on lab work using real Kubernetes clusters.
- Practical exercises applying AI models to real operational scenarios.
Course Customization Options
- To tailor this course to your platform setup or operational requirements, please contact us for customization.
Certified Kubernetes Administrator (CKA) - exam preparation
21 HoursThe Certified Kubernetes Administrator (CKA) programme was established by The Linux Foundation and the Cloud Native Computing Foundation (CNCF).
Kubernetes has emerged as the premier platform for container orchestration.
NobleProg has been providing Docker and Kubernetes training since 2015. With over 360 successfully completed training projects, we have become one of the most recognised training companies globally in the field of containerisation.
Since 2019, we have also assisted our customers in validating their skills within Kubernetes environments by preparing them to pass the CKA and CKAD exams.
This instructor-led, live training (available online or onsite) is designed for System Administrators and Kubernetes users who wish to validate their knowledge by passing the CKA exam.
Furthermore, the training focuses on gaining practical experience in Kubernetes Administration; therefore, we recommend participating even if you do not intend to sit for the CKA exam.
Course Format
- Interactive lectures and discussions.
- Extensive exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customisation Options
- To request customised training for this course, please contact us to arrange.
- To learn more about the CKA certification, please visit: https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka
Certified Kubernetes Application Developer (CKAD) - exam preparation
21 HoursThe Certified Kubernetes Application Developer (CKAD) programme has been developed by The Linux Foundation and the Cloud Native Computing Foundation (CNCF), the host of Kubernetes.
This instructor-led, live training (online or onsite) is aimed at Developers who wish to confirm their skills in design, build, configure, and expose cloud native applications for Kubernetes.
On the other hand, training is also focused on gaining practical experience in Kubernetes application development, so we recommend taking part in it, even if you don't intend to take CKAD exam.
NobleProg have been delivering Docker & Kubernetes training from 2015. With more than 360 successfully completed training projects, we became one of the best known training company worldwide in field of containerization. Since 2019 we are also helping our customers to confirm their performance in k8s environment by preparing them and encouraging to pass CKA and CKAD exams.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
- To learn more about CKAD, please visit: https://training.linuxfoundation.org/certification/certified-kubernetes-application-developer-ckad/
Certified Kubernetes Security Specialist (CKS)
21 HoursThis instructor-led, live training in Botswana (online or on-site) is aimed at Kubernetes practitioners who wish to prepare for the CKS exam.
By the end of this training, participants will know how to secure Kubernetes environments and container-based applications throughout the different stages of an application's life cycle: build, deployment and runtime.
Cloud‑Native DevOps Mastery: Designing, Deploying & Operating Scalable Kubernetes Microservices
49 HoursThis seven-day programme offers an in-depth, practical exploration into the design, deployment, and operation of cloud-native applications through contemporary DevOps methodologies.
Participants will examine how to architect scalable microservices, optimise containerised environments, and manage production workloads using Kubernetes. The curriculum spans advanced deployment techniques, GitOps-driven automation, and observability frameworks to guarantee system reliability and performance.
There is a significant emphasis on addressing real-world operational challenges, such as incident response, failure simulation, and root cause analysis. The programme wraps up by demonstrating the use of AI-powered tools to aid troubleshooting and expedite operational decision-making.
Upon completion, participants will possess a solid understanding of how to construct, deploy, monitor, and sustain resilient distributed systems within a Kubernetes-based ecosystem.
Introduction to Containers, Kubernetes & OpenShift
35 HoursGain a solid grounding in containers, Kubernetes, and OpenShift through practical, hands-on training designed for developers, DevOps engineers, and IT professionals. Participants will learn how to construct containerized applications, deploy workloads, manage Kubernetes resources, and utilise OpenShift to streamline modern application delivery within cloud and hybrid environments.
Docker, Kubernetes and OpenShift 3 for Administrators
35 HoursIn this instructor-led, live training in Botswana, participants will learn how to manage Red Hat OpenShift Container Platform.
By the end of this training, participants will be able to:
- Create, configure, manage, and troubleshoot OpenShift clusters.
- Deploy containerised applications on-premise, in public cloud or on a hosted cloud.
- Secure OpenShift Container Platform
- Monitor and gather metrics.
- Manage storage.
Docker and Kubernetes: Building and Scaling a Containerized Application
21 HoursIn this instructor-led live training in Botswana (onsite or remote), participants will learn how to create and manage Docker containers, then deploy a sample application inside a container. Participants will also learn how to automate, scale, and manage their containerized applications within a Kubernetes cluster. Finally, the training goes on to more advanced topics, walking participants through the process of securing, scaling and monitoring a Kubernetes cluster.
By the end of this training, participants will be able to:
- Set up and run a Docker container.
- Deploy a containerized server and web application.
- Build and manage Docker images.
- Set up a Docker and Kubernetes cluster.
- Use Kubernetes to deploy and manage a clustered web application.
- Secure, scale and monitor a Kubernetes cluster.
Docker and Kubernetes
21 HoursTraining Objectives: Acquire theoretical and practical skills in Docker and Kubernetes.
Docker and Kubernetes advanced
35 HoursThe 'Advanced Docker and Kubernetes' course is a comprehensive training covering the Kubernetes platform and its ecosystem. It targets participants with varying levels of experience, ensuring a full range of both basic and advanced Kubernetes features. It includes key concepts such as Containers, Labels, Controllers, Services, Secrets, Persistent Data Volumes, Namespaces, Quotas, Container Networking Model, Service Discovery, Scaling, Load Balancing, Cluster Management, Kubernetes Installation, Cluster Security, Access Control, Controller High Availability, Monitoring and Logging, Application Auto-scaling, Advanced Scheduling, Microservices-based Applications, Application Design Patterns, and Application Deployment on a Kubernetes Cluster.
Edge AI on Kubernetes: Deploying Intelligent Applications at the Edge
21 HoursEdge AI focuses on executing machine learning inference close to data sources to ensure low-latency, efficient, and scalable processing.
This instructor-led live training (available online or onsite) targets intermediate to advanced practitioners looking to deploy, orchestrate, and optimise AI workloads within Kubernetes-based edge environments.
Upon completing this course, participants will be able to:
- Set up lightweight Kubernetes distributions suitable for edge deployments.
- Effectively deploy AI inference workloads across resource-constrained edge nodes.
- Address connectivity challenges and implement synchronization patterns.
- Optimise performance, storage, and networking for real-world edge scenarios.
Course Format
- Guided presentations supported by real-world examples.
- Scenario-based labs and practical edge deployment exercises.
- Hands-on experience with Kubernetes edge frameworks.
Course Customisation Options
- To request customised training tailored to your specific edge platform requirements, please contact us to arrange.
Deploying Kubernetes Applications with Helm
7 HoursThis instructor-led, live training in Botswana (online or onsite) is aimed at engineers who wish to use Helm to streamline the process of installing and managing Kubernetes applications.
By the end of this training, participants will be able to:
- Install and configure Helm.
- Create reproducible builds of Kubernetes applications.
- Share applications as Helm charts.
- Run third-party applications saved as Helm charts.
- Manage releases of Helm packages.
Introduction to Minikube and Kubernetes
21 HoursThis instructor-led live training in Botswana (online or on-site) is designed for software developers and DevOps professionals at beginner to intermediate levels who wish to learn how to set up and manage a local Kubernetes environment using Minikube.
By the end of this training, participants will be able to:
- Install and configure Minikube on their local machine.
- Grasp the fundamental concepts and architecture of Kubernetes.
- Deploy and manage containers using kubectl and the Minikube dashboard.
- Set up persistent storage and networking solutions for Kubernetes.
- Utilize Minikube for developing, testing, and debugging applications.
Minikube for Developers
14 HoursThis instructor-led, live training in Botswana (online or onsite) is designed for intermediate developers and DevOps engineers who want to utilise Minikube as part of their development workflow.
By the end of this training, participants will be able to:
- Set up and manage a local Kubernetes environment using Minikube.
- Understand how to deploy, manage, and debug applications on Minikube.
- Integrate Minikube into their continuous integration and deployment pipelines.
- Optimize their development process using Minikube's advanced features.
- Apply best practices for local Kubernetes development.