Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs Training Course
Parameter-Efficient Fine-Tuning (PEFT) represents a suite of methodologies designed to allow large language models (LLMs) to be adapted efficiently by adjusting only a limited subset of their parameters.
This instructor-led, live training session (available online or onsite) is tailored for intermediate-level data scientists and AI engineers seeking to fine-tune LLMs with greater cost-effectiveness and efficiency, utilizing techniques such as LoRA, Adapter Tuning, and Prefix Tuning.
Upon completing this training, participants will be equipped to:
- Grasp the theoretical underpinnings of parameter-efficient fine-tuning methods.
- Apply LoRA, Adapter Tuning, and Prefix Tuning through Hugging Face PEFT.
- Assess the performance and cost implications of PEFT methods relative to full fine-tuning.
- Deploy and scale fine-tuned LLMs while minimizing compute and storage demands.
Course Structure
- Engaging lectures paired with open discussions.
- Extensive exercises and practical application.
- Live-lab environment for hands-on implementation.
Customization Opportunities
- To request a bespoke training experience for this course, please reach out to us to make arrangements.
Course Outline
Foundations of Parameter-Efficient Fine-Tuning (PEFT)
- The rationale for PEFT and the constraints of full fine-tuning
- Overview of PEFT objectives and advantages
- Industrial applications and real-world use cases
LoRA (Low-Rank Adaptation)
- Core concepts and intuitive understanding of LoRA
- Implementing LoRA with Hugging Face and PyTorch
- Practical exercise: Fine-tuning a model using LoRA
Adapter Tuning
- Functionality of adapter modules
- Integration strategies with transformer-based models
- Practical exercise: Applying Adapter Tuning to a transformer model
Prefix Tuning
- Leveraging soft prompts for model adaptation
- Advantages and constraints when compared with LoRA and adapters
- Practical exercise: Executing Prefix Tuning on an LLM task
Assessment and Comparison of PEFT Methods
- Key metrics for evaluating performance and efficiency
- Trade-offs regarding training speed, memory consumption, and accuracy
- Benchmarking strategies and interpreting experimental results
Deploying Fine-Tuned Models
- Processes for saving and loading fine-tuned models
- Considerations for deploying PEFT-based models
- Integration into existing applications and pipelines
Best Practices and Advanced Extensions
- Combining PEFT with quantization and distillation techniques
- Applicability in low-resource and multilingual environments
- Emerging trends and active areas of research
Requirements
- A solid grasp of machine learning fundamentals
- Practical experience with large language models (LLMs)
- Proficiency in Python and PyTorch
Intended Audience
- Data scientists
- AI engineers
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
Parameter-Efficient Fine-Tuning (PEFT) Techniques for LLMs Training Course - Enquiry
Related Courses
Advanced LangGraph: Optimization, Debugging, and Monitoring Complex Graphs
35 HoursBuilding Coding Agents with Devstral: From Agent Design to Tooling
14 HoursGain mastery over Devstral through this live training session in Botswana. The curriculum addresses system architecture, the design of agentic workflows, and seamless tool integration. Perfect for seasoned engineers, the course utilizes hands-on labs to construct coding agents that bolster productivity and security in contemporary development settings.
Open-Source Model Ops: Self-Hosting, Fine-Tuning and Governance with Devstral & Mistral Models
14 HoursThis live training in Botswana is designed for ML engineers aiming to excel in self-hosting and fine-tuning Devstral and Mistral models. Learners will acquire practical skills in governance, monitoring, and compliance to confidently deploy open-source AI solutions in production environments.
LangGraph Applications in Finance
35 HoursLangGraph Foundations: Graph-Based LLM Prompting and Chaining
14 HoursLangGraph in Healthcare: Workflow Orchestration for Regulated Environments
35 HoursLangGraph for Legal Applications
35 HoursBuilding Dynamic Workflows with LangGraph and LLM Agents
14 HoursLangGraph for Marketing Automation
14 HoursLe Chat Enterprise: Private ChatOps, Integrations & Admin Controls
14 HoursThis live training in Botswana focuses on Le Chat Enterprise, a private ChatOps solution designed for secure and governed conversational AI. Participants will learn to deploy, configure, and integrate the platform with enterprise applications, enabling RBAC, SSO, and compliance-oriented controls to support scalable and secure AI workflows.
Cost-Effective LLM Architectures: Mistral at Scale (Performance / Cost Engineering)
14 HoursMaster cost-effective Mistral LLM architectures with this 14-hour live training in Botswana. Learn to deploy, optimize, and scale Mistral Medium 3 for maximum throughput and minimum cost, using batching, quantization, and efficient serving strategies for enterprise workloads.
Productizing Conversational Assistants with Mistral Connectors & Integrations
14 HoursThis live training in Botswana focuses on embedding Mistral conversational assistants into enterprise workflows. Learners will gain proficiency in connecting SaaS tools, applying retrieval-augmented generation, and crafting effective UX patterns for both customer-facing and internal chat interfaces.
Enterprise-Grade Deployments with Mistral Medium 3
14 HoursThis live, instructor-led course in Botswana empowers senior AI engineers to deploy and secure Mistral Medium 3. Participants will learn to optimize inference costs, build multimodal solutions, and enforce enterprise compliance standards, supported by hands-on labs for both API and self-hosted configurations.
Mistral for Responsible AI: Privacy, Data Residency & Enterprise Controls
14 HoursThis live Mistral Responsible AI training in Botswana is customized for compliance and security professionals. It enables you to gain proficiency in privacy-preserving methods, data residency planning, and enterprise controls like RBAC and SSO, ensuring secure and compliant AI operations.
Multimodal Applications with Mistral Models (Vision, OCR, & Document Understanding)
14 HoursDevelop proficiency in creating multimodal applications with Mistral models, emphasizing OCR, document comprehension, and vision-text search. This practical training covers pipeline construction, NLP integration, and enterprise-grade design principles, delivered either online or in Botswana.
