Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.) Training Course
Fine-tuning open-source LLMs has become a preferred approach for organisations looking to tailor AI capabilities within secure, cost-effective, and private settings.
This instructor-led, live training (available online or onsite) is designed for intermediate-level ML practitioners and AI developers who want to fine-tune and deploy open-weight models such as LLaMA, Mistral, and Qwen for specific business or internal use cases.
Upon completion of this training, participants will be able to:
- Grasp the ecosystem and distinctions among open-source LLMs.
- Prepare datasets and fine-tuning configurations for models like LLaMA, Mistral, and Qwen.
- Run fine-tuning pipelines using Hugging Face Transformers and PEFT.
- Evaluate, save, and deploy fine-tuned models in secure environments.
Course Format
- Interactive lectures and discussions.
- Ample exercises and practical practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Course Outline
Introduction to Open-Source LLMs
- What are open-weight models and why they're important
- Overview of LLaMA, Mistral, Qwen, and other community models
- Use cases for private, on-premise, or secure deployments
Environment Setup and Tools
- Installing and configuring Transformers, Datasets, and PEFT libraries
- Choosing appropriate hardware for fine-tuning
- Loading pre-trained models from Hugging Face or other repositories
Data Preparation and Preprocessing
- Dataset formats (instruction tuning, chat data, text-only)
- Tokenization and sequence management
- Creating custom datasets and data loaders
Fine-Tuning Techniques
- Standard full fine-tuning vs. parameter-efficient methods
- Applying LoRA and QLoRA for efficient fine-tuning
- Using Trainer API for quick experimentation
Model Evaluation and Optimization
- Assessing fine-tuned models with generation and accuracy metrics
- Managing overfitting, generalization, and validation sets
- Performance tuning tips and logging
Deployment and Private Use
- Saving and loading models for inference
- Deploying fine-tuned models in secure enterprise environments
- On-premise vs. cloud deployment strategies
Case Studies and Use Cases
- Examples of enterprise use of LLaMA, Mistral, and Qwen
- Handling multilingual and domain-specific fine-tuning
- Discussion: Trade-offs between open and closed models
Summary and Next Steps
Requirements
- An understanding of large language models (LLMs) and their architecture
- Experience with Python and PyTorch
- Basic familiarity with the Hugging Face ecosystem
Audience
- ML practitioners
- AI developers
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.) Training Course - Enquiry
Related Courses
Advanced LangGraph: Optimization, Debugging, and Monitoring Complex Graphs
35 HoursThis live, instructor-led training in Botswana is designed for senior AI platform engineers, AI DevOps professionals, and ML architects aiming to optimise, troubleshoot, monitor, and operate production-grade LangGraph systems that manage stateful, multi-actor LLM applications through composable graph structures.
Building Coding Agents with Devstral: From Agent Design to Tooling
14 HoursGain mastery over Devstral through this live training session in Botswana. The curriculum addresses system architecture, the design of agentic workflows, and seamless tool integration. Perfect for seasoned engineers, the course utilizes hands-on labs to construct coding agents that bolster productivity and security in contemporary development settings.
Open-Source Model Ops: Self-Hosting, Fine-Tuning and Governance with Devstral & Mistral Models
14 HoursThis live training in Botswana is designed for ML engineers aiming to excel in self-hosting and fine-tuning Devstral and Mistral models. Learners will acquire practical skills in governance, monitoring, and compliance to confidently deploy open-source AI solutions in production environments.
LangGraph Applications in Finance
35 HoursThis instructor-led session in Botswana empowers professionals to architect finance solutions using LangGraph. Learners will master implementing workflows compliant with regulatory standards, integrating data protocols, and deploying systems characterized by strong governance, observability, and compliance frameworks.
LangGraph Foundations: Graph-Based LLM Prompting and Chaining
14 HoursThis instructor-led LangGraph Foundations course in Botswana prepares developers to create graph-based LLM applications. Participants will master core concepts such as nodes, edges, state management, and branching logic. The course covers integrating tools, retrieval, and external APIs, followed by testing and deploying reliable workflows through hands-on laboratories.
LangGraph in Healthcare: Workflow Orchestration for Regulated Environments
35 HoursDiscover LangGraph's applications in healthcare through this live training held in Botswana, designed for intermediate to advanced professionals. Learn to create stateful, compliant workflows, integrate standards like FHIR and SNOMED CT, and tackle ethical and operational challenges within regulated environments.
LangGraph for Legal Applications
35 HoursGain mastery of LangGraph for legal applications through this 35-hour instructor-led programme in Botswana. Designed for intermediate to advanced professionals, the course delves into workflow design, compliance controls, and human-in-the-loop approval processes. Build auditable, production-ready LLM solutions with assurance and precision.
Building Dynamic Workflows with LangGraph and LLM Agents
14 HoursDesigned for intermediate engineers, this live course (available online or onsite) in Botswana provides a deep dive into constructing dynamic LLM workflows using LangGraph. Participants will master the design of stateful graphs, the implementation of branching logic, and the integration of retrieval systems, APIs, and memory to build resilient, context-aware applications.
LangGraph for Marketing Automation
14 HoursLe Chat Enterprise: Private ChatOps, Integrations & Admin Controls
14 HoursThis live training in Botswana focuses on Le Chat Enterprise, a private ChatOps solution designed for secure and governed conversational AI. Participants will learn to deploy, configure, and integrate the platform with enterprise applications, enabling RBAC, SSO, and compliance-oriented controls to support scalable and secure AI workflows.
Cost-Effective LLM Architectures: Mistral at Scale (Performance / Cost Engineering)
14 HoursMaster cost-effective Mistral LLM architectures with this 14-hour live training in Botswana. Learn to deploy, optimize, and scale Mistral Medium 3 for maximum throughput and minimum cost, using batching, quantization, and efficient serving strategies for enterprise workloads.
Productizing Conversational Assistants with Mistral Connectors & Integrations
14 HoursThis live training in Botswana focuses on embedding Mistral conversational assistants into enterprise workflows. Learners will gain proficiency in connecting SaaS tools, applying retrieval-augmented generation, and crafting effective UX patterns for both customer-facing and internal chat interfaces.
Enterprise-Grade Deployments with Mistral Medium 3
14 HoursThis live, instructor-led course in Botswana empowers senior AI engineers to deploy and secure Mistral Medium 3. Participants will learn to optimize inference costs, build multimodal solutions, and enforce enterprise compliance standards, supported by hands-on labs for both API and self-hosted configurations.
Mistral for Responsible AI: Privacy, Data Residency & Enterprise Controls
14 HoursThis live Mistral Responsible AI training in Botswana is customized for compliance and security professionals. It enables you to gain proficiency in privacy-preserving methods, data residency planning, and enterprise controls like RBAC and SSO, ensuring secure and compliant AI operations.
Multimodal Applications with Mistral Models (Vision, OCR, & Document Understanding)
14 HoursDevelop proficiency in creating multimodal applications with Mistral models, emphasizing OCR, document comprehension, and vision-text search. This practical training covers pipeline construction, NLP integration, and enterprise-grade design principles, delivered either online or in Botswana.
