Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Introduction to LLMOps
- Comparing LLMOps with MLOps: addressing the unique challenges of operating LLMs.
- The LLM application lifecycle: covering prompt creation, evaluation, deployment, and monitoring.
- Production readiness checklist for Generative AI applications.
Prompt Management and Versioning
- Systems for prompt templating and variable injection.
- Semantic versioning for prompts combined with automated regression testing.
- Prompt registries and collaborative workflows.
LLM Evaluation at Scale
- Evaluation dimensions including accuracy, relevance, safety, and groundedness.
- Utilizing LLM-as-judge metrics and human evaluation pipelines.
- Automated evaluation frameworks such as RAGAS, DeepEval, and custom evaluators.
- Implementing quality gates within CI/CD pipelines for LLM deployments.
Safety Guardrails and Content Governance
- Input and output guardrails using tools like NeMo Guardrails and Guardrails AI.
- Strategies for PII detection, toxicity filtering, and defining topic boundaries.
- Defence strategies against jailbreaks and prompt injection attacks.
- Conducting red-teaming exercises on LLM applications to ensure safety assurance.
LLM Observability and Monitoring
- Tracking telemetry data including token usage, latency, cost, and quality metrics.
- Detecting drift in LLM outputs and embedding spaces.
- Implementing session-level tracing for multi-turn agent conversations.
- Utilizing dashboards and alerting systems with LangSmith, Arize, and OpenTelemetry.
AI Gateway and Model Orchestration
- Multi-provider routing using LiteLLM and Portkey.
- Developing fallback strategies, retry logic, and circuit breaker mechanisms.
- Cost-aware model selection and load balancing techniques.
- Managing rate limits, quotas, and API key governance.
Performance Optimization
- Semantic caching utilizing vector stores and exact-match strategies.
- Enforcing structured outputs through constrained decoding.
- Implementing batching, streaming, and concurrency patterns.
- Optimizing latency across various model providers.
Governance, Compliance, and Audit
- Maintaining LLM audit trails comprising prompt logs, response logs, and decision provenance.
- Addressing data residency and privacy considerations for LLM APIs.
- Implementing policy-as-code for regulating LLM usage within organizations.
- Developing an internal playbook for LLM operations.
Requirements
- Experience in building or integrating applications powered by Large Language Models.
- Familiarity with Python and REST APIs.
- Basic understanding of prompt engineering concepts.
Audience
- ML engineers and MLOps practitioners transitioning into LLM operations.
- Platform engineers responsible for LLM infrastructure.
- Technical leads managing production Generative AI deployments.
14 Hours
Testimonials (2)
use of proper and effective prompt
Marses Pacaldo
Course - Generative AI and Prompt Engineering for Corporate Professionals
The interactive style, the exercises
