Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Performance Fundamentals and Key Metrics
- Analysis of latency, throughput, energy consumption, and resource utilization
- Distinguishing between system-wide and model-specific constraints
- Profiling techniques for inference versus training workloads
Profiling on Huawei Ascend
- Leveraging CANN Profiler and MindInsight tools
- Diagnosing kernels and operators
- Managing offload patterns and memory mapping
Profiling on Biren GPU
- Exploring Biren SDK performance monitoring capabilities
- Addressing kernel fusion, memory alignment, and execution queues
- Profiling with consideration for power and thermal limits
Profiling on Cambricon MLU
- Utilizing BANGPy and Neuware performance suites
- Gaining kernel-level visibility and interpreting logs
- Integrating MLU profilers with deployment frameworks
Graph and Model-Level Optimization
- Strategies for graph pruning and quantization
- Operator fusion and restructuring computational graphs
- Standardizing input sizes and optimizing batch processing
Memory and Kernel Optimization
- Enhancing memory layout and reuse efficiency
- Managing buffers effectively across different chipsets
- Platform-specific kernel tuning methods
Cross-Platform Best Practices
- Achieving performance portability through abstraction strategies
- Creating unified tuning pipelines for multi-chip environments
- Case study: Optimizing an object detection model across Ascend, Biren, and MLU
Conclusion and Recommended Next Steps
Requirements
- Practical experience with AI model training or deployment workflows
- Knowledge of GPU/MLU compute principles and model optimization concepts
- Foundational familiarity with performance profiling instruments and relevant metrics
Target Audience
- Performance engineers
- Machine learning infrastructure teams
- AI system architects
