Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Introduction to Multimodal LLMs in Vertex AI
- An overview of multimodal capabilities within Vertex AI
- Exploration of Gemini models and their supported modalities
- Applications in enterprise and research settings
Setting Up the Development Environment
- Configuration of Vertex AI for multimodal workflows
- Management of datasets spanning various modalities
- Hands-on lab: Environment setup and dataset preparation
Long Context Windows and Advanced Reasoning
- Comprehending the mechanics of long-context workflows
- Applications in strategic planning and decision-making
- Hands-on lab: Implementation of long-context analysis
Cross-Modal Workflow Design
- Integration of text, audio, and image analysis components
- Sequencing multimodal steps within pipelines
- Hands-on lab: Designing a comprehensive multimodal pipeline
Working with Gemini API Parameters
- Configuration of multimodal inputs and outputs
- Optimization of inference processes and operational efficiency
- Hands-on lab: Tuning Gemini API parameters
Advanced Applications and Integrations
- Development of interactive multimodal agents and assistants
- Integration of external APIs and third-party tools
- Hands-on lab: Construction of a multimodal application
Evaluation and Iteration
- Testing strategies for multimodal performance
- Metrics for assessing accuracy, alignment, and data drift
- Hands-on lab: Evaluation of multimodal workflows
Summary and Next Steps
Requirements
- Strong proficiency in Python programming
- Background in developing machine learning models
- Working knowledge of multimodal data types, including text, audio, and images
Target Audience
- AI researchers
- Advanced developers
- Machine learning scientists
