Get in Touch
 Duration 16 hours

Course Outline

Module 1: Overview of Confluent Apache Kafka Architecture and Cluster Setup

  • Understanding Apache Kafka and the Confluent Platform.
  • Core concepts of Kafka brokers, producers, consumers, and topics.
  • The function of Kafka in data streaming and messaging pipelines.
  • Confluent enhancements: REST Proxy, Schema Registry, and Control Center.
  • Kafka cluster structure: brokers, partitions, and replication mechanisms.

Module 2: Configuring Zookeeper Quorums

  • An introduction to Zookeeper.
  • The role of Zookeeper within a Kafka cluster.
  • Determining Zookeeper quorum size.
  • Configuring Zookeeper parameters.
  • Setting up SSH access on our servers.
  • Practical Exercise: Configuring Zookeeper (as a team and as a service).
  • Utilizing the Zookeeper Command Line Interface (CLI).
  • Practical Exercise: Establishing a Zookeeper quorum.
  • Understanding Zookeeper’s internal file system.
  • Performance considerations impacting Zookeeper.
  • Demonstration: Using Zookeeper management tools and Zoonavigator.

Module 3: Setting Up the Kafka Cluster

  • Fundamentals of Kafka.
  • General Kafka configuration.
  • Practical Exercise: Configuring Kafka on a single broker.
  • Practical Exercise: Running Kafka commands.
  • Practical Exercise: Setting up a multi-broker Kafka cluster.
  • Practical Exercise: Testing the Kafka cluster.
  • Verifying connectivity to the Kafka cluster.
  • Configuring Advertised.listeners: a critical setting.
  • Managing topic configurations.
  • Establishing message ingestion and delivery within topics.
  • Practical Exercise: Demonstrating Kafka resilience.
  • Kafka performance: Impact of I/O, network, RAM, CPU, and OS.
  • Kafka performance: Additional influencing factors.
  • Practical Exercise: Modifying broker configurations.

Module 4: Advanced Kafka Configuration

  • Configuring Landoop Kafka Topics UI, Confluent REST Proxy, and Confluent Schema Registry.
  • Message transmission and reception via CLI, Java, and Spring Framework.
  • Monitoring and metrics tools (Confluent Control Center, Elasticsearch, etc.).
  • Managing log files and offsets.
  • High availability and disaster recovery planning.
  • Achieving high availability through replication.
  • Optimizing producer and consumer performance.
  • Defining disaster recovery strategies.
  • Controlling failover and managing data recovery.
  • Setting up connectors.
  • Implementing Kafka Connect.
  • Reviewing Kafka security features.

Summary and Recommendations for Further Action

Requirements

  • Proficiency with Linux-based operating systems and command-line utilities
  • Knowledge of distributed systems principles
  • Comprehension of fundamental networking and system configuration practices

Intended Audience

  • DevOps and SRE teams responsible for managing Kafka deployments
  • System administrators working with Confluent and Zookeeper
  • Kafka developers and architects focused on building scalable systems

Testimonials (2)

Related Categories