I think The most efficient way is to read the core source code , Let’s take a look at one ?Kafka chart ? as well as ?Kafka Source panorama
Sort out about ?Kafka frame , Find the focus of learning .
front 5 The chapters are :Kafka brief introduction 、Kafka The architecture of 、Broker summary 、Broker Basic module of 、Broker Control management module of
-
** The first 1 Chapter Kafka brief introduction :** Introduce Kafka Birth background 、Kafka stay LinkedIn Internal applications 、Kafka The main design objectives of and why to use message system
-
** The first 2 Chapter Kafka The architecture of :** Introduce Kafka Basic composition of 、 Topology and its internal communication protocol
-
** The first 3 Chapter Broker summary :** describe Kafka The basic elements of a cluster Broker Server Start up and internal module composition
-
** The first 4 Chapter Broker Basic module of :** describe Broker Server Internal nine basic modules : SocketServer 、KafkaRequestHandlerPool 、LogManager、ReplicaManager 、OffsetManager、KafkaScheduler. KafkaApis 、KafkaHalthcheck and TopicConfigManager
-
** The first 5 Chapter Broker Control management module of :** Introduce BrokerServer Control management module of KafkaController, This module is responsible for the whole Kafka Cluster management , for example :Topic New and deleted . Transition between partition state and replica state 、 Cluster load balancing management, etc
after 5 The chapters are Topic Management tools 、 producer 、 consumer 、Kafka Typical application of 、Kafka A comprehensive case of
-
** The first 6 Chapter Topic Management tools :** Introduce three maintenance scripts : kafka-topics.sh 、kafka-reassign-partitions.sh and kafka preferredreplica-election.sh, They relate to Topic Life cycle management of 、Topic Reassignment of partitions and selection of preferred copies of partitions .
-
** The first 7 Chapter producer :** From the design principle 、 Sample code 、 The four parts of module composition and sending mode introduce the relevant knowledge of message producers , From design principles to client Programming , From client programming to internal implementation principle , From shallow to deep , Step by step .
-
** The first 8 Chapter consumer :** Introduce two kinds of consumers : Simple consumers and advanced consumers . For each kind of consumer, we will start from the design principles 、 Consumer processes 、 The four parts of sample code and principle analysis introduce the relevant knowledge of consumers .
-
** The first 9 Chapter Kafka Typical application of :** Introduce Kafka Integration with typical big data systems , Include : Kafka and Storm Integration of 、Kafka and ELK Integration of 、Kafka and Hadoop Integration and Kafka and Spark Integration of .
-
** The first 10 Chapter Kafka A comprehensive case of :** A comprehensive example is used to describe Kafka Application , Case description Kafka As the role of data bus in the overall security solution , Through the storage of vehicle face image data 、 Storage of video data 、 Monitoring of data delay 、 Data quality monitoring 、 Control statistics and disaster recovery backup 6 One business , Briefly describe the internal implementation principle .
-
Kafka What is the design of ?
-
What are the three definitions of data transmission ?
-
Kafka There are two conditions for judging whether a node is still alive ?
-
producer Whether to send data directly to broker Of leader( Master node )?
-
Kafa consumer Whether the specified partition message can be consumed ?
-
Kafka The message is to adopt Pull Pattern , still Push Pattern ?
-
Kafka What is the message format stored on the hard disk ?
-
Kafka Design features of efficient file storage
-
Kafka There are three key differences with traditional messaging systems
-
Kafka establish Topic How to place the partition in different Broker in
-
Kafka Under which directory will the new partition be created
-
partition How to save the data to the hard disk
-
kafka Of ack Mechanism
-
Kafka How consumers consume data
-
Consumer load balancing strategy
-
Data order
-
kafaka Data grouping strategy when producing data
Kafka Advanced knowledge points (12)
44 individual Kafka Knowledge point ( Basics + Advanced + senior ) The analysis is as follows
Golden three silver four will be here soon , I hope you can learn these technical points well
Learning video :
The real interview question of Dachang :
This article has been CODING Open source project :【 A big factory Java Analysis of interview questions + Core summary learning notes + The latest explanation video + Actual project source code 】 Included
– .