#11Very Hard
Design a Distributed Stream Processing System like Kafka
Asked by:AmazonMicrosoftWiseConfluent
Recommended Solutions from the Web
Read the original Kafka paper, but don't feel pressured to complete it
https://notes.stephenholiday.com/Kafka.pdf
Great overview
https://medium.com/better-programming/system-design-series-apache-kafka-from-10-000-feet-9c95af56f18d
Solution from Meta Engineer
https://www.youtube.com/watch?v=DU8o-OTeoCc
Good visualization
https://www.youtube.com/watch?v=HZklgPkboro
Key Details & Edge Cases You Should Know
•
Go over how Kafka might be designed.
•
What is the difference between Kafka and RabbitMQ?
•
How does Kafka ensure fast writes to the stream? Why is it fast?
•
How to scale it so that it can handle Facebook-level stream of data?
•
How to do the following: Fault Tolerance, Partitioning, Balancing Workloads across Nodes