Crawler And Indexing Pipeline

WORKSPACE

55 FAANG Questions Real-world systems Implementations Resources Archive (233 Topics)

CURRICULUM & QUESTIONS

#1 Design a URL Shortening service like TinyURL #2 Design Pastebin #3 Design Instagram #4 Design Dropbox or Google Drive #5 Design Facebook Messenger or WhatsApp #6 Design Twitter for millions of users #7 Design Youtube or Netflix #8 Design Typeahead Suggestion/Autocomplete #9 Design an API Rate Limiter #10 Design a Distributed Metrics Logging and Aggregation System #11 Design a Distributed Stream Processing System like Kafka #12 Design a Key-Value Store #13 Identify the K Most Shared Articles in Various Time Windows (24 hours, 1 hour, 5 minutes)#14 Design Web Crawler #15 Design Facebook's News Feed #16 Design Yelp or Nearby Friends #17 System to Collect Performance Metrics from Thousands of Servers #18 Design Google Calendar #19 Design a Distributed Queue like RabbitMQ #20 Design Google Analytics - User Analytics Dashboard and Pipeline #21 Design a System for Sorting Large Data Sets #22 Top K Elements: App Store Rankings, Amazon Bestsellers, etc.#23 Design a Job Scheduler #24 Design a Notification Service at Scale #25 Surge Pricing System: Uber - Stream Processing, etc.#26 Netflix: Limit the Number of Screens Each User Can Watch #27 Design an ETA Service and Location Sharing Between Driver and Rider #28 Design a Hotel Booking System: Room Availability, Reservation, Booking #29 Design an A/B Testing System (like Optimizely)#30 Design a Price Alert System for Amazon (or for Stock prices)#31 Design an IoC/Dependency Injection Framework #32 Design a Credit Card Processing System #33 Count Facebook Likes, Especially for High-Profile Users #34 Design a Control Plane for a Distributed Database #35 Design a User Login and Authentication System for a Website #36 Develop a Weather Application #37 Create a Document Management System like Wikipedia, Notion or Google Docs #38 Build a Marketplace Feature for Facebook #39 Design a System to Monitor the Health of a Cluster #40 Find a Rider for Uber or Uber Eats #41 Design a Distributed Tracing System #42 Design Backend for an App to Distribute 6 Million Free Burgers in One Hour #43 Design a File Downloader Library from Frontend to Backend #44 Design a System to View Latest Stock Prices Worldwide #45 Develop a Photo Sharing Platform like Flickr or Google Photos #46 Design an On-Call Escalation System #47 Design and Implement a Wire Transfer API #48 Design a Live Comments Feature for Facebook #49 Design a Feature to Show the Number of Users Viewing a Page #50 Design Facebook Likes Feature with Live Updates #51 Create a System to Migrate Large Data to Google Cloud #52 Design a Distributed Botnet #53 Create a Distributed File Transfer System like Bittorrent #54 Design a Parts Compatibility Feature for an eCommerce Site #55 Develop an Ads Management and Display System for a Social Feed

Back to Archive

FREE · TOPIC 104

Crawler And Indexing Pipeline

543 words·Updated 2026-07-18·

#system-design#search#indexing

Primary References & Engineering Sources

·[[wiki/search-resource-guide]]
·[[wiki/search-index-synchronization]]
·[[wiki/information-retrieval-system-design]]
·https://research.google/pubs/the-anatomy-of-a-large-scale-hypertextual-web-search-engine/

← Previous Topic

Counting At Scale

Custom Binary File Format

© 2026 TeamInHome System Design AI Research Pricing Account Terms Privacy