Archived.pdf

Google's search engine requires significant computation per request, with a single query reading hundreds of megabytes of data and consuming tens of billions of CPU cycles. To support thousands of queries per second, Google uses a cluster of over 15,000 commodity-class PCs with fault-tolerant software, which is more cost-effective than a smaller number of high-end servers. The architecture is designed for energy efficiency and price-performance ratio, with easy parallelization of different queries and partitioning of the overall index to use multiple processors. Google's software architecture provides reliability through replication of services across many machines and automatic detection and handling of failures, allowing for a high-end computing cluster at a low-end price. The system consists of multiple clusters distributed worldwide, with a DNS-based load-balancing system and hardware-based load balancers to minimize round-trip time and ensure available capacity. Query execution involves two phases: index servers consulting an inverted index and determining relevant documents, and document servers fetching documents from disk to extract titles and summaries. The system uses replication for capacity and fault-tolerance, with dozens of copies of the Web stored across clusters and multiple server replicas responsible for handling each shard.