13 Design a Content Distribution Network (CDN)

 

This chapter covers

  • Discussing the pros, cons, and unexpected situations of a system that you’re asked to design.
  • Satisfying user requests with frontend-metadata-storage architecture.
  • Designing a basic distributed storage system.

A CDN (Content Distribution Network) is a cost-effective and geographically-distributed file storage service that is designed to replicate files across its multiple data centers to serve static content to a large number of geographically-distributed users quickly, serving each user from the data center which can serve her fastest. There are secondary benefits, such as fault-tolerance, allowing users to be served from other data centers if any particular data center is unavailable. Let’s discuss a design for a CDN, which we name CDNService.

13.1 Advantages and disadvantages of a CDN

Before we discuss the requirements and system design for our CDN, we can first discuss the advantages and disadvantages of using a CDN, which may help us understand our requirements.

13.1.1 Advantages of using a CDN

If our company hosts services on multiple data centers, we likely have a shared object store that is replicated across these data centers for redundancy and availability. This shared object store provides many of the benefits of a CDN. We use a CDN if our geographically-distributed userbase can benefit from the extensive network of data centers that a CDN provides. 

13.1.2 Disadvantages of using a CDN

13.1.3 Example of an unexpected issue from using a CDN to serve images

13.2 Requirements

13.3 CDN authentication and authorization

13.3.1 Steps in CDN authentication and authorization

13.3.2 Key rotation

13.4 High-level architecture

13.5 Storage Service

13.5.1 In-cluster

13.5.2 Out-cluster

13.5.3 Evaluation

13.6 Common operations

13.6.1 Reads - Downloads

13.6.2 Writes - Directory creation, file upload, and file deletion

13.7 Cache invalidation

13.8 Logging, monitoring, and alerting

13.9 Other possible discussions on downloading media files

13.10 Summary