Why CDN Purging Isn’t Always “Instant”
April 20, 2015 | Robert GibbInstant purging has always been a key element of CDN as it allows enterprises to remove stale or unwanted content from the edge in a rapid fashion. This feature is important for publishers, as well as other industries we’ll cover below. When talking about purging though, there’s not a one-size-fits-all approach. Some CDNs do it in seconds or minutes (referred to in the industry as “instant”), while others can take hours or longer. The CDNs that do it in seconds or minutes adopt an asynchronous approach and have newer hardware and optimized scripting. Ones that take longer also adopt an asynchronous approach, but may have outdated systems or bloated networks. This post will discuss various purging tactics, why instant purging is important for various industries, and what tactics some of the leading CDNs employ.
Why Instant Purging is ImportantOne industry that relies heavily on instant purging is the publishing industry. Popular news sites like CNN and Fox News - even tech news sites like The Next Web - purge files from the CDN cache frequently. On any given day, there are hundreds of instances of digital media sites releasing stories, then needing to purge them due to story updates, misinformation from sources, and editorial mistakes. Depending on the reach of the website and importance of the piece, each second a false piece remains live could equate to a high-percentage loss of reputation. Consider the following equation for a given minute, assuming that the purge was requested at second 0. A realistic and achievable percentage of reputation degradation ranges anywhere from >0 to 5 percent. CDNs with instant purging can help content owners achieve this. As for CDNs without instant purging like Akamai and Amazon Cloudfront (see chart below), this equation probably can’t even be used for a given minute. The timeframe would have to switch to a given hour, meaning there would be much more reputation degradation regardless of a low or high percentage. Purging isn’t just vital for publishers and decreasing reputation loss though. It’s also vital for other industries and decreasing customer experience loss.
|Industry||Importance of Instant Purging|
|Publishing||As noted above, instant purging prevents publishers from incurring a high percentage of reputation loss due to factual errors.|
|Advertising||Updated, relevant ads on high volume publishing sites drive revenue for advertisers. If old ad campaigns are being served instead of new ones, conversions for the page being promoted will suffer.|
|Gaming||Mobile and online games are updated frequently, therefore the latest version always needs to be available. In addition, uploaded patches with bad coding need purged instantly.|
|Software||Immediately after a new software version is announced, it must be made available for download. Delivering older software versions leave software advocates scratching their heads upon install.|
The Difference Between Purging and InvalidationAs mentioned, purging today is done asynchronously. Years ago, this wasn’t always the case. Purging would be completed synchronously. While purging requests could be submitted at the same, they ultimately had to wait in line. One purging request had to complete before another could start. This “waiting around” purging process that developers and content administrators can’t stand still lives on today, despite asynchronous purging. One just has to look at Akamai’s purging process for example. In Akamai’s own words: "When submitting requests through OPEN [Akamai’s API], customers have access to an emergency queue for their account. Objects submitted to the emergency queue are serviced before the default queue belonging to the same account. To preserve the emergency queue for emergencies, the queue limit is set to 10 objects. For instance, imagine your default queue has 5,000 items to purge and you have a web page that needs to be purged in a hurry. Waiting for the entire default queue to drain will take too long. This would be a good use for the emergency queue. Objects in the emergency queue do not get serviced any faster than the default queue and do not take precedence over requests from other customers.” As you can see, there are a lot of limitations … a lot of queues. Just because asynchronous purging is being used doesn’t mean the purging process is “instant.” A simplified depiction of purge requests. Depending on the CDN, thousands of purge requests can be submitted asynchronously. With that said, today files are cleared from the cache in one of two ways:
- Purging (Asynchronous)
Purging: Secure, But Not Instant with Every ProviderPurging occurs when the CDN looks for the file on all edges and deletes it from the disk. Depending on the CDN you’re using, this can be a slower process, but it’s always more secure. The speed of the purge depends on several factors including hardware type, network size, and the amount of CDN customers requesting purges at the same time. While many next-gen CDNs like Fastly, EdgeCast, and MaxCDN can achieve instant purge times, CDNs with larger networks and outdated hardware take longer. [Compare purging types and other features by downloading this CDN Comparison Sheet] As for security, some news agencies have SLAs in place that require all content be completely removed from all physical disks. This also applies to video for content providers. SLAs state that if a video has expired - such as an HBO episode - it must be removed from all physical media. Invalidation is not an option here. In some cases though, it can be ...
Invalidation: Instantaneous, But Doesn’t Meet SLAsInvalidating content forces the Last-Modified (or Etag) header on the content to change. This change says that "the content is not fresh" and to validate the latest copy on the origin server. If the Last Modified flag has not changed, it will use what is stored locally on the edge or get the latest from the origin. Invalidation can be performed instantaneously - often regardless of the CDN provider you have - and uses much less resources. This is because the CDN only checks the header information and doesn’t require disk cycles for deletion. The downside of invalidation for CDN providers is that content is still locally cached on servers and requires disk space on the Edge. If the content is a large media file, that space could be used for more popular objects. Note: Statements in the previous paragraph are related to the interests of the CDN provider. They mean moot to the customer. Whether your needs require more resources on behalf of the provider shouldn’t concern you.
Purging Methods Used by Different CDNsToday, content delivery networks have purge or revalidation options in the user portal and through the API. Below we’ll evaluate the purging method used by MaxCDN as well as other CDNs.
|MaxCDN||Instant purging. After the purge is requested, the cache is purged within seconds. In this image of our control panel, you can see that files can be purged by zone or file path.|
|Akamai||Purging with Queue. Akamai lets users purge content through its content control utility (CCU) that involves things like queueing and emergency queueing.|
|Fastly||Instant Purging. On their purging page, Fastly claims the following: “When you or your database app updates an object, the old object is automatically purged in less than two seconds from every server in our network.”|
|Cloudfront||Invalidation. According to this Cloudfront doc, you can only invalidate objects or use object versioning to serve a different version of the object.|
|EdgeCast||Instant Purging. Referred to as Pirahna Purge, Edgecast claims it can purge the cache 40 to 80 percent faster than competitors. (Down for a faceoff, EdgeCast?)|