An article I wrote about well-designed blogs got more than 1,200 diggs yesterday, causing an influx of some 20,000 to 30,000 new visitors, each pulling down the following page elements:
- 10 large images totalling 969,582 bytes
- CSS and JS totalling 21,655 bytes
- HTML totalling 45,492 bytes
With over a dozen files per request, and 1 MB of transfer, it’s clear that a Digg or Slashdot will overload my poor server. However, there exists a global cache system called Coral Cache which can be used in times of stress to distribute abnormal, spiky load. This will save your onsite HTML and Wordpress system from being hit every time a visitors comes through one of those high traffic sites. Since these visitors will be setting their referrer field, we can identify them, create a special cache redirect URL for them, send them there, and exit. No messy rendering needed.
Yesterday I opted for a solution with mod_rewrite and .htaccess to send traffic to Flickr and Coral Cache instead of my server. It looked like this:
# Digg Effect RewriteBase /wp/ RewriteRule ^archives/2006/04/16/top-ten-best-designed-blogs[/]? elliottback.com.nyud.net:8080/wp/?p=1351 [R,L] RewriteRule ^wp-content/uploads/2006/04/10-best-sites-01.* static.flickr.com/56/130439205_51ccaa4a21_o.jpg [R,L] [...] RewriteRule ^wp-content/uploads/2006/04/10-best-sites-10.* static.flickr.com/49/130439322_f0ba846651_o.jpg [R,L] RewriteRule ^wp-content/themes/greenmarinee/index.css elliottback.com.nyud.net:8080/wp/wp-content/themes/greenmarinee/style.css [R,L]
Making a Plugin
Clearly this won’t work for a general solution. Instead, we’ll take advantage of Wordpress’ new cache hook, which is the symlink advanced-cache.php in the wp-content folder. There’s one problem–there could already be something there. So, we’ll have to follow the following flow:
- If advanced-cache.php doesn’t exist, create it pointing to us
- If advanced-cache.php exists, create it pointing to us, and include_once() whatever it pointed to when we’re done.
Then we just redirect to a Coral Cache version of our page. This way, when your Wordpress blog gets hit by a DDOS from the popular linksites, it can respond quickly and fluidly without having to load all of your plugins, Wordpress, or even make a single mySQL database call. This will optimize the performance significantly. There’s going to be a problem if something else–say WP-Cache is installed after us, and steals our symlink, but I’ll just have Digg Defender repeat the initialization steps on every activation.
The second part will be a post_content filter that rewrites all external references in the post, such as js or images, to use Coral Cache links–but only for the Coral Cache robots. This way, Coral Cache will cache a copy of the html of a page which has been injected with Coral Cache links everywhere else. Not a single Digg user request will ever touch your website, because Coral Cache will act as a continuous buffer.
- Prevents high-traffic sites from overloading your site
- Integrates Coral Cache with Wordpress
- Runs before WP loads
- Compatible with WP-Cache
- Extremely fast (benchmarks coming soon!)
Version 1.0 of the plugin can be downloaded as a 4KB zip file: digg-defender.zip. It contains four files:
These are contained in a folder called digg-defender.
Extract the folder digg-defender, copy it to your wordpress/wp-content/plugins/ folder, and activate the plugin in your admin interface. It will spit out numerous textual warnings and errors if permissions are incorrectly set. Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.
If you encounter any bugs, or can suggest a website I add to the hitlist, please leave a comment. Currently the plugin recognizes the following sites: digg.com, slashdot.com, slashdot.org, fark.com, somethingawful.com, kuro5hin.org, engadget.com, boingboing.net, and del.icio.us. I am testing this on my main production blog as we speak, so if there are problems I will probably encounter them first. Update: Please note that Coral Cache is hit/miss and can be slow sometimes to load. That’s not my fault–reload or wait for it to finish loading.
A little update fixes a couple of possible typos / bugs in servers with base_dir restrictions.
|This entry was posted on Friday, April 21st, 2006 at 6:57 pm and is tagged with referrer field, page elements, content themes, flickr, cache system, content folder, rewriterule, general solution, digg, style css, influx, rewrite, htaccess, js, wp, hook, stress, blogs, traffic, images. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback.|