Elliott C. Back: Internet & Technology

I have a unique problem, which is that I have installed my wordpress to a subdirectory, and symlinked httpdocs from several subdomains to that directory. The structure looks like this:

httpdocs/wp/ -> WP Install
subdomains/gadgets/httpdocs/ -> /elliottback.com/httpdocs/wp/
subdomains/books/httpdocs/ -> /elliottback.com/httpdocs/wp/

This means that from my domain, we’re always sticking an extra /wp onto things, but from the subdomains, they go directly into the wp-content directories from the root , in both relative and absolute sense. I consolidated my subdomains this way so that I could run a single WP install and maintain them together. Here’s the .htaccess file that lets WP Super Cache work on either of them:

# BEGIN WPSuperCache
<ifmodule mod_rewrite.c>
RewriteEngine On
AddDefaultCharset UTF-8
RewriteBase /
RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !=POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:Cookie} !^.*(comment_author_|wordpress|wp-postpass_).*$
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{REQUEST_URI} ^(/wp)?/
RewriteCond %{DOCUMENT_ROOT}%1/wp-content/cache/supercache/%{HTTP_HOST}/%1/$1/index.html.gz -f
RewriteRule ^(.*) %1/wp-content/cache/supercache/%{HTTP_HOST}/%1/$1/index.html.gz [L]

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !=POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:Cookie} !^.*(comment_author_|wordpress|wp-postpass_).*$
RewriteCond %{REQUEST_URI} ^(/wp)?/
RewriteCond %{DOCUMENT_ROOT}%1/wp-content/cache/supercache/%{HTTP_HOST}%1/$1/index.html -f
RewriteRule ^(.*) %1/wp-content/cache/supercache/%{HTTP_HOST}%1/$1/index.html [L]
</ifmodule>
# END WPSuperCache

Let me know what you think–performance stats show that it’s working fine for both the /wp subdirectory and the other subdomains!

I just added mod_deflate to my server by using the following configuration:

LoadModule deflate_module modules/mod_deflate.so
<IFModule mod_deflate.c>
	SetOutputFilter DEFLATE
	SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
	SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
	SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
	BrowserMatch ^Mozilla/4 gzip-only-text/html
	BrowserMatch ^Mozilla/4\.0[678] no-gzip
	BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
</IFModule>

The static parts of my pages now render with about 70% compression, meaning that I’m saving 2/3 of static page bandwidth now. Server load seems fine!

Update: You can also check the status of your site’s compression using this free online tool. For my site, it read:

URL: elliottback.com/wp/
Web server type: Apache/2.2.9 (Fedora)
Compression status: Compressed (gzip)
File Size Comparison (in bytes):

Original size: 36949 bytes
Compressed size: 11023 bytes
Savings in bytes: 25926 bytes

Percentage saved by compression: 71.0%
Transfer speed improvement: 3.3 X

An article I wrote about well-designed blogs got more than 1,200 diggs yesterday, causing an influx of some 20,000 to 30,000 new visitors, each pulling down the following page elements:

• 10 large images totalling 969,582 bytes
• CSS and JS totalling 21,655 bytes
• HTML totalling 45,492 bytes

With over a dozen files per request, and 1 MB of transfer, it’s clear that a Digg or Slashdot will overload my poor server. However, there exists a global cache system called Coral Cache which can be used in times of stress to distribute abnormal, spiky load. This will save your onsite HTML and Wordpress system from being hit every time a visitors comes through one of those high traffic sites. Since these visitors will be setting their referrer field, we can identify them, create a special cache redirect URL for them, send them there, and exit. No messy rendering needed.

Yesterday I opted for a solution with mod_rewrite and .htaccess to send traffic to Flickr and Coral Cache instead of my server. It looked like this:

# Digg Effect
RewriteBase /wp/
RewriteRule ^archives/2006/04/16/top-ten-best-designed-blogs[/]? elliottback.com.nyud.net:8080/wp/?p=1351 [R,L]
RewriteRule ^wp-content/uploads/2006/04/10-best-sites-01.* static.flickr.com/56/130439205_51ccaa4a21_o.jpg [R,L]
[...]
RewriteRule ^wp-content/uploads/2006/04/10-best-sites-10.* static.flickr.com/49/130439322_f0ba846651_o.jpg [R,L]
RewriteRule ^wp-content/themes/greenmarinee/index.css elliottback.com.nyud.net:8080/wp/wp-content/themes/greenmarinee/style.css [R,L]

Making a Plugin

Clearly this won’t work for a general solution. Instead, we’ll take advantage of Wordpress’ new cache hook, which is the symlink advanced-cache.php in the wp-content folder. There’s one problem–there could already be something there. So, we’ll have to follow the following flow:

1. If advanced-cache.php doesn’t exist, create it pointing to us
2. If advanced-cache.php exists, create it pointing to us, and include_once() whatever it pointed to when we’re done.

Then we just redirect to a Coral Cache version of our page. This way, when your Wordpress blog gets hit by a DDOS from the popular linksites, it can respond quickly and fluidly without having to load all of your plugins, Wordpress, or even make a single mySQL database call. This will optimize the performance significantly. There’s going to be a problem if something else–say WP-Cache is installed after us, and steals our symlink, but I’ll just have Digg Defender repeat the initialization steps on every activation.

The second part will be a post_content filter that rewrites all external references in the post, such as js or images, to use Coral Cache links–but only for the Coral Cache robots. This way, Coral Cache will cache a copy of the html of a page which has been injected with Coral Cache links everywhere else. Not a single Digg user request will ever touch your website, because Coral Cache will act as a continuous buffer.

Features

• Prevents high-traffic sites from overloading your site
• Integrates Coral Cache with Wordpress
• Runs before WP loads
• Compatible with WP-Cache
• Extremely fast (benchmarks coming soon!)

Download

Version 1.0 of the plugin can be downloaded as a 4KB zip file: digg-defender.zip. It contains four files:

• digg-defender-config.php
• digg-defender-functions.php
• digg-defender-phase1.php
• digg-defender.php

These are contained in a folder called digg-defender.

Installation

Extract the folder digg-defender, copy it to your wordpress/wp-content/plugins/ folder, and activate the plugin in your admin interface. It will spit out numerous textual warnings and errors if permissions are incorrectly set. Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.

Support

If you encounter any bugs, or can suggest a website I add to the hitlist, please leave a comment. Currently the plugin recognizes the following sites: digg.com, slashdot.com, slashdot.org, fark.com, somethingawful.com, kuro5hin.org, engadget.com, boingboing.net, and del.icio.us. I am testing this on my main production blog as we speak, so if there are problems I will probably encounter them first. Update: Please note that Coral Cache is hit/miss and can be slow sometimes to load. That’s not my fault–reload or wait for it to finish loading.

Update

A little update fixes a couple of possible typos / bugs in servers with base_dir restrictions.