Elliott C. Back: Technology FTW!

I just added mod_deflate to my server by using the following configuration:

LoadModule deflate_module modules/mod_deflate.so
<IFModule mod_deflate.c>
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
</IFModule>

The static parts of my pages now render with about 70% compression, meaning that I’m saving 2/3 of static page bandwidth now. Server load seems fine!

Update: You can also check the status of your site’s compression using this free online tool. For my site, it read:

URL: elliottback.com/wp/
Web server type: Apache/2.2.9 (Fedora)
Compression status: Compressed (gzip)
File Size Comparison (in bytes):

Original size: 36949 bytes
Compressed size: 11023 bytes
Savings in bytes: 25926 bytes

Percentage saved by compression: 71.0%
Transfer speed improvement: 3.3 X

An article I wrote about well-designed blogs got more than 1,200 diggs yesterday, causing an influx of some 20,000 to 30,000 new visitors, each pulling down the following page elements:

• 10 large images totalling 969,582 bytes
• CSS and JS totalling 21,655 bytes
• HTML totalling 45,492 bytes

With over a dozen files per request, and 1 MB of transfer, it’s clear that a Digg or Slashdot will overload my poor server. However, there exists a global cache system called Coral Cache which can be used in times of stress to distribute abnormal, spiky load. This will save your onsite HTML and Wordpress system from being hit every time a visitors comes through one of those high traffic sites. Since these visitors will be setting their referrer field, we can identify them, create a special cache redirect URL for them, send them there, and exit. No messy rendering needed.

Yesterday I opted for a solution with mod_rewrite and .htaccess to send traffic to Flickr and Coral Cache instead of my server. It looked like this:

# Digg Effect
RewriteBase /wp/

RewriteRule ^archives/2006/04/16/top-ten-best-designed-blogs[/]? elliottback.com.nyud.net:8080/wp/?p=1351 [R,L]

RewriteRule ^wp-content/uploads/2006/04/10-best-sites-01.* static.flickr.com/56/130439205_51ccaa4a21_o.jpg [R,L]

[...]

RewriteRule ^wp-content/uploads/2006/04/10-best-sites-10.* static.flickr.com/49/130439322_f0ba846651_o.jpg [R,L]

RewriteRule ^wp-content/themes/greenmarinee/index.css elliottback.com.nyud.net:8080/wp/wp-content/themes/greenmarinee/style.css [R,L]

Making a Plugin

Clearly this won’t work for a general solution. Instead, we’ll take advantage of Wordpress’ new cache hook, which is the symlink advanced-cache.php in the wp-content folder. There’s one problem–there could already be something there. So, we’ll have to follow the following flow:

1. If advanced-cache.php doesn’t exist, create it pointing to us
2. If advanced-cache.php exists, create it pointing to us, and include_once() whatever it pointed to when we’re done.

Then we just redirect to a Coral Cache version of our page. This way, when your Wordpress blog gets hit by a DDOS from the popular linksites, it can respond quickly and fluidly without having to load all of your plugins, Wordpress, or even make a single mySQL database call. This will optimize the performance significantly. There’s going to be a problem if something else–say WP-Cache is installed after us, and steals our symlink, but I’ll just have Digg Defender repeat the initialization steps on every activation.

The second part will be a post_content filter that rewrites all external references in the post, such as js or images, to use Coral Cache links–but only for the Coral Cache robots. This way, Coral Cache will cache a copy of the html of a page which has been injected with Coral Cache links everywhere else. Not a single Digg user request will ever touch your website, because Coral Cache will act as a continuous buffer.

Features

• Prevents high-traffic sites from overloading your site
• Integrates Coral Cache with Wordpress
• Runs before WP loads
• Compatible with WP-Cache
• Extremely fast (benchmarks coming soon!)

Download

Version 1.0 of the plugin can be downloaded as a 4KB zip file: digg-defender.zip. It contains four files:

• digg-defender-config.php
• digg-defender-functions.php
• digg-defender-phase1.php
• digg-defender.php

These are contained in a folder called digg-defender.

Installation

Extract the folder digg-defender, copy it to your wordpress/wp-content/plugins/ folder, and activate the plugin in your admin interface. It will spit out numerous textual warnings and errors if permissions are incorrectly set. Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.

Support

If you encounter any bugs, or can suggest a website I add to the hitlist, please leave a comment. Currently the plugin recognizes the following sites: digg.com, slashdot.com, slashdot.org, fark.com, somethingawful.com, kuro5hin.org, engadget.com, boingboing.net, and del.icio.us. I am testing this on my main production blog as we speak, so if there are problems I will probably encounter them first. Update: Please note that Coral Cache is hit/miss and can be slow sometimes to load. That’s not my fault–reload or wait for it to finish loading.

Update

A little update fixes a couple of possible typos / bugs in servers with base_dir restrictions.

I got the following cool email today:

I just stumbled on to your blog via someone’s installation of your hashcash plugin. I was wondering what in the world hash cash was and then got distracted by your blog (that I had to read via Google’s cache because your site seems to be unreachable this morning). I just had to email someone who has (almost) the same name as me. And, we also share a similar goal! One of my goals in life is to be known as just “Eliot” without the use of my last name. I love how infrequently I meet other Eliot’s (or Elliot’s or Elliott’s) and I’ve even rarely met someone who knew another Eliot (or Elliot or Elliott). I want to take advantage of that and become like Madonna or Prince or Beck. A one-name wonder, you might say.

Yes, being named Elliott is indeed pretty cool. We’re not named for ET–think T.S. Eliot. And, even cooler, is that any of us can be effectively named E(l+)iot(+) without difficulty. Elllliotttttttttts of the world, unite!

For those of you who don’t know the origin or history of the name Eliot, try this list of people by surname El from wikipedia, or read about how Eliot derives from Hebrew Elijah, meaning “The Lord is my God.”

And, contrary to popular opinion, it’s a statistically significant American name:

Elliot is a common male first name, ranking 582 out of 1219 for males of all ages in the 1990 U.S. Census. Elliot is a very popular surname, ranking 1841 out of 88799 for people of all ages in the 1990 U.S. Census.

Update: Did you know that Elyot is another variant of this name? It’s a rare spelling from the 16^th century!

Something changed at the host (1and1), and all permalinks are not working. I’ve contacted their support… we’ll see how fast this gets resolved. Hopefully they just installed a new apache or something and it’ll kick in in minutes…

Update:

Everything should be back to normal now. Before my .htaccess was giving a lot of 404 errors, and none of the mod_rewrite rules were working. It turns out that 1and1 (without telling me) enabled the apache Multiviews setting, which plays havoc with the rewrite rules:

RewriteRule ^archives/…$ /wp/index.php?… [QSA]
RewriteRule ^archives/$ /wp/archives.php

Since I also had a file called archives.php, Apache couldn’t decide whether to apply the rewrite rules in .htaccess or the multiviews logic that said all /archives/ were really archives.php. The solution? Disabling MultiViews in this directory by adding this line to the .htaccess file:

Options -MultiViews

Due to the resurrection of my GLAT post, I wrote a redirect to send visitors looking for the old one over to the new one. You can just add the following mod_rewrite rules to your .htaccess:

# For the GLAT post
RewriteCond %{QUERY_STRING} postid=106
RewriteRule ^(.*)$ elliottback.com/?blah…) until I realized it was in the QUERY_STRING variable…

Comments