Elliott C. Back: Internet & Technology

Digg Defender: A Plugin For WordPress

An article I wrote about well-designed blogs got more than 1,200 diggs yesterday, causing an influx of some 20,000 to 30,000 new visitors, each pulling down the following page elements:

  • 10 large images totalling 969,582 bytes
  • CSS and JS totalling 21,655 bytes
  • HTML totalling 45,492 bytes

With over a dozen files per request, and 1 MB of transfer, it’s clear that a Digg or Slashdot will overload my poor server. However, there exists a global cache system called Coral Cache which can be used in times of stress to distribute abnormal, spiky load. This will save your onsite HTML and WordPress system from being hit every time a visitors comes through one of those high traffic sites. Since these visitors will be setting their referrer field, we can identify them, create a special cache redirect URL for them, send them there, and exit. No messy rendering needed.

Yesterday I opted for a solution with mod_rewrite and .htaccess to send traffic to Flickr and Coral Cache instead of my server. It looked like this:

# Digg Effect
RewriteBase /wp/
RewriteRule ^archives/2006/04/16/top-ten-best-designed-blogs[/]? elliottback.com.nyud.net:8080/wp/?p=1351 [R,L]
RewriteRule ^wp-content/uploads/2006/04/10-best-sites-01.* static.flickr.com/56/130439205_51ccaa4a21_o.jpg [R,L]
[...]
RewriteRule ^wp-content/uploads/2006/04/10-best-sites-10.* static.flickr.com/49/130439322_f0ba846651_o.jpg [R,L]
RewriteRule ^wp-content/themes/greenmarinee/index.css elliottback.com.nyud.net:8080/wp/wp-content/themes/greenmarinee/style.css [R,L]

Making a Plugin

Clearly this won’t work for a general solution. Instead, we’ll take advantage of WordPress’ new cache hook, which is the symlink advanced-cache.php in the wp-content folder. There’s one problem–there could already be something there. So, we’ll have to follow the following flow:

  1. If advanced-cache.php doesn’t exist, create it pointing to us
  2. If advanced-cache.php exists, create it pointing to us, and include_once() whatever it pointed to when we’re done.

Then we just redirect to a Coral Cache version of our page. This way, when your WordPress blog gets hit by a DDOS from the popular linksites, it can respond quickly and fluidly without having to load all of your plugins, WordPress, or even make a single mySQL database call. This will optimize the performance significantly. There’s going to be a problem if something else–say WP-Cache is installed after us, and steals our symlink, but I’ll just have Digg Defender repeat the initialization steps on every activation.

The second part will be a post_content filter that rewrites all external references in the post, such as js or images, to use Coral Cache links–but only for the Coral Cache robots. This way, Coral Cache will cache a copy of the html of a page which has been injected with Coral Cache links everywhere else. Not a single Digg user request will ever touch your website, because Coral Cache will act as a continuous buffer.

Features

  • Prevents high-traffic sites from overloading your site
  • Integrates Coral Cache with WordPress
  • Runs before WP loads
  • Compatible with WP-Cache
  • Extremely fast (benchmarks coming soon!)

Download

Version 1.0 of the plugin can be downloaded as a 4KB zip file: digg-defender.zip. It contains four files:

  • digg-defender-config.php
  • digg-defender-functions.php
  • digg-defender-phase1.php
  • digg-defender.php

These are contained in a folder called digg-defender.

Installation

Extract the folder digg-defender, copy it to your wordpress/wp-content/plugins/ folder, and activate the plugin in your admin interface. It will spit out numerous textual warnings and errors if permissions are incorrectly set. Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.

Support

If you encounter any bugs, or can suggest a website I add to the hitlist, please leave a comment. Currently the plugin recognizes the following sites: digg.com, slashdot.com, slashdot.org, fark.com, somethingawful.com, kuro5hin.org, engadget.com, boingboing.net, and del.icio.us. I am testing this on my main production blog as we speak, so if there are problems I will probably encounter them first. Update: Please note that Coral Cache is hit/miss and can be slow sometimes to load. That’s not my fault–reload or wait for it to finish loading.

Update

A little update fixes a couple of possible typos / bugs in servers with base_dir restrictions.

This entry was posted on Friday, April 21st, 2006 at 6:57 pm and is tagged with . You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback.

72 Responses to “Digg Defender: A Plugin For WordPress”

  1. [...] Digg Defender- This is a WordPress plugin that will save your site from going down under massive traffic by redirecting it through the Coral Cache network. [...]

  2. [...] Digg Defender This is a WordPress plugin that will save your site from going down under massive traffic by [...]

  3. [...] Digg Defender This is a WordPress plugin that will save your site from going down under massive traffic by redirecting it through the Coral Cache network. [...]

  4. [...] Digg Defender This is a WordPress plugin that will save your site from going down under massive traffic by redirecting it through the Coral Cache network. [...]

  5. [...] Digg Defender This is a WordPress plugin that will save your site from going down under massive traffic by [...]

  6. YouYap.com says:

    i like it too. nice plugin. keep up the good work

  7. Video Reup says:

    Quality plugin, thanks a lot Mr. Back!

  8. [...] Digg Defender: Find out how to keep your blog running quickly and smoothly even if your posts become top-rated Digg stories. [...]

  9. [...] Digg Defender: Find out how to keep your blog running quickly and smoothly even if your posts become top-rated Digg stories. [...]

  10. [...] Er zijn nog wat dingen die je liever niet gebruikt, maar die je wel kunnen redden, zodra je echt krap zit met je dataverkeer. Kijk bijvoorbeeld eens naar de plugin, Digg Defender. Deze plugin zorgt ervoor dat alle bezoekers die via Digg jouw site benaderen, worden doorverwezen naar een cache-pagina van Coral Cache, waar jouw inhoud op te zien is. Dan zijn er ook nog zaken, die niet iedereen kan gebruiken. Ik praat dan over bijvoorbeeld eaccelerator, een ‘versneller’ voor PHP. Omdat niet iedereen technische dingen kan aanpassen op de server waar zijn/haar weblog op draait, is dit vaak niet haalbaar. Sommige hosters staan wel open voor dit soort ideeën, dus het kan nooit kwaad om er eens contact mee op te nemen. Deel dit artikelVind u dit een leuk artikel? Deel het dan met anderen via een van deze sites: [...]

  11. [...] Digg Defender – Caching plugin that directs traffic coming from Digg to Coral Cache to try and prevent a server crash from the Digg effect. [...]

  12. [...] Consider using the Digg Defender plugin I didn’t get a chance to fully test this one, but it may just save your site in the event of a traffic spike such as Digg. It works with the Coral CDN service to send Social media traffic to a mirror site. [...]

  13. [...] Information/Background The idea behind this script came from Digg defender by Elliot Back. Sadly, Elliot’s script is only a WordPress plug in, and well, Word Press blogs aren’t the only websites that can experience high traffic. Although I wrote this script myself, the credit for the idea does reside with Elliot. Note: At the time of writing this script, I was using a custom CMS for the website however as of recently I have switched to WordPress. [...]

  14. technabob says:

    Does anyone know if there’s a way to automatically redirect to the post itself if someone tries to link directly to an image?

    I know that might be a tall order, but people keep submitting images from my site to Digg and Stumbleupon, rather than the blog posts, so then I lose out on all the traffic, subscribers and ad revenues that might come from people actually coming to the site.

  15. [...] Digg Defender [...]

  16. [...] Digg Defender – Caching plugin that directs traffic coming from Digg to Coral Cache to try and prevent a server crash from the Digg effect. [...]

  17. Vic says:

    This is a grea plugin. This type of is priceless for the new kid on the block.

    Thanks

    Vic

  18. Lincoln says:

    What a brilliant plugin. I have it installed and the options indicate the necessary files are writeable, but is there anyway to give this a test run to see if it’ll actually work? I submitted my site to Reddit, then travelled back to my site via the reddit link, but I didn’t notice a difference. I was expecting to see the suffix address nyud.net:8080 added to my address but it didn’t show. Thanks for contributing such a nifty plugin though. :-D

  19. Rob says:

    I think there is a small bug in digg-defender.php digg_defender_check_global_config.

    I think the check for “!is_writable($global) || !digg_defender_replace_line(…))” should instead read something like:

    if (!WP_CACHE && !is_writeable($global) && !digg_defender_replace_line(…))

    To put it another way, if WP_CACHE is enabled, OR if the file is writeable, OR if modifying the file succeeded, then the config is good (return true).

  20. Elliott Back says:

    You probably included some random whitespace somewhere after one of the files php tags.

  21. I just installed digg defender on my website www.mypersonalfinancejourney.com and activated it and now my entire site is blank including the admin pages. Help! what happened?

  22. [...] Next I’ve got Digg Defender set up. This plugin notes when hits are coming from various sites (notably in this case Digg) and redirects the incoming request from ArsGeek to a mirror site. People can still read and reach the article, but aren’t bringing ArsGeek down with more traffic. [...]

  23. A global cache system? I’ve never heard of that…how does it work?

  24. [...] Download Digg Defender here Installation instructions here. [...]

  25. [...] Alternatively, you can use Digg Defender. Digg Defender uses the WP-Cache system hook to redirect requests from high-traffic sites like Digg and Slashdot to a free global cache system. It’s compatible with WP-Cache, so no worries. I am contemplating whether or not to use it. I guess an extra line of defense wouldn’t hurt. You can download Digg Defender here, and the installation instructions are here. [...]

  26. [...] The deal’s not done, but my earlier allusions to D.C. mythology (and alternate Earths) were project-specific. I hope to be able to say what it is soon. And yes, it’s cruel of me to leave it floating out there. It would have been a lot worse if I hadn’t installed a plug-in which automatically cached the page on Coral. ↩ February 22, 2007 @ 10:35 am | Share This | Filed under: Projects, News, Los Angeles, The Movie [...]

  27. [...] However, where stumbleupon shines for the site owner, digg is painful. A front-page story on digg will get you a bazillion hits in the space of an hour, at which point your servers will catch fire. You enjoy the exposure, but your users don’t because they can’t see your site. [...]

  28. [...] Digg This Reloaded Authors Description: Adds Digg story link/button on detection of Digg referrer, otherwise a submit to Digg link is displayed. My overview: A massive improvement on the Digg This Plugin that I’ve mention previously and personally had issues with. I’ve seen this working fine elsewhere but won’t work properly (at the moment) with Digg Defender. I’ve been in touch with the author and they’re hopefully going to work on a merged plugin tomorrow to resolve this issue. You can see the little Digg button at the bottom of each post. [...]

  29. Rob says:

    Don’t spose you know if this will be WP2.1 compatible do you?

    Cheers,

  30. oh man i hope this one works couse i can’t find anything that works for me or maybe i’m just plain stupid….pffffff

  31. [...] Digg Defender: A Plugin For WordPress by Elliott Back Hvis man bliver digget kan man forsvare sig med et plugin (tags: wordpress plugin) [...]

  32. Oops, I may be an idiot; thanks.

  33. Is the criteria for using coral based on HTTP referer? Is it possible to use this to redirect to coral if a database connection fails

  34. [...] Due to an earlier request, I set about combining the Digg Defender Plugin and my own Digg This Reloaded WordPress plugin. [...]

  35. [...] Digg This Reloaded Authors Description: Adds Digg story link/button on detection of Digg referrer, otherwise a submit to Digg link is displayed. My overview: A massive improvement on the Digg This Plugin that I’ve mention previously and personally had issues with. I’ve seen this working fine elsewhere but won’t work properly (at the moment) with Digg Defender. I’ve been in touch with the author and they’re hopefully going to work on a merged plugin tomorrow to resolve this issue. You can see the little Digg button at the bottom of each post. [...]

  36. [...] You ‘ll need two plugins for this: wp-cache(direct download) and Digg Defender(direct download) [...]

  37. Rob says:

    Great idea and it seemed to work for the first item I submitted to digg but now I get this error when the plugin is enabled:

    Warning: Cannot modify header information – headers already sent by (output started at /blog/index.php:1) in //blog/wp-content/plugins/digg-defender/digg-defender-phase1.php on line 14

    Can anyone tell me why or offer a solution please?

    Many thanks in advance.

  38. Zach Fine says:

    And now the zip file is working fine. Wonder why it didn’t work before…

  39. Stephen says:

    I am wondering how I can permanently coralize all my image content. I syndicate a lot of my content from other sites and have to manually make sure I am not hotlinking images from them. If I could re-write all external images to coral that would be really cool. Any ideas?

  40. Thilak says:

    Hi Elliot, I moved my WordPress Blog. And I tried to enable your plugin but it refused to work, Please help me out !!

  41. [...] So far, I’ve installed the Digg Defender and WP-Cache plugins for WordPress. Digg Defender sees traffic from digg.com and redirects it to an external cached version of the page on a free service called Coral Cache. Ricardo Galli, author of WP-Cache, describes his plugin as: An extremely efficient WordPress page caching system to make your site much faster and responsive. It works by caching Worpress pages and storing them in a static file for serving future requests directly from the file rather than loading and compiling the whole PHP code and then building the page from the database. [...]

  42. [...] Digg Defender: A Plugin For WordPress by Elliott Back A plugin to prepare your site for what was formerly known as the /. effect. (tags: wordpress plugin 404) [...]

  43. [...] Digg Defender: A Plugin For WordPress by Elliott Back (tags: wordpress plugin cache) [...]

  44. tim says:

    Dude, you saved my *ss!! we just got boingboing’d and i was watching server load go through the roof! At 8pm PST!

    We’ve got an online comic here: smithmag.us/shootingwar and I’d already offloaded all images to photobucket. Alas, I was still getting hammered.

    Since people are reading all our comic chapters and not just linking to one post (the boingboing link only goes to our homepage), I actually tweaked your function to redirect everything. :) I’ll change it back once things calm down.

    thanks!

  45. Jim Munroe says:

    Hey Elliott–

    Great job on the plugin! I had a post on Digg front page linking to my server last night, so lucky I installed DD earlier that day! It definitely did the trick most of the time, but occasionally I was getting the wordpress static “cannot connect to the database” being served up by coralcdn for some reason, and at times when a direct link was working fine. Any ideas? Is it just caching it at a moment when it’s overloaded, and if so is it possible to set an expiry time on it or induce recaching?

    Also, it seemed to work better when I deactivated WP-Cache. When you say “Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.” do you mean the stylesheets and graphics are not coralcdn’ed and are being served by the original server?

    Sorry for all the questions, and thanks again for a great and useful plugin.

  46. Alan says:

    Digg Defender works well however it doesn’t play nice with WP Cache for me.
    I Use:
    WordPress 1.5.2
    WP Cache version 2.0.17

    By itself WP Cache works good but as soon as Digg Defender is activated WP Cache no longer works. Is there a setting somewhere that I need to change?

  47. [...] WordPress is a firm believer of being completely dynamic. All the content is stored in the database, and a page is created on the fly before it is served. A high-traffice website can cause high load on the database and can slow down things as WordPress hasto depend on the database for every serving of the page. However, there are certain plugins available that can cache the system and offer better performance. [...]

  48. [...] If you suspect the possibility that your website might get a huge influx of traffic someday, there are ways to protect against that. Well, if you are running a weblog using the WordPress blogging platform, you can use Elliot Back’s Digg Defender plugin (found via Stupid WordPress Tricks). [...]

  49. [...] The plugin, digg defender, by Elliott Back, redirects traffic to Coral Cache, thus relieving traffic from your site and possibly preventing your provider from shutting your site down.  Elliott also explains the new  WordPress cache hook, advanced-cache.php, and modifying .htaccess to redirect URLs to the Coral Cache.  [...]

  50. nm, anyone that wants to add a site; see digg-defender-functions.php and add the domain to the array.

  51. What about OSNews.com? My site bit the dust yesterday from there….

  52. [...] WP Plugin: Digg Defender pre-WP-cache caching plugin that sends you over to Coral Cache if you’re coming from /., digg, etc. It will give an “order-of” performance increase, according to my tests, even on shared hosting. Thanks Elliott Technorati Tags: digg defender wordpress plugin   [...]

  53. [...] Digg effect wins again! Poor poor host. If I ever get enough traffic to warrant it, I might eventually get a domain with a GoDaddy/CacheFly combo. For now, I’ve just installed the Digg Defender plugin in the hopes of preventing this type of situation from happening again. [...]

  54. Digg Defender…

    I don’t anticipate any of my rantings making it onto Digg or Slashdot, but just in case I’ve installed the Digg Defender plugin, which redirects any visitors from such high-traffic sites to the Coral Cache system to reduce the chance of the…

  55. Keith Curtis says:

    It would be cool if you could build something which would automatically recognize when you are getting overloaded and have it transparently switch over. There are a surprising number of websites which can take down a server, new ones growing every day (reddit.com) and only while the load is bad do I need the cache’s help–once you get off the digg.com main page, everything should be fine again.

    This is a great plugin and it would be cool to build it into wordpress!

  56. raju says:

    You can see how people have fixed these issues on other Digg Tools

  57. I arrived via populicio.us to
    elliottback.com.nyud.net:8080/
    instead of
    elliottback.com/
    however the page wouldn’t finish loading — and I had to view source to see what all the fuss was about. I looks very interesting, and I can obviously get to the site now, but that initial hic-up is a bit concerning… maybe you were fine tuning your .htaccess settings tonight?

  58. Ahmer says:

    I’m using php 5.1.2

    What version do you use?

    The only other version I can use is 4.4.2

    My webhost is dreamhost. They run php as CGI.

  59. Ahmer says:

    There’s a problem. I’m not sure what the cause of it is but it keeps saying

    return $str; } ?> ?> ction(‘admin_footer’, ‘digg_defender_check_link’); add_action(‘admin_footer’, ‘digg_defender_check_global_config’); add_action(‘deactivate_’ . substr(dirname(__FILE__), strrpos(dirname(__FILE__), ‘/’) + 1) . ‘/’ . basename(__FILE__), ‘digg_defender_restore_old_link’); ?>

    at the top of my blog. All of the pages have this at the top, even in the panel.

    BTW, I can host a forum for the plugin if you want.

  60. Otto says:

    Meh. What the heck, I figured I’d fix up the above mod_rewrite code.

    Look for the easier solution here: ottodestruct.com/diggprotectionrules.txt

  61. Its all done… So how do I know that it is working?

  62. Otto says:

    Ugh. That last RewriteRule *should* contain an http :// in front of it. Won’t work otherwise.

  63. Elliott Back says:

    The files /wp/wp-config.php, /wp/wp-content/plugins/digg-defender/digg-defender-config.php, and /wp/wp-content/advanced-cache symlink all need to be writeable. This means the directories need to be writeable as well.

  64. Otto says:

    It seems like this is a bit of overkill. A slightly simpler way would be to just make your Rewrite’s more generalized.

    RewriteEngine on
    RewriteCond %{HTTP_REFERER} ^http://(www\.)?digg\.com
    RewriteCond %{REMOTE_ADDR} !^aaa.bbb.ccc.ddd$
    RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
    RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
    RewriteRule ^(.*)$ www.yourdomain.com.nyud.net:8080/$1 [R,L]

    Replace aaa.bbb.ccc.ddd with your own IP (so you always get to your site).
    Replace www.yourdomain.com with your actual domain.

    Voila. Anybody coming from a digg link gets redirected to the Coral Cache. You can duplicate this whole block and thus add slashdot.org or other possible high traffic links as well, or you could modify the first condition alone, if you liked, instead of duplicating the block as a whole.

    Just don’t mess with the user_agent and query_string lines. Those ensure that the Coral servers themselves can retrieve your page when they need to.

    This is not a perfect solution, since if somebody doesn’t send referrer info, they don’t get redirected, but the number of people not sending referrers is few enough for this to be viable and prevent most servers from dying on a slashdotting.

  65. I think there is something wrong with the plugin. I have wp-cache installed and the line is in wp-config.php and i get this message : Error: WP_CACHE is not enabled in your wp-config.php file and I couldn’t modified it.
    Edit /home/httpd/vhosts/technopedia.info/httpdocs/tech/wp-config.php and add the following line:
    define(‘WP_CACHE’, true);
    Otherwise, WP-Cache will not be executed by WordPress core.

    techopedia.info/tech is my website.

  66. zach fine says:

    I’d like to use your digg-defender plugin, but something is wrong with the zip file. I can’t uncompress it on my Mac or on my NetBSD box.

Leave a Reply