Digg Defender: A Plugin For Wordpress
An article I wrote about well-designed blogs got more than 1,200 diggs yesterday, causing an influx of some 20,000 to 30,000 new visitors, each pulling down the following page elements:
- 10 large images totalling 969,582 bytes
- CSS and JS totalling 21,655 bytes
- HTML totalling 45,492 bytes
With over a dozen files per request, and 1 MB of transfer, it’s clear that a Digg or Slashdot will overload my poor server. However, there exists a global cache system called Coral Cache which can be used in times of stress to distribute abnormal, spiky load. This will save your onsite HTML and Wordpress system from being hit every time a visitors comes through one of those high traffic sites. Since these visitors will be setting their referrer field, we can identify them, create a special cache redirect URL for them, send them there, and exit. No messy rendering needed.
Yesterday I opted for a solution with mod_rewrite and .htaccess to send traffic to Flickr and Coral Cache instead of my server. It looked like this:
# Digg Effect
RewriteBase /wp/RewriteRule ^archives/2006/04/16/top-ten-best-designed-blogs[/]? elliottback.com.nyud.net:8080/wp/?p=1351 [R,L]
RewriteRule ^wp-content/uploads/2006/04/10-best-sites-01.* static.flickr.com/56/130439205_51ccaa4a21_o.jpg [R,L]
[…]
RewriteRule ^wp-content/uploads/2006/04/10-best-sites-10.* static.flickr.com/49/130439322_f0ba846651_o.jpg [R,L]
RewriteRule ^wp-content/themes/greenmarinee/index.css elliottback.com.nyud.net:8080/wp/wp-content/themes/greenmarinee/style.css [R,L]
Making a Plugin
Clearly this won’t work for a general solution. Instead, we’ll take advantage of Wordpress’ new cache hook, which is the symlink advanced-cache.php in the wp-content folder. There’s one problem–there could already be something there. So, we’ll have to follow the following flow:
- If advanced-cache.php doesn’t exist, create it pointing to us
- If advanced-cache.php exists, create it pointing to us, and include_once() whatever it pointed to when we’re done.
Then we just redirect to a Coral Cache version of our page. This way, when your Wordpress blog gets hit by a DDOS from the popular linksites, it can respond quickly and fluidly without having to load all of your plugins, Wordpress, or even make a single mySQL database call. This will optimize the performance significantly. There’s going to be a problem if something else–say WP-Cache is installed after us, and steals our symlink, but I’ll just have Digg Defender repeat the initialization steps on every activation.
The second part will be a post_content filter that rewrites all external references in the post, such as js or images, to use Coral Cache links–but only for the Coral Cache robots. This way, Coral Cache will cache a copy of the html of a page which has been injected with Coral Cache links everywhere else. Not a single Digg user request will ever touch your website, because Coral Cache will act as a continuous buffer.
Features
- Prevents high-traffic sites from overloading your site
- Integrates Coral Cache with Wordpress
- Runs before WP loads
- Compatible with WP-Cache
- Extremely fast (benchmarks coming soon!)
Download
Version 1.0 of the plugin can be downloaded as a 4KB zip file: digg-defender.zip. It contains four files:
- digg-defender-config.php
- digg-defender-functions.php
- digg-defender-phase1.php
- digg-defender.php
These are contained in a folder called digg-defender.
Installation
Extract the folder digg-defender, copy it to your wordpress/wp-content/plugins/ folder, and activate the plugin in your admin interface. It will spit out numerous textual warnings and errors if permissions are incorrectly set. Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.
Support
If you encounter any bugs, or can suggest a website I add to the hitlist, please leave a comment. Currently the plugin recognizes the following sites: digg.com, slashdot.com, slashdot.org, fark.com, somethingawful.com, kuro5hin.org, engadget.com, boingboing.net, and del.icio.us. I am testing this on my main production blog as we speak, so if there are problems I will probably encounter them first. Update: Please note that Coral Cache is hit/miss and can be slow sometimes to load. That’s not my fault–reload or wait for it to finish loading.
Update
A little update fixes a couple of possible typos / bugs in servers with base_dir restrictions.
This entry was posted on Friday, April 21st, 2006 at 6:57 pm and is tagged with referrer field, page elements, content themes, flickr, cache system, content folder, rewriterule, general solution, digg, style css, influx, rewrite, htaccess, js, wp, hook, stress, blogs, traffic, images. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback.
on April 22nd, 2006 at 2:59 am
I’d like to use your digg-defender plugin, but something is wrong with the zip file. I can’t uncompress it on my Mac or on my NetBSD box.
on April 22nd, 2006 at 7:46 am
I think there is something wrong with the plugin. I have wp-cache installed and the line is in wp-config.php and i get this message : Error: WP_CACHE is not enabled in your wp-config.php file and I couldn’t modified it.
Edit /home/httpd/vhosts/technopedia.info/httpdocs/tech/wp-config.php and add the following line:
define(’WP_CACHE’, true);
Otherwise, WP-Cache will not be executed by Wordpress core.
techopedia.info/tech is my website.
on April 22nd, 2006 at 12:54 pm
It seems like this is a bit of overkill. A slightly simpler way would be to just make your Rewrite’s more generalized.
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://(www\.)?digg\.com
RewriteCond %{REMOTE_ADDR} !^aaa.bbb.ccc.ddd$
RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteRule ^(.*)$ www.yourdomain.com.nyud.net:8080/$1 [R,L]
Replace aaa.bbb.ccc.ddd with your own IP (so you always get to your site).
Replace www.yourdomain.com with your actual domain.
Voila. Anybody coming from a digg link gets redirected to the Coral Cache. You can duplicate this whole block and thus add slashdot.org or other possible high traffic links as well, or you could modify the first condition alone, if you liked, instead of duplicating the block as a whole.
Just don’t mess with the user_agent and query_string lines. Those ensure that the Coral servers themselves can retrieve your page when they need to.
This is not a perfect solution, since if somebody doesn’t send referrer info, they don’t get redirected, but the number of people not sending referrers is few enough for this to be viable and prevent most servers from dying on a slashdotting.
on April 22nd, 2006 at 12:55 pm
The files /wp/wp-config.php, /wp/wp-content/plugins/digg-defender/digg-defender-config.php, and /wp/wp-content/advanced-cache symlink all need to be writeable. This means the directories need to be writeable as well.
on April 22nd, 2006 at 12:55 pm
Ugh. That last RewriteRule *should* contain an http :// in front of it. Won’t work otherwise.
on April 22nd, 2006 at 1:05 pm
Its all done… So how do I know that it is working?
on April 22nd, 2006 at 1:25 pm
Meh. What the heck, I figured I’d fix up the above mod_rewrite code.
Look for the easier solution here: ottodestruct.com/diggprotectionrules.txt
on April 22nd, 2006 at 4:06 pm
There’s a problem. I’m not sure what the cause of it is but it keeps saying
return $str; } ?> ?> ction(’admin_footer’, ‘digg_defender_check_link’); add_action(’admin_footer’, ‘digg_defender_check_global_config’); add_action(’deactivate_’ . substr(dirname(__FILE__), strrpos(dirname(__FILE__), ‘/’) + 1) . ‘/’ . basename(__FILE__), ‘digg_defender_restore_old_link’); ?>
at the top of my blog. All of the pages have this at the top, even in the panel.
BTW, I can host a forum for the plugin if you want.
on April 22nd, 2006 at 4:36 pm
I’m using php 5.1.2
What version do you use?
The only other version I can use is 4.4.2
My webhost is dreamhost. They run php as CGI.
on April 22nd, 2006 at 6:23 pm
I arrived via populicio.us to
elliottback.com.nyud.net:8080/…
instead of
elliottback.com/…
however the page wouldn’t finish loading — and I had to view source to see what all the fuss was about. I looks very interesting, and I can obviously get to the site now, but that initial hic-up is a bit concerning… maybe you were fine tuning your .htaccess settings tonight?
on April 22nd, 2006 at 9:31 pm
You can see how people have fixed these issues on other Digg Tools
on April 23rd, 2006 at 12:01 am
[…] Digg Defender […]
on April 23rd, 2006 at 12:22 am
It would be cool if you could build something which would automatically recognize when you are getting overloaded and have it transparently switch over. There are a surprising number of websites which can take down a server, new ones growing every day (reddit.com) and only while the load is bad do I need the cache’s help–once you get off the digg.com main page, everything should be fine again.
This is a great plugin and it would be cool to build it into wordpress!
on April 23rd, 2006 at 5:08 am
Digg Defender…
I don’t anticipate any of my rantings making it onto Digg or Slashdot, but just in case I’ve installed the Digg Defender plugin, which redirects any visitors from such high-traffic sites to the Coral Cache system to reduce the chance of the…
on April 28th, 2006 at 12:47 am
[…] Digg effect wins again! Poor poor host. If I ever get enough traffic to warrant it, I might eventually get a domain with a GoDaddy/CacheFly combo. For now, I’ve just installed the Digg Defender plugin in the hopes of preventing this type of situation from happening again. […]
on April 28th, 2006 at 8:59 am
[…] WP Plugin: Digg Defender pre-WP-cache caching plugin that sends you over to Coral Cache if you’re coming from /., digg, etc. It will give an “order-of” performance increase, according to my tests, even on shared hosting. Thanks Elliott Technorati Tags: digg defender wordpress plugin […]
on May 2nd, 2006 at 1:58 pm
What about OSNews.com? My site bit the dust yesterday from there….
on May 2nd, 2006 at 2:10 pm
nm, anyone that wants to add a site; see digg-defender-functions.php and add the domain to the array.
on May 6th, 2006 at 8:13 am
[…] The plugin, digg defender, by Elliott Back, redirects traffic to Coral Cache, thus relieving traffic from your site and possibly preventing your provider from shutting your site down. Elliott also explains the new Wordpress cache hook, advanced-cache.php, and modifying .htaccess to redirect URLs to the Coral Cache. […]
on May 13th, 2006 at 2:36 am
[…] If you suspect the possibility that your website might get a huge influx of traffic someday, there are ways to protect against that. Well, if you are running a weblog using the WordPress blogging platform, you can use Elliot Back’s Digg Defender plugin (found via Stupid WordPress Tricks). […]
on May 17th, 2006 at 1:23 pm
[…] Wordpress is a firm believer of being completely dynamic. All the content is stored in the database, and a page is created on the fly before it is served. A high-traffice website can cause high load on the database and can slow down things as Wordpress hasto depend on the database for every serving of the page. However, there are certain plugins available that can cache the system and offer better performance. […]
on May 19th, 2006 at 11:38 pm
Digg Defender works well however it doesn’t play nice with WP Cache for me.
I Use:
WordPress 1.5.2
WP Cache version 2.0.17
By itself WP Cache works good but as soon as Digg Defender is activated WP Cache no longer works. Is there a setting somewhere that I need to change?
on May 21st, 2006 at 12:03 pm
Hey Elliott–
Great job on the plugin! I had a post on Digg front page linking to my server last night, so lucky I installed DD earlier that day! It definitely did the trick most of the time, but occasionally I was getting the wordpress static “cannot connect to the database” being served up by coralcdn for some reason, and at times when a direct link was working fine. Any ideas? Is it just caching it at a moment when it’s overloaded, and if so is it possible to set an expiry time on it or induce recaching?
Also, it seemed to work better when I deactivated WP-Cache. When you say “Note that if WP-Cache is also installed, only raw pages will be buffered, and not their contents, because WP-Cache will have already cached the page.” do you mean the stylesheets and graphics are not coralcdn’ed and are being served by the original server?
Sorry for all the questions, and thanks again for a great and useful plugin.
on June 6th, 2006 at 12:05 am
Dude, you saved my *ss!! we just got boingboing’d and i was watching server load go through the roof! At 8pm PST!
We’ve got an online comic here: smithmag.us/shootingwar and I’d already offloaded all images to photobucket. Alas, I was still getting hammered.
Since people are reading all our comic chapters and not just linking to one post (the boingboing link only goes to our homepage), I actually tweaked your function to redirect everything.
I’ll change it back once things calm down.
thanks!
on June 6th, 2006 at 11:57 pm
[…] Digg Defender: A Plugin For Wordpress by Elliott Back (tags: wordpress plugin cache) […]
on June 11th, 2006 at 9:26 pm
[…] Digg Defender: A Plugin For Wordpress by Elliott Back A plugin to prepare your site for what was formerly known as the /. effect. (tags: wordpress plugin 404) […]
on June 24th, 2006 at 3:14 am
[…] So far, I’ve installed the Digg Defender and WP-Cache plugins for WordPress. Digg Defender sees traffic from digg.com and redirects it to an external cached version of the page on a free service called Coral Cache. Ricardo Galli, author of WP-Cache, describes his plugin as: An extremely efficient WordPress page caching system to make your site much faster and responsive. It works by caching Worpress pages and storing them in a static file for serving future requests directly from the file rather than loading and compiling the whole PHP code and then building the page from the database. […]
on June 26th, 2006 at 10:05 am
Hi Elliot, I moved my Wordpress Blog. And I tried to enable your plugin but it refused to work, Please help me out !!
on July 31st, 2006 at 9:39 am
I am wondering how I can permanently coralize all my image content. I syndicate a lot of my content from other sites and have to manually make sure I am not hotlinking images from them. If I could re-write all external images to coral that would be really cool. Any ideas?
on August 4th, 2006 at 12:26 pm
And now the zip file is working fine. Wonder why it didn’t work before…
on August 31st, 2006 at 8:07 am
Great idea and it seemed to work for the first item I submitted to digg but now I get this error when the plugin is enabled:
Warning: Cannot modify header information - headers already sent by (output started at /blog/index.php:1) in //blog/wp-content/plugins/digg-defender/digg-defender-phase1.php on line 14
Can anyone tell me why or offer a solution please?
Many thanks in advance.
on October 23rd, 2006 at 12:21 pm
[…] You ‘ll need two plugins for this: wp-cache(direct download) and Digg Defender(direct download) […]
on November 1st, 2006 at 8:18 am
[…] Digg This Reloaded Authors Description: Adds Digg story link/button on detection of Digg referrer, otherwise a submit to Digg link is displayed. My overview: A massive improvement on the Digg This Plugin that I’ve mention previously and personally had issues with. I’ve seen this working fine elsewhere but won’t work properly (at the moment) with Digg Defender. I’ve been in touch with the author and they’re hopefully going to work on a merged plugin tomorrow to resolve this issue. You can see the little Digg button at the bottom of each post. […]
on November 2nd, 2006 at 2:16 am
[…] Due to an earlier request, I set about combining the Digg Defender Plugin and my own Digg This Reloaded Wordpress plugin. […]
on December 13th, 2006 at 1:56 pm
Is the criteria for using coral based on HTTP referer? Is it possible to use this to redirect to coral if a database connection fails
on December 14th, 2006 at 4:27 pm
Oops, I may be an idiot; thanks.
on December 19th, 2006 at 5:32 am
[…] Digg Defender: A Plugin For Wordpress by Elliott Back Hvis man bliver digget kan man forsvare sig med et plugin (tags: wordpress plugin) […]
on December 19th, 2006 at 12:45 pm
oh man i hope this one works couse i can’t find anything that works for me or maybe i’m just plain stupid….pffffff
on January 20th, 2007 at 1:01 pm
Don’t spose you know if this will be WP2.1 compatible do you?
Cheers,
on January 26th, 2007 at 6:17 am
[…] Digg This Reloaded Authors Description: Adds Digg story link/button on detection of Digg referrer, otherwise a submit to Digg link is displayed. My overview: A massive improvement on the Digg This Plugin that I’ve mention previously and personally had issues with. I’ve seen this working fine elsewhere but won’t work properly (at the moment) with Digg Defender. I’ve been in touch with the author and they’re hopefully going to work on a merged plugin tomorrow to resolve this issue. You can see the little Digg button at the bottom of each post. […]
on February 15th, 2007 at 4:36 pm
[…] However, where stumbleupon shines for the site owner, digg is painful. A front-page story on digg will get you a bazillion hits in the space of an hour, at which point your servers will catch fire. You enjoy the exposure, but your users don’t because they can’t see your site. […]
on February 22nd, 2007 at 4:22 pm
[…] The deal’s not done, but my earlier allusions to D.C. mythology (and alternate Earths) were project-specific. I hope to be able to say what it is soon. And yes, it’s cruel of me to leave it floating out there. It would have been a lot worse if I hadn’t installed a plug-in which automatically cached the page on Coral. ↩ February 22, 2007 @ 10:35 am | Share This | Filed under: Projects, News, Los Angeles, The Movie […]
on February 27th, 2007 at 1:04 am
[…] Alternatively, you can use Digg Defender. Digg Defender uses the WP-Cache system hook to redirect requests from high-traffic sites like Digg and Slashdot to a free global cache system. It’s compatible with WP-Cache, so no worries. I am contemplating whether or not to use it. I guess an extra line of defense wouldn’t hurt. You can download Digg Defender here, and the installation instructions are here. […]
on February 27th, 2007 at 1:18 am
[…] Download Digg Defender here Installation instructions here. […]
on March 13th, 2007 at 7:27 pm
[…] Digg Defender […]
on March 25th, 2007 at 2:39 pm
A global cache system? I’ve never heard of that…how does it work?
on April 6th, 2007 at 8:48 am
[…] Next I’ve got Digg Defender set up. This plugin notes when hits are coming from various sites (notably in this case Digg) and redirects the incoming request from ArsGeek to a mirror site. People can still read and reach the article, but aren’t bringing ArsGeek down with more traffic. […]
on April 9th, 2007 at 12:40 pm
I just installed digg defender on my website www.mypersonalfinancejourney.com and activated it and now my entire site is blank including the admin pages. Help! what happened?
on April 9th, 2007 at 5:17 pm
You probably included some random whitespace somewhere after one of the files php tags.
on May 18th, 2007 at 11:44 pm
I think there is a small bug in digg-defender.php digg_defender_check_global_config.
I think the check for “!is_writable($global) || !digg_defender_replace_line(…))” should instead read something like:
if (!WP_CACHE && !is_writeable($global) && !digg_defender_replace_line(…))
To put it another way, if WP_CACHE is enabled, OR if the file is writeable, OR if modifying the file succeeded, then the config is good (return true).
on June 3rd, 2007 at 10:02 pm
What a brilliant plugin. I have it installed and the options indicate the necessary files are writeable, but is there anyway to give this a test run to see if it’ll actually work? I submitted my site to Reddit, then travelled back to my site via the reddit link, but I didn’t notice a difference. I was expecting to see the suffix address nyud.net:8080 added to my address but it didn’t show. Thanks for contributing such a nifty plugin though.
on August 3rd, 2007 at 12:47 am
This is a grea plugin. This type of is priceless for the new kid on the block.
Thanks
Vic
on August 17th, 2007 at 12:56 am
[…] Digg Defender - Caching plugin that directs traffic coming from Digg to Coral Cache to try and prevent a server crash from the Digg effect. […]
on August 20th, 2007 at 8:44 pm
[…] Digg Defender […]
on August 29th, 2007 at 5:52 pm
Does anyone know if there’s a way to automatically redirect to the post itself if someone tries to link directly to an image?
I know that might be a tall order, but people keep submitting images from my site to Digg and Stumbleupon, rather than the blog posts, so then I lose out on all the traffic, subscribers and ad revenues that might come from people actually coming to the site.
on October 3rd, 2007 at 1:27 pm
[…] Information/Background The idea behind this script came from Digg defender by Elliot Back. Sadly, Elliot’s script is only a WordPress plug in, and well, Word Press blogs aren’t the only websites that can experience high traffic. Although I wrote this script myself, the credit for the idea does reside with Elliot. Note: At the time of writing this script, I was using a custom CMS for the website however as of recently I have switched to WordPress. […]
on October 11th, 2007 at 3:56 am
[…] Consider using the Digg Defender plugin I didn’t get a chance to fully test this one, but it may just save your site in the event of a traffic spike such as Digg. It works with the Coral CDN service to send Social media traffic to a mirror site. […]
on October 19th, 2007 at 2:26 pm
[…] Digg Defender - Caching plugin that directs traffic coming from Digg to Coral Cache to try and prevent a server crash from the Digg effect. […]
on October 27th, 2007 at 10:22 am
[…] Er zijn nog wat dingen die je liever niet gebruikt, maar die je wel kunnen redden, zodra je echt krap zit met je dataverkeer. Kijk bijvoorbeeld eens naar de plugin, Digg Defender. Deze plugin zorgt ervoor dat alle bezoekers die via Digg jouw site benaderen, worden doorverwezen naar een cache-pagina van Coral Cache, waar jouw inhoud op te zien is. Dan zijn er ook nog zaken, die niet iedereen kan gebruiken. Ik praat dan over bijvoorbeeld eaccelerator, een ‘versneller’ voor PHP. Omdat niet iedereen technische dingen kan aanpassen op de server waar zijn/haar weblog op draait, is dit vaak niet haalbaar. Sommige hosters staan wel open voor dit soort ideeën, dus het kan nooit kwaad om er eens contact mee op te nemen. Deel dit artikelVind u dit een leuk artikel? Deel het dan met anderen via een van deze sites: […]
on October 30th, 2007 at 9:27 am
[…] Digg Defender: Find out how to keep your blog running quickly and smoothly even if your posts become top-rated Digg stories. […]
on October 30th, 2007 at 9:27 am
[…] Digg Defender: Find out how to keep your blog running quickly and smoothly even if your posts become top-rated Digg stories. […]
on November 29th, 2007 at 5:49 pm
[…] Er zijn nog wat dingen die je liever niet gebruikt, maar die je wel kunnen redden, zodra je echt krap zit met je dataverkeer. Kijk bijvoorbeeld eens naar de plugin, Digg Defender. Deze plugin zorgt ervoor dat alle bezoekers die via Digg jouw site benaderen, worden doorverwezen naar een cache-pagina van Coral Cache, waar jouw inhoud op te zien is. Dan zijn er ook nog zaken, die niet iedereen kan gebruiken. Ik praat dan over bijvoorbeeld eaccelerator, een ‘versneller’ voor PHP. Omdat niet iedereen technische dingen kan aanpassen op de server waar zijn/haar weblog op draait, is dit vaak niet haalbaar. Sommige hosters staan wel open voor dit soort ideeën, dus het kan nooit kwaad om er eens contact mee op te nemen. Gerelateerde artikelen: Fatal error: Call to undefined function: related_posts() in /home/coenjacobs/domains/coenjacobs.net/public_html/temp/wp-content/themes/glossyblue-1-4/single.php on line 15 […]
on April 7th, 2008 at 11:59 pm
Quality plugin, thanks a lot Mr. Back!
on June 3rd, 2008 at 11:08 pm
i like it too. nice plugin. keep up the good work
on June 17th, 2008 at 11:05 pm
[…] Digg Defender […]
on June 19th, 2008 at 6:08 am
[…] Digg Defender […]