Easy Setup for WP Super Cache

One of the things that has bugged me about the WP Super Cache settings page was how it was laid out. Well, the next version of the plugin will display a simplified settings page to new users. If you’re upgrading, you’ll get the same old page as ever, don’t worry.

This version also adds a new method of serving cache files. It uses PHP, but serves supercache files. So, it’s a halfway house between using mod_rewrite (difficult to install for some users), and the legacy caching of WP Cache. That caching will be what is activated for users who use the simplified settings page.

There are lots of other bug fixes. The cache tester works if WordPress is installed in a sub directory, the admin page is separated out into tabs now to make it easier to find things. Error messages show up as “update messages” at the top of the browser now, making it easier for new users to figure out when mod_rewrite rules need updating and when other house keeping tasks need doing.

The code is red hot, liable to bend and break and may cause problems but it works fine here and on a test multi site install but I need testers to hammer on it and do things I don’t expect. If you’re brave, grab the development version off the download page. Thanks!

WIP: the Super Cache admin page

A small update, I’m slowly working through the WP Super Cache admin page in an effort to make it better. You can in fact download the development version if you want to follow along.

What you see above is my first pass. An effort to make the first options section match the look and feel of the standard Settings pages in WordPress. It’s all likely to be mixed around and moved about before the next release, so please, dig in and lend a hand!

WP Super Cache 0.9.9.2

WP Super Cache 0.9.9.2 has just been released! This version works with WordPress 3.0 and adds a number of new features and bug fixes:

  • Cache Preloading will cache every post on your site.
  • A cache tester that will check if your homepage is cached.
  • Much better support for mobile plugins.
  • Mod rewrite rules can now be updated from within the admin page.
  • Lots and lots of bugs fixed. See the changelog for more details.

Preloading creates lots of files on your server so if you have many thousands of posts please be aware of this. Filesystem limitations may cause problems if you use a flat permalink structure. For example, ext2 or ext3 only allows 32,000 directories in a directory. If you have more than that number of posts you may run into problems.

The plugin does not preload category or tag pages but because your single posts will be cached you’ll find the load on your server will be reduced. Uncached pages will be served more quickly and your visitors will have a quicker and better experience on your site.
You may also see an increase in site traffic if your server was previously underpowered!

Update! I just released 0.9.9.3 to address the (mostly minor) bugs that were reported overnight. If you don’t notice anything wrong there’s no need to upgrade.

Preload the cache in WP Super Cache

See that nice dip in the graph for this week? I started to preload the cache used by WP Super Cache last Sunday and it’s made a noticeable difference in the load on my server here. The big spike is the preloading process.

I’ve always discouraged users from preloading the cache (Askapache Crazy Cache will do this for any cache plugin), mainly because of the possible problems so many files will cause for hosting companies. If you have thousands of cache files, it’s going to take so much longer to recover from a disk crash.
On the other hand, Google will now be using speed as a metric for judging how “good” a website is. In the past this plugin ignored the pages visited by bots because the bots only visited each page once so caching a page after the fact was pointless. The page, all pages, have to be cached first before Google ever visits.

That’s what it looks like. Once you start preloading it launches a wp-cron job to fetch 100 posts, then schedules another job 10 seconds in the future to fetch another 100 posts until it finishes. It also disables garbage collection of old pages, but making comments or posts will still clear out the appropriate cached files.
It only caches single posts right now. It may not be worth caching archive or tag pages because many sites already tell bots to ignore those pages as the server is doing less work it will serve those archive pages more quickly anyway.

The preloading only works if you’re using the plugin in Supercache or “ON” mode. It’s still a work in progress but has worked fine here. As well as the preloader the development version of the plugin has:

  1. Better support for mobile plugins.
  2. A cache tester.
  3. Can be configured to only delete the page a comment is left on, rather than the front page and associated pages.
  4. Works in WordPress 3.0.

It also has a number of bug fixes and other features added too.

I need testers though, so grab the development version from the download page. Install it and please leave feedback here or preferably on the support forum.

Remove unused utm_source from your urls

Sometime last year I noticed that links to my blog on Feedburner had attracted a few extra parameters. A simple link to a post became this huge monstrosity:

http://ocaoimh.ie/exploit-scanner-095/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+HolyShmoly+(Holy+Shmoly!)&
utm_content=Google+Reader

It’s a marketing thing right? It’s all useful information but I don’t really care about it, have never used it and don’t like my URLs getting mangled. It annoys me for two reasons:

  • People will probably use that big long url in their own posts. Other people will use the shortened custom permalink that my blog provides. Won’t the pagerank earned by the post be split in two now?
  • It makes caching less efficient. Supercache won’t create a static cached file of the page. It will create a regular php powered cache file but when you’re running Supercache you want the very best performance don’t you?

So I added a new option to Supercache to redirect the url and get rid of the utm_source bloat.

If you want to give it a go, grab the development version of the plugin and upgrade.

Oh, and if someone has decent docs on utm_source and it’s friends I’d love to read it. Google didn’t return much when I went looking.

WP Super Cache 0.9.9

Well, the new WP Super Cache is available now.

This release adds experimental object cache support. Don’t go looking for it unless you have an external object cache already. It won’t show up. I recommend using the Memcached object cache.

Some of the other major changes include more translations: Chinese (Pseric), Ukranian (Vitaly) and Japanese (Tai). The Italian and Japanese translations have since been updated but not included in 0.9.9. You can grab them from the languages directory if you don’t want to wait until the next release.

If you have WordPress Mobile Edition installed the plugin will grab the list of mobile user agents from that and warn if your .htaccess is outdated.

And, a small but significant change is that the PHP cache loader will use the static “super” cache if necessary. This might happen if your rewrite rules aren’t working properly and not serving cache files. At least your anonymous visitors will see some sort of cached file. Use the debugging system built into the plugin to determine where the cache comes from.

See the changelog for the complete list of changes.

WP Super Cache with Object Cache support

Here’s a quick post to encourage brave testers. I’m adding object cache support to WP Super Cache so you’ll be able to store your cached files in a memcached backend instead of disk.

It’s not complete but it’s running on this blog and well, you’re reading this which means it’s doing something and not breaking! If you want to give it a go grab the development version from the download page.

There are few caveats, but three spring to mind:

  • It won’t cache anything for “known” users. That is users who are logged in or leave comments. Usually a tiny minority of the visitors to any site.
  • Refreshing of the cache is very incomplete. If you leave a comment, the cache for that page may not update immediately. The cache lifetime is set to 30 seconds, and after you leave a comment you become a “known user” and see the uncached version of the page anyway.
  • When posts are updated the whole cache is invalidated.

If you don’t know what memcached is, or how to set it up then you probably don’t want to test this. If you do, use Google and find out about them. Unfortunately I don’t have time to explain how to install it.

Inspiration and some code taken from batcache, the excellent caching plugin we use on WordPress.com.

Update! I updated the Changelog in the readme.txt and I’m looking for testers. Here’s what’s new in the development version:

* Added experimental object cache support.
* Added Chinese(Traditional) translation by Pseric.
* Added FAQ on WP-Cache vs Supercache files.
* Use Supercache file if WP-Cache file not found. Useful if mod_rewrite rules are broken or not working.
* Get mobile browser list from WP Mobile Edition if found. Warn user if .htaccess out of date.
* Make sure writer lock is unlocked after writing cache files.
* Added link to developer docs in readme.
* Added Ukranian translation by Vitaly Mylo.
* Added Upgrade Notice section to readme.
* Warn if zlib compression in PHP is enabled.
* Added compression troubleshooting answer. Props Vladimir (http://blog.sjinks.pro/)
* Added Japanese translation by Tai (http://tekapo.com/)
* Updated Italian translation.

The biggest changes are the addition of the object cache and a small change to the php code that serves cached wp-cache files. If the mod_rewrite rules on your site don’t work for whatever reason the plugin will look for the Supercache file and serve that instead. An extra header is added to the served page when this happens. It’s all in the readme.txt!

WordPress, Nginx and WP Super Cache

If you host your own WordPress blog, it’s probably on Apache. That all fine and good. For most sites Apache works wonderfully, especially as it’s so easy to find information on it, on mod_rewrite and everything else that everyone else uses.

One of the alternatives is Nginx, a really fast webserver that streaks ahead of Apache in terms of performance, but isn’t quite as easy to use. That’s partly because Apache is the default webserver on most Linux distributions and hosts. Want to try Nginx? Here’s how.

Install Nginx. On Debian based systems that’s as easy as

aptitude install nginx

Nginx doesn’t talk PHP out of the box but one way to do it is via spawn-fcgi. Here’s where it gets complicated. (Docs summarised from here)

  1. Install php5-cgi. Again, on Debian systems, that’s
    aptitude install php5-cgi
  2. Edit /etc/nginx/sites-available/default and add the following chunk of code to the “server” section:
    location ~ \.php$ {
            include /etc/nginx/fastcgi_params;
            fastcgi_pass  127.0.0.1:9000;
            fastcgi_index index.php;
            fastcgi_param  SCRIPT_FILENAME  /var/www/nginx-default$fastcgi_script_name;
    }
  3. Install lighttpd for the spawning command.
    apt-get install lighttpd

    You’ll probably get an error at the end of the install if Apache is already running on port 80. Edit /etc/lighttpd/lighttpd.conf and uncomment the line

    server.port = 80

    and change 80 to 81. Now run the apt-get command again and it will install.

    /etc/init.d/lighttpd stop

    will stop lighttpd running. (You don’t need it)

  4. Create a new text file, /usr/bin/php-fastcgi with this:
    #!/bin/sh
    /usr/bin/spawn-fcgi -a 127.0.0.1 -p 9000 -u nobody -f /usr/bin/php5-cgi

    The user “nobody” should match the user Apache runs as to make things easier to transition.
    Make it executable with

    chmod 755 /usr/bin/php-fastcgi
  5. Create another new file /etc/init.d/init-fastcgi and make it executable with the chmod command too. Put this in the file:
    #!/bin/bash
    PHP_SCRIPT=/usr/bin/php-fastcgi
    RETVAL=0
    case "$1" in
        start)
          $PHP_SCRIPT
          RETVAL=$?
      ;;
        stop)
          killall -9 php
          RETVAL=$?
      ;;
        restart)
          killall -9 php
          $PHP_SCRIPT
          RETVAL=$?
      ;;
        *)
          echo "Usage: php-fastcgi {start|stop|restart}"
          exit 1
      ;;
    esac
    exit $RETVAL
  6. Start the PHP processes with
    /etc/init.d/init-fastcgi start

    and make sure it starts on every reboot with

    update-rc.d init-fastcgi defaults

That’s the PHP part of things. In Debian, the default root is “/var/www/nginx-default” so put an index.php in there to test things out. Stop Apache and start Nginx (if this is a test server only!) and visit your site. Works? Now to get WordPress and WP Super Cache working.

Open up /etc/nginx/sites-enabled/default in your editor and comment out the text already there with # characters. Paste the following in. Change paths and domains to suit your site. (via)

server {
        server_name  example.com www.example.com;
        listen   80;
        error_log   /www/logs/example.com-error.log;
        access_log  /www/logs/example.com-access.log;

        location ~ \.php$ {
                include /etc/nginx/fastcgi_params;
                fastcgi_pass  127.0.0.1:9000;
                fastcgi_index index.php;
                fastcgi_param  SCRIPT_FILENAME  /www/example.com/htdocs$fastcgi_script_name;
        }

        location / {
               gzip  on;
               gzip_http_version 1.0;

               gzip_vary on;

               gzip_comp_level 3;

               gzip_proxied any;

               gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

               gzip_buffers 16 8k;
               root   /www/example.com/htdocs;
               index  index.php index.html index.htm;
# if the requested file exists, return it immediately
               if (-f $request_filename) {
                       break;
               }

               set $supercache_file '';
               set $supercache_uri $request_uri;

               if ($request_method = POST) {
                       set $supercache_uri '';
               }

# Using pretty permalinks, so bypass the cache for any query string
               if ($query_string) {
                       set $supercache_uri '';
               }

               if ($http_cookie ~* "comment_author_|wordpress|wp-postpass_" ) {
                       set $supercache_uri '';
               }

# if we haven't bypassed the cache, specify our supercache file
               if ($supercache_uri ~ ^(.+)$) {
                       set $supercache_file /wp-content/cache/supercache/$http_host/$1index.html;
               }

# only rewrite to the supercache file if it actually exists
               if (-f $document_root$supercache_file) {
                       rewrite ^(.*)$ $supercache_file break;
               }

# all other requests go to WordPress
               if (!-e $request_filename) {
                       rewrite . /index.php last;
               }
        }
}

I think the gzip settings above will compress cached files if necessary but Nginx can use the already gzipped Supercache files. The version of Debian I use doesn’t have gzip support compiled in, but if your system does, take a look at the gzip_static directive. Thanks sivel.

Finally, edit /etc/nginx/nginx.conf and make sure the user in the following line matches the user above:

user www-data;

I changed it to “nobody nogroup”.

Now, stop Apache and start Nginx:

/etc/init.d/apache stop; /etc/init.d/nginx start

WP Super Cache will complain about mod_rewrite missing, and you should disable mobile support.

How has it worked out? I only switched on Friday. The server did do more traffic than normal, but I put that down to the floods in Cork. Weekend traffic was perfectly normal.

Load on the site is slightly higher, probably because my anti-bot mod_rewrite rules aren’t working now. Pingdom stats for the site haven’t changed drastically and I think the Minify plugin stopped working, must debug that this week. Switching web servers is a huge task. I disabled mobile support in Supercache because I need to translate those rules to Nginx ones. A little birdie told me that he’s going to be writing a blog post on this very subject soon. Here’s hoping he’ll put fingers to keys soon.

Have you switched to Nginx? How has your switch worked out for you?

WP Super Cache 0.9.8

WP Super Cache version 0.9.8 is now available. WP Super Cache is a page caching plugin for WordPress that will significantly speed up your website.

New in this release are 2 translations. The Spanish translation is by Omi and the Italian by Gianni Diurno. Please, if you use their translations, drop by their sites and leave a thank you comment! They’ve been very patient with me as I fixed gettext bugs and added new text. Both have blogged about the translations if you need to know more: Gianni, Omi.

The second major feature to go in is an “advanced” section to the debugger. This allows the plugin to check the front page every 5 minutes to make sure everything is ok. It monitors for 2 very rare problems:

  1. Very very occasionally, the front page becomes a gzip file that downloads. It happened here once and I examined the cache file. There was nothing wrong with it. It was perfect. I suspect Apache and mod_rewrite got confused somehow but clearing the cache fixed it. The file generated after was exactly the same size as the old one, so no chance it got “double gzipped”.
  2. In certain rare cases, where a blog has a static front page, and uses a permalink structure of /%category%/%postname%/, the wrong page may be cached as the front page. Even if your blog satisfies the two conditions above it may not suffer from this problem. I tried it on this blog for a few days and couldn’t reproduce it at all!

Nevertheless, if you’re concerned edit your wp-cache-config.php and add this line:

$wp_super_cache_advanced_debug = 1;

Reload the admin page and you’ll see this added to the debug section:

advanced-debug

If activated, it will check your front page every 5 minutes. It’s not activated by default because these errors only happen to a small number of blogs. I’ve also noticed that WordPress seems to randomly forget to run the page checker from time to time. I debugged it and the job simply disappears from the wp-cron system! I’ve no idea why, but reloading the admin page schedules it again.
If you’re still paranoid, set your cache expiry low so at least the cache files will be recycled quickly.

Caching, Minification and CDNs

Oh, there’s a new caching plugin on the scene. W3 Total Cache works like Supercache’s half-on mode but can store to memory as well as disk (like Batcache) but also does minification and supports CDNs. I’ve been asked a few times if I’ll support those features too but I don’t see why as other plugins already have that covered (and frankly, I don’t have time to maintain such complex features):

  1. WP Minify “integrates the Minify engine into your WordPress blog. Once enabled, this plugin will combine and compress JS and CSS files to improve page load time.” Thaya is very responsive and fixed a bug I reported quickly.
  2. There are any number of CDN plugins for WordPress. I don’t use a CDN so I can’t recommend one but OSSDL CDN Off Linker might be worth a shot. This post on it mentions Supercache plus, a fork of this plugin.

Traffic Spikes and Benchmarks

I really should collect more of these. A few weeks ago Mark Pilgrim blogged about how his book had been republished by a 3rd party and put up for sale on Amazon. His book was published under the GNU Free Documentation License so that’s perfectly legal to do, even if a little unusual as it can be downloaded from Mark’s website and is for sale by his publisher. The blog post generated a lot of interest and a few days later I received a donation from Mark, followed by a thank you email. I’m a big fan of what Mark does, so if it had been a physical cheque or a letter I’d have framed it!
A few days after that he tweeted the following graph. Nice spike of traffic eh? His server held up fine with help from WP Super Cache.

diveintomark.org-dashboard

And finally, some benchmarks, in Russian unfortunately but the pages translates well.

caching-benchmarks

Summary of changes in 0.9.8:

  • Added Spanish translation by Omi.
  • Added Italian translation by Gianni Diurno.
  • Addded advanced debug code to check front page for category problem. Enable by setting $wp_super_cache_advanced_debug to 1 in the config file.
  • Fixed wordpress vs wordpress_logged_in cookie mismatch in cookie checking function.
  • Correctly check if WP_CACHE is set or not. PHP is weird.
  • Added wp_cache_clear_cache() to clear out cache directory.
  • Only show logged in message when debugging enabled.
  • Added troubleshooting point 20. PHP vs Apache user.
  • Fixed problem deleting cache file.
  • Don’t delete cache files when moderated comments are deleted.

PS. WordCamp Ireland is on in early March next year in picturesque Kilkenny. Here’s Sabrina’s launch post. Sign up! I’ll be going!

WP Super Cache Developer Documentation

I’ve finally found the time to write up some documentation for developers who want to work with WP Super Cache.

It’s a work in progress but should help other plugin developers who want to interact with the cache.

Suggestions and comments welcome.

PS. If you’re in Cork on November 14th, head along to BarCamp Cork III. I’ll be giving a talk, “How WP Super Cache Works”. It’ll be less technical than this but I’ll answer questions too. Check out the other sessions too.