Writing WP Super Cache Plugins

WP Super Cache is a full page caching plugin for WordPress. When a page is cached almost all of WordPress is skipped and the page is sent to the browser with the minimum amount of code executed. This makes the page load much faster.

Unfortunately if you want to run code on every page load you’re out of luck as regular WordPress plugins are not loaded or executed. You’ll need to write a WP Super Cache plugin. This short introduction will not teach you how to write plugins but the example plugins that ship with WP Super Cache will get you a long way towards understanding how they work.

Screen Shot 2017-10-25 at 16.49.10

WP Super Cache ships with some example plugins in wp-super-cache/plugins/. Some of them even do useful tasks like help with domain mapping and Jetpack integration. There’s one called “awaitingmoderation.php” which removes the text “Your comment is awaiting moderation.” when someone writes a moderated comment.
There’s also dynamic-cache-test.php which is a complicated beast but it’s heavily commented. It allows you to add template tags to your site that are replaced when the cached page is loaded.

Before you get started writing a plugin you should be aware that you should not use the wp-super-cache/plugins/ directory. Every time WP Super Cache is updated this directory is deleted. So, edit your wp-config.php and set $wp_cache_plugins_dir to another directory where you’ll put your plugin.

These plugins run before most of WordPress has loaded. That means you can’t rely on some of the nice features of WordPress such as filters and actions. However, WP Super Cache has it’s own action/filter system that is similar to actions and filters in WordPress:

  • add_cacheaction( $action, $func )
  • do_cacheaction( $action, $value = ” )

A cacheaction is also a filter. If you hook on to a cache action that has a parameter, you must return that parameter at the end of the function like you would with a WordPress filter.

Screen Shot 2017-10-25 at 16.52.59

If you need to hook into a WordPress filter use the imaginatively named cache action “add_cacheaction”. That runs on “init” so the function that is executed can use add_action() or add_filter(). You can see this in action in the plugins/dynamic-cache-test.php or plugins/awaitingmoderation.php scripts.

Two very useful filters are the WordPress filter, “wpsupercache_buffer” (in wp-cache-phase2.php) that is used to modify the page before it is cached and the cache action “wpsc_cachedata” (in wp-cache-phase1.php) is used to modify the cached page just before it’s served.

WP Super Cache 1.5.0

WP Super Cache is a fast full-page caching plugin for WordPress. Download it from your dashboard or get it here.

Version 1.5.0 has been in development for some time. It has a ton of bug fixes and new features.

REST API

The headline new feature is REST API access to the settings. This will allow developers to create their own interface to the settings of the plugin. Unfortunately it isn’t yet documented but you can see the code in the rest directory. Start with load.php where you’ll find the code that registers all the endpoints. Users who access the API must be logged in as admin users. If you want to test the API, see the end of this post.

Settings Page

We have also simplified the settings page to make it easier to choose which caching method is used.

Instead of maybe confusing the user with technical words like PHP, mod_rewrite and WP-Cache we have split them up into “Simple” and “Expert” delivery methods, and done away with mentioning WP-Cache completely. Simple delivery uses PHP, expert uses mod_rewrite and well, WP-Cache got the boot because it’s always active anyway.

WP-Cache caching is always active, but it can be disabled in different ways.

  • Disable caching for known users.
  • Don’t cache pages with GET parameters
  • Disable caching of feeds

Headers

We expanded the number of headers cached by the plugin. The list of headers was borrowed from Comet Cache. However, anonymous users will still only see the bare minimum like content-length or content-type. If you need to use security headers like “X-Frame-Options” or “Content-Security-Policy” you should enable caching of HTTP headers. This unfortunately disables super caching so only WP-Caching is used but it’s still very fast (and faster in this release than before which I will get to below). You can also use the “wpsc_known_headers” filter to modify the list of recognised headers.

WP-Cache Reorganisation

WP-Cache cache files are split into two files – one holds the page content, the other (meta file) holds information about the page such as cookies, headers and url. In the past these files were stored in two directories which could become a problem if there were many thousands of those files. Even with only a few hundred files, maintenance could be an issue as deleting related files (like page archives, or copies of the front page) needed every meta file to be inspected.
Now the files are stored in the supercache directory structure that mirrors your permalink structure. Deleting related files is is simpler and serving files will be faster as the operating system won’t need to open a directory of thousands of files.
If you currently rely on WP-Cache files, the plugin will still look for them where they are, but new WP-Cache files will be created in cache/supercache/example.com/ (where example.com is your hostname).

Sitemaps

We added support for caching sitemaps, but your sitemap plugin will need to cooperate to get it to work. The sitemap plugin needs to identify the sitemap request as a feed. Jetpack 5.1 now supports this since #7397. You can disable the caching by excluding feeds from caching.

Debugging Improved

The debug log is now protected by a username/password. For convenience, the username and password are the same but they are a long md5 string:

Deleting the log file clears it and resets it ready for more logging. Before, toggling debugging would create a new debug log and the old one would be kept around, but not linked, until deleted by garbage collection, and of course they were text files anyone could access.

This release includes lots of other small bug fixes and changes. Take a look at the number of closed PRs for an exhaustive list of those changes!

Testing the REST API

There are a number of ways to send POST requests to a web server but one I like is using curl in a shell script. You’ll need two bits of information from the website:

  1. The “wordpress_logged_in” cookie from your browser.
  2. The wp_rest nonce which you can get by adding `echo wp_create_nonce( ‘wp_rest’ );` somewhere on your site where you’re logged in. It’s good for 24 hours.

My test script looks something like this:
export NONCE='1234567890'
export COOKIE='wordpress_logged_in_xxxxxxxxxxxxxxxxxxxx=1234567890'
curl -v -X "GET" -H "Content-Type: application/json" -H "X-WP-Nonce: $NONCE" -H "Cache-Control: no-cache" -H "Cookie: wordpress_test_cookie=WP+Cookie+check; $COOKIE" \
-d '{}' "https://example.com/wp-json/wp-super-cache/v1/settings/"

Thank you Translators!

Well, well. WP Super Cache translations are coming along well. 100% of strings in the development version are translated into Canadian English and Romanian! (That might change over the next few days as I have a couple of UI changes in mind, sorry!)

Other languages like Persian, Japanese, Spanish, Russian and Italian are roaring along and sometimes only need a handful of words or sentences translated to hit that magic 100% too. The next version of the plugin will grab translations from WordPress.org if they’re available which is awesome for users around the world. We’re still shipping translation files but I think the release after that may ship without and rely entirely on the up-to-date online translations.

So, thank you so much to the translators. Code is nigh on useless if the user and developer don’t speak the same language and the interface is incomprehensible. WP Super Cache has lots of technical language which must be a pain to translate. Your work is invaluable!

Mulțumesc! متشکرم! ありがとうございました!¡Gracias! Спасибо! Grazie! Hvala ti! 谢谢!Vielen Dank!

Take a sneak peek at WP Super Cache

WP Super Cache is a full page caching plugin for WordPress that makes your site faster, and helps deal with unexpected surges in traffic.

Over the last few months we’ve been busy working on the plugin to add new features and fix bugs and we can almost call it ready. It’s stable and usable and runs on this site but we would love more people to test it out before we release a new version.

Here’s just some of the new features and bug fixes we’ve been working on:

  • The plugin was based on WP-Cache which stored cache files in a single directory, and those (legacy caching) files were for the most part stored the same way all this time but now they’re being placed in the supercache directories (#177). This makes it easier to manage these files. The plugin doesn’t have to search through potentially hundreds of cache files for those that need to be deleted if a page updates or someone leaves a comment. Now all those files will be in the same directory structure the anonymous “supercache” files will be. I’m really excited about this feature as it makes caching for logged in users/users who comment and caching of pages with parameters so much faster now.
  • We’re adding a REST API to the plugin because in the future not everyone is going to use wp-admin to take care of their sites. Take a look in the rest directory for the code we’re working on.
  • Debug logs now have a username and password to protect them from prying eyes.
  • And many bugs fixed over the last few months.

Since “legacy caching” or “WP-Cache caching” is now more maintainable and faster we want to change the language describing how the plugin caches and delivers pages.

Currently the plugin asks you to choose between mod_rewrite, PHP and “Legacy page caching” which isn’t really useful. Most users won’t recognise those terms. It’s also not accurate as “legacy page caching” is active all the time as long as caching is enabled.
Instead we should have “Standard Caching” and “Super Caching”. Super Caching will then have simple and expert delivery methods.

Simple delivery is through PHP, while expert delivery uses mod_rewrite rules which means the .htaccess file has to be updated and hopefully the warning below it will discourage casual users from testing it.

Due to the huge number of changes in the plugin we really need people to give it a try and check if everything works ok. The changes to the settings page will hopefully make it easier for new users to get to grips with it too.

You can find the newest code on Github. The changes to the settings page are in #255 if you want to comment on them.

Thanks in advance! 🙂

How to Auto Schedule WordPress Posts

If you post to a WordPress blog on a regular basis like I do on In Photos dot Org you’ll no doubt recognise the fatigue that comes from adjusting the publish date every single time on a new post so it appears a day later. If you have multiple posts like on a daily photoblog you have to remember what day the last post was made and adjust the date accordingly.

A few years ago I wrote a small plugin that I never released to help schedule posts. In the media uploader you could select multiple photos and click a few buttons to be brought to a new page where you could enter title, content and tags for each image. Based on this experience, I suggested it as an idea to one of the teams at Automattic who built Post Bot. I used that for a long time and it has its strengths. If you’re posting content that has the same or similar tags you can copy and paste the tags from one post to another. I posted lots of black and white street images from my home town this way and it was super useful!

I got tired of manually typing out tags, and unfortunately the site broke a few times, with posts not scheduling or one time they scheduled all in one go. Luckily the problems were quickly fixed. However, I started using the WordPress post editor again and scheduling a bunch of photos that way.

Manually editing the publish date quickly became a chore. Lazarus, the form saver Chrome extension, would sometimes popup if I didn’t click exactly on the date, or as I said before I had to remember when the last post was made. They say there’s a plugin for everything, and there is for this too. Check out Publish to Schedule.

You tell “Publish to Schedule” which days and how many posts should be published and when you go into the post editor the next available date is picked for you! The date doesn’t change until you hit Publish but I already used it to schedule a number of posts and it works really well.

Edit: I forgot to mention Daily Image a new plugin by Sam Hotchkiss that does the same sort of job as Postbot but it runs on your own server. The first time you load the plugin it will show you every single unattached image in your media library which can be quite a number of images but it allows you to enter tags and quickly schedule images for posting in a simple manner.

Since my focus here is on image posts I should really mention the WordPress Export Plugin for Lightroom. When installed you can create a new export target that will resize and sharpen your image and upload it to your blog, even if it’s not a WordPress.com site.

Coming up in WP Super Cache 1.5.0

The next version of WP Super Cache will be one with some big changes! There are many small bug fixes and improvements but the one I’m most excited about is moving the legacy cache files into the supercache directory.

The legacy cache files were the files created by the old WP-Cache plugin upon which this plugin is based. They’re really useful as they store the headers sent from the server as well as the page contents. If you’re serving pages that aren’t regular html, such as JSON or XML you don’t want to tell the browser they’re text/html documents. This caching method is also used for anyone who is logged into your site, or left a comment.
There is a problem however. They’re stored in one directory. If you have many thousands of visitors interacting with your site you may end up with a directory containing thousands of files. The names of the cache files are a hash of the URL, gzip support and browser cookies so one file can match one user, or one file can be used by thousands of anonymous users. In the event that someone left a comment on a popular post the plugin has to search through all those files looking for the pages cached for other users who were also looking at that page. On a busy server that can cause problems.

So, in #177 I added code that moves the legacy cache files into the supercache directory. That means the files are stored in directories that reflect the URL of the page that was served which makes it very easy to delete the cached files belonging to that page as they’re all in the same directory!

The new code will look in the old location for legacy files first as some sites will have a large collection of cached files, but any new cache files will be created in the supercache directory.

Ian Dunn submitted code to cache the REST API. It’s not yet complete but we’ll be able to build on the changes to the legacy cache to make caching the API more efficient than it would have been before.

I really need people to help test this. The latest code is running on this site so I’m very confident in how well it works but just because it works on my odd little server doesn’t mean it will work right everywhere. If you want to give it a spin, visit the plugin Github repository and click on the “Clone or download” button. If you don’t know how to clone a Git respository just grab the zip file and install it on your server, overwriting the files in the plugins/wp-super-cache/ directory. If the changes to where cache files go doesn’t interest you, some of the changes in this list might:

WP Super Cache 1.4.9

There’s a new release of WP Super Cache out and it’s a security release to fix XSS problems in the settings pages. Those pages are only accessible by admin users so an anonymous visitor to your site can’t come along and enable it to steal your login cookies but along with those fixes come many bug fixes so it’s worth upgrading if you’re using an old version.

From the Changelog:

  • Fixed bug when not running sem_remove after sem_release. See https://github.com/Automattic/wp-super-cache/issues/85
  • Fixed a PHP error impacting PHP 7.1.
  • Fixed a bug where we cached PUT and DELETE requests. We’re treating them like POST requests now.
  • Delete supercache cache files, even when supercache is disabled, because mod_rewrite rules might still be active.
  • Updated the settings page, moving things around. #173
  • Make file locking less attractive on the settings page and fixed the WPSC_DISABLE_LOCKING constant so it really disables file locking even if the user has enabled it already.
  • Added a WPSC_REMOVE_SEMAPHORE constant that must be defined if sem_remove() is to be used as it may cause problems. #174
  • Added a “wpsc_delete_related_pages_on_edit” filter that on returning 0 will disable deletion of pages outside of page being edited. #175
  • Fixed plugin deleting all cached pages when a site had a static homepage. #175
  • Make sure $cache_path has a trailing slash #177
  • Remove flush() #127 but also check if headers are empty and flush and get headers again. #179
  • Add fix for customizer #161 and don’t cache PUT AND DELETE requests #178
  • Check for superglobals before using them. #131

You can click through to each of the Github pull requests above to see discussion around each bug fix.

If you’re hosting many sites that use WP Super Cache and you’re seeing issues with semaphores it may mean that your users are using file locking. It’s really not needed and in #174 there’s a fix that went into this release. You can disable file locking completely by setting the constant “WPSC_DISABLE_LOCKING” in a global configuration file. The file locking simply slowed down how fast cache files were created and is a hold-over from WP Cache when that plugin used to write directly to the cache files. This plugin writes to temporary files before moving to the final cache files so that locking isn’t really needed, but some sites still use it which is why it’s still around.

I’ve already been working on the next release with efforts to move the legacy cache files into the supercache directories. This will make it easier to maintain them and improve performance. We really need to find a better name for this caching method however. It caches everything – page contents and http headers so it’s quite useful!
If you’re going to test that PR, try #176 too. The plugin only deletes index.html type files right now but this chunk of code cleans up various for loops in the plugin and also deletes any file in the named directory. There are some restrictions on it so it won’t delete anything outside the cache directory.

Thanks to everyone who contributed to this release!

WordPress is thirteen!

You could have knocked me over with a feather today when I read Matt’s post announcing that WordPress was celebrating a birthday!

It didn’t seem so long ago that we were working on b2++, hacking the multiuser bits in and doing all sorts of crazy things with it.

Now I’m “typing” this on a mobile phone by swiping my finger across a virtual keyboard. Back then the closest to this that I could imagine would be some sort of SMS integration!

WordPress today is unrecognisable from what it was back then, especially if you use the slick Calypso interface.

I’m looking forward to seeing what the next few years bring.

The Web Won’t Forget Alex King

If you use a WordPress site, either as a visitor or owner, you’re using code that Alex King, one of the original developers of WordPress, worked on.

He passed away after fighting cancer for 2 years but his online presence lives on in the form of his blog with it’s deep archive of posts going back years, and in so much code that it’s humbling to look at his projects page. Looking through the svn log of WordPress trunk shows he still had a hand in helping the WordPress project until relatively recently:

trunk$ svn log|grep alexkingorg
props alexkingorg for the initial, long-suffering patch.
props alexkingorg. fixes #24162.
Props alexkingorg
`wp.media` instead of just `media`. props alexkingorg, see #22676.
Add $post_ID context to the pre_ping filter. props alexkingorg, devesine. fixes #18506.
Add filter so the users can select custom image sizes added by themes and plugins, props alexkingorg, fixes #18520
esc_textarea() and application for obvious textarea escaping. props alexkingorg. fixes #15454
Escape links by default. Props alexkingorg. see #13051
Safely include class-json.php, class-simplepie.php and class-snoopy.php, props alexkingorg, fixes #11827
Fix user creation from admin after changes for #10751. Fixes #10811 props alexkingorg.
Hooks needed to allow alternate category admin inteface. Props alexkingorg. fixes #3408
Wrap cat name in CDATA. props alexkingorg. fixes #3252

I’m sorry I never met Alex, however I remember working virtually with him and Adam Tow on AllThingsD which seems like a lifetime away now. Adam has a great article on Alex on his blog, as does Matt who went into detail about Alex’s involvement with WordPress going back to the days of b2. I had completely forgotten the CSS competition he mentioned!

Alex, your legacy lives on.

WP Super Cache 1.4.5

WP Super Cache is a fast caching plugin for WordPress. It will help your site run faster and serve more traffic.

This is a security and bugfix release.

  • Some servers display a directory index when no index.html is found in a directory. That may reveal the filenames of cache files.
  • There were issues in the settings page that might allow an attacker to browse or delete files named index.html.
  • PHP Object Injection could occur if an attacker managed to inject malicious code into the legacy cache meta files.

When you upgrade, your “legacy cache” files for logged in users will be deleted. This may have an impact on your site:

  • If your site is slow at generating new pages.
  • If you have many known users (logged in users or people who comment).

Your site will suddenly have to generate new cache files for all visiting known users.

Relying on caching like this is not recommended for these types of users as it’s very inefficient. Each user has a separate cache file that must be checked whenever the plugin does administration work like cleaning up stale cache files.

If most of your traffic is anonymous users who don’t comment you don’t need to worry about this.

Directory Listings

If a server is configured to show directory listings it will show files and directories in the cache directory to visitors who access those directories directly through their browser. This might reveal private posts, and in the case where legacy caching is enabled for known users the login cookie was stored in “.meta” files that could be downloaded.

2013

Files named “index.html” were added to the main cache directories to stop remote users viewing the contents of the cache directories. Unfortunately it’s not possible to add empty index.html files to the supercache directories because those files could be served by accident to legitimate visitors of the site. However, the plugin will also add a directive that disables directory listings to the file cache/.htaccess. You can now also change the location of the cache directory on the Advanced Settings page of the plugin. If you can’t disable directory indexing on your server and you have private posts you should change this location and use PHP mode to serve cache files.

cache-location

If a directory index is found in the cache directory it will show a warning like this to administrators:

index.html warnings

Clicking the logout link will log everyone out, except the user who clicks it, but it guarantees that the login cookies are updated, just in case someone has copied the cookie from an old meta file.

Directory Traversal and File Deletion

User input in the settings page wasn’t properly sanitised. The code that sanitised directory paths when deleting cache files wasn’t secure and might allow an attacker to view or delete files named index.html. Deletes are protected by a nonce, limiting the useful lifetime of the URL however.

PHP Object Injection

The plugin used serialize and unserialize to store data in “legacy cache” meta files. This might be used to perform a PHP object injection attack. Serialised data is now stored as JSON data.

The format of legacy cached files has changed. The files in the meta directory no longer have a .meta extension. They are .php files now and each file has a “die()” command to stop anyone loading them.
The data stored in those files is now stored as JSON serialised data. The login cookie is an MD5 hash now as well.
When you upgrade the plugin your existing legacy cache files will be deleted and regenerated as visitors use your site.

Apart from those security fixes there have been a number of enhancements and bugfixes:

  • Disabling the plugin no longer deletes the configuration file. Uninstalling will do that however.
  • Enhancement: Only preload public post types. Props webaware.
  • It’s now possible to deactivate the plugin without visiting the settings page.
  • Fixed the cache rebuild system. Rebuild files were deleted immediately but now survive up to 10 seconds longer than the request that generate them.
  • Minor optimisations: prune_super_cache() exits immediately if the file doesn’t exist.
  • The output of wp_cache_get_cookies_values() is now cached per visit.
  • Added PHP pid to the debug log to aid debugging.
  • Various small bug fixes.
  • Fixed reset of expiry time and GC settings when updating advanced settings.
  • Removed CacheMeta class to avoid APC errors. It’s not used any more.
  • Fixed reset of advanced settings when using “easy” settings page.

This release wouldn’t be possible without the help of Brandon Kraft, Dane Odekirk, Ben Bidner, Jouko Pynnönen and Scrutinizer. Thank you all!