See that nice dip in the graph for this week? I started to preload the cache used by WP Super Cache last Sunday and it’s made a noticeable difference in the load on my server here. The big spike is the preloading process.
I’ve always discouraged users from preloading the cache (Askapache Crazy Cache will do this for any cache plugin), mainly because of the possible problems so many files will cause for hosting companies. If you have thousands of cache files, it’s going to take so much longer to recover from a disk crash.
On the other hand, Google will now be using speed as a metric for judging how “good” a website is. In the past this plugin ignored the pages visited by bots because the bots only visited each page once so caching a page after the fact was pointless. The page, all pages, have to be cached first before Google ever visits.
That’s what it looks like. Once you start preloading it launches a wp-cron job to fetch 100 posts, then schedules another job 10 seconds in the future to fetch another 100 posts until it finishes. It also disables garbage collection of old pages, but making comments or posts will still clear out the appropriate cached files.
It only caches single posts right now. It may not be worth caching archive or tag pages because many sites already tell bots to ignore those pages as the server is doing less work it will serve those archive pages more quickly anyway.
The preloading only works if you’re using the plugin in Supercache or “ON” mode. It’s still a work in progress but has worked fine here. As well as the preloader the development version of the plugin has:
- Better support for mobile plugins.
- A cache tester.
- Can be configured to only delete the page a comment is left on, rather than the front page and associated pages.
- Works in WordPress 3.0.
It also has a number of bug fixes and other features added too.
I need testers though, so grab the development version from the download page. Install it and please leave feedback here or preferably on the support forum.
I have WordPress 2.7.1.
Yes, update your version of WordPress. There’s no good reason to be using 2.7.1 when you can be hacked in 2 minutes. I need to update the minimum version requirements if it says that 2.7.1 is supported.
Hi Donncha. Ok. I have an big german site with 16.000 User per day and i have big big Problems with the Wp-chache Plugin. I make all right .htacces, installation and so on, the wp-files where delivered right but in php (X-Powered-By PHP/5.2.12-nmm1). You can take a look at
http://news.preisgenau.de/
sometimes it take 7 seconds to load a site. The other problem. When 20 sites are chached new at the same time, than the hell is going on. 🙂
sorry for my bad english.
but i can’t find the mistake, thats the problem.
i have big big Problems with the Wp-super-chache Plugin (right)
the preload function ist absolutly necessary
@Gordon: you need to install WordPress 2.9.2, and if you’re going to use the preload function make sure not to preload them all at the same time.
wordpress should be the last / latest version (final) of course.. and the apache rewrite rules from the super cache plugin should be the latest rules of the plugin 🙂
Hi Donncha, ok thanks, but why the wp-cache files delivered as php files (firebug -> X-Powered-By PHP/5.2.12-nmm1). Also your blog posts here at ocaoimh.ie are loaded with php (firebug -> X-Powered-By PHP/5.2.6-1+lenny3) . I don’t understand that.
when you compare for expamle this 2 site (both using wp-super-cache)
1. http://www.vip-chicks.de/2010/multimedia/wp-super-cache-wp-cache-preloader-und-wordpress-3-0.html
2. http://news.preisgenau.de/nr-1-apple-der-erfolgreichste-handy-hersteller-in-den-usa-7094.html
The first one delivered static html files the second on not. Why?
That’s because you’re a “known user” with a cookie set in your browser. Only anonymous users get static files. It’s all in the readme.txt 🙂
Hi Donncha, your are right. (hand going to the head) 🙂
Hi donncha,
When i delete the cookies the most files are delivered als static HTML Files, but for exempale this file is delivered with php
http://news.preisgenau.de/ab-100-euro-gunstige-lcd-fernseher-handys-notebooks-und-pcs-6934.html
When you check it (you are No logged in User), you will see that. Ist this a bug?
@Gordon: It shows up as a cached page for me (in the source, I can see it).
Hi Ipstenu,
ok. right. Yesterday firebug told me these site is delivered with php. Strange.
@Donncha
I have a few suggestions to make wp-super-cache much better. I would foward to hear your opinon
1. Start the Preload Function with a regular external cronjob (that would be very nice)
2. Input Box for searching a post (per Post_ID) -> to re-chache only the single file (when you have maybe 3000 chached files a list is unhandy)
3. Recache single tags and cats
4. The Preload Function should maybe include the caching of tags and cats (Tags and Cats are very important)
6. When a post is publisched the tag and the cat of the Post should be recached (i use wp-super-cache 0.9.9, think this version don’t have this function)
i think its a good idea to let the user choose if he wants to use wp-cron or a real cron made by server (?)
@Christian Kopp: @Gordon: you can already set up a cron job. Just have it call a script that does the same as wp_cron_preload_cache() in wp-cache.php without the scheduling bits.
@Donncha:
Hi Donncha,
you mean a script thad contents the following (beside the variable definations – maybe wp-load..)
function wp_cron_preload_cache_cron2() {
global $wpdb, $wp_cache_preload_posts;
if ( get_option( ‘preload_cache_stop’ ) ) {
delete_option( ‘preload_cache_stop’ );
return true;
}
$c = (int)get_option( ‘preload_cache_counter’ );
if ( $c == 0 && function_exists( ‘wp_cache_clear_cache’ ) )
wp_cache_clear_cache();
if ( $c get_col( “SELECT ID FROM {$wpdb->posts} WHERE post_status = ‘publish’ ORDER BY post_date DESC LIMIT $c, 100” );
} else {
$posts = false;
}
update_option( ‘preload_cache_counter’, ($c + 100) );
if ( $posts ) {
$count = $c + 1;
foreach( $posts as $post_id ) {
$url = get_permalink( $post_id );
wp_remote_get( $url, array(‘timeout’ => 60, ‘blocking’ => true ) );
$count++;
}
} else {
update_option( ‘preload_cache_counter’, 0 );
}
}
wp_cron_preload_cache_cron2();
Looks like saving a new post as draft wipes out the cache. 😕
That doesn’t happen for me. I just tried it here and all 5890 cached pages are still cached. Do you have the “clear all cache files when post is published” checkbox enabled?
@Donncha: I do not have “clear all cache files when post is published” checked.
I DO have ” Only refresh current page when comments made.”
I had 36 pages cached. Went in and edited a post in draft, and wham. None. I’m using your dev version from … looks like May 3rd at 3pm US Central Time.
@Donncha: In case it matters, I’m running 3.0 Beta 2 on Multi-Site (though this happened on the nightly from 2 days ago).
Interesting feature. What I’ve done on highly trafficked sites is to regenerate an anonymous version of the latest X posts once a minute. Once posts are no longer within X, they fall back to the normal caching layer. The anonymous version is served to anonymous users via nginx, which sits in front of Apache.
@Ipstenu: The “clear all cache files” trumps any other options I’m afraid. It shouldn’t have cleared the cache when a post wasn’t published, I think. I’ll check that.
Mark – you’ll love this then, but there shouldn’t be any need to refresh every minute as they’ll be refreshed when comments are made. I just moved this site onto the same VPS that inphotos.org is on, on a new install of Ubuntu and once I tamed Postfix (Stop smtp! Down boy!) the load average has plummeted, even with Apache.
@Ipstenu: Ooops, I read your comment wrong. Sorry. Can you enable debugging in the plugin and check that when you save a draft post?
Donncha: Same problem as Ipstenu here, so I disabled the plugin for now =)
(I don’t have time to do the debugging though so I leave that to Ipstenu).
Just so you know it’s not a isolated occurrence (I’m on WPMU 2.9.2 myself).
@Daniel: It’d only take you a minute to activate the debug system, limit it to your own IP and save a new post but what it reports will help me fix the problem. Please do debug it if you have a spare few minutes!
I added a small check to the post change function so cached files aren’t deleted if the post is a draft one. I still don’t know how all the cache files were deleted for you but at least this should stop it. Previewing a post is not cached so there’s no point deleting any cache files.
Okay, tested on my 3.0 Single Site, no problems. Ran it on MU and blam. Well, there’s something. If this doesn’t show up right, let me know where I can email you. I edited out my cookie info 🙂
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_test_cookie
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_xxxxxxxx
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_logged_in_xxxxxxx
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 supercache dir: /home/jorjafox/public_html/wp-content/cache/supercache/jorjafox.net/wp-admin/post.php/
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_test_cookie
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_d41d8cd98f00b204e9800998ecf8427e
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_logged_in_d41d8cd98f00b204e9800998ecf8427e
13:17:13 /wp-admin/post.php?post=2389&action=edit&message=10 No wp-cache file exists. Must generate a new one.
13:17:14 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_test_cookie
13:17:14 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_xxxxxxx
13:17:14 /wp-admin/post.php?post=2389&action=edit&message=10 Cookie detected: wordpress_logged_in_xxxxxxx
13:17:14 /wp-admin/post.php?post=2389&action=edit&message=10 In WP Cache Phase 2
13:17:14 /wp-admin/post.php?post=2389&action=edit&message=10 Setting up WordPress actions
13:17:14 /wp-admin/post.php?post=2389&action=edit&message=10 Not caching wp-admin requests.
Upgraded to your modified version, and it appears to be the same error. Only on Multi-site, though.
@Ipstenu: Odd, none of the debug messages above mention deleting the cache at all!
Okay, this is weird. It keeps wiping out my log (I’m saving to a text file) every time I perform some admin action like deleting a spam comment, or edit a post. The cache, though, ONLY gets whacked when I edit a post.
So I do the logical thing and wonder ‘Holy Shmoly! Has this been happening the whole time?’ Roll back to 0.9.9 and guess what? The log gets whacked but the cache stays put.
Back to the dev version, but now I UNCHECK “Only refresh current page when comments made.” Doesn’t help.
Bugger. I’m back on .9.9 because this cache killing on MU is killing me! Argh.
Ok. Preloaded 100 posts (updated the settings after 100 posts which cancels the timer to the next the preloading).
Removed a comment which I before had put as “unmoderated” (on a page if that matters). Cache emptied:
11:36:58 /wp-admin/admin-ajax.php Post 11 changed. Update cache.
11:36:58 /wp-admin/admin-ajax.php Post change: deleting cache files in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/
11:36:58 /wp-admin/admin-ajax.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//om-daniel/.
11:36:58 /wp-admin/admin-ajax.php rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//om-daniel/feedback/index.html
11:36:58 /wp-admin/admin-ajax.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
11:36:58 /wp-admin/admin-ajax.php Post change: deleting page_on_front and_page_for_posts pages.
11:36:58 /wp-admin/admin-ajax.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//.
11:36:58 /wp-admin/admin-ajax.php rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/blogg/2010/04/02/caolings-historiska-vandringsled/index.html
And so on for the rest of the blog posts.
I also made the same thing, but edited a draft for a post:
11:42:49 /wp-admin/post.php Cookie detected: wordpress_
11:42:49 /wp-admin/post.php Cookie detected: wordpress_logged_in_
11:42:49 /wp-admin/post.php In WP Cache Phase 2
11:42:49 /wp-admin/post.php Setting up WordPress actions
11:42:49 /wp-admin/post.php Not caching wp-admin requests.
11:42:49 /wp-admin/post.php Post change: deleting cache files in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//?p=1292/.
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
11:42:49 /wp-admin/post.php Post change: deleting page_on_front and_page_for_posts pages.
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//.
11:42:49 /wp-admin/post.php rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/om-daniel/feedback/index.html
11:42:49 /wp-admin/post.php rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/blogg/2010/04/02/caolings-historiska-vandringsled/index.html
Same thing once again.
Or, well. None of my cases isn’t really good I guess.
(One is moderating a post and the other one editing a draft…).
But it will have to do for now =P
But looking into your code the
add_action(‘edit_post’, ‘wp_cache_post_change’, 0)
line do look a little odd… shouldn’t it be wp_cache_post_edit?
And I also do not understand why doing anything with a comment than posting it should clear the whole cache?
if ( [some code] && strpos( $_SERVER[ ‘REQUEST_URI’ ], ‘wp-comments-post.php’ ) ) {
$all = false;
} else {
$all = true;
}
A little more looking into the code seems like that when a post is updated a two calls to the wp_cache_post_id_gc function is made. This call I guess is supposed to delete frontpage and the blog page. (static homepage option).
What this function does is that it calls the prune function on homepage and blogpage. The problem is that the prune function seem to clear all subfolders.
Correct me if I’m wrong in anything above but to me it seems like this is the case… a hint of this is given by:
11:42:49 /wp-admin/post.php Post change: deleting page_on_front and_page_for_posts pages.
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//.
11:42:49 /wp-admin/post.php rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/om-daniel/feedback/index.html <– The page I removed a comment from
[deleting every post/page in the cache]
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/.
11:42:49 /wp-admin/post.php wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
@Daniel: I noticed something odd immediately – “supercache/www.soilheart.com//?p=1292/” but your blog does have permalinks. Did you save the draft post before you added a post title?
Daniel – Try the dev version in about 15 minutes. I made a change so only the “page for posts” is deleted.
As I said, the test was done with a draft, so that’s why the number…
I don’t think wordpress reserves a name before it gets published?
Ok, will do.
That seem to be the main problem right now at least. (We’ll see later 😉 )
CASE 1: Draft editing and saving:
19:55:08 /wp-admin/post.php wp_cache_post_change: draft post, not deleting any cache files.Nothing deleted, so far so good.
CASE 2: Editing published post:
Clearing cache for post 1282 on post edit.
Post change: deleting cache files in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/2010/05/07/sma-ting/.
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/2010/05/07/sma-ting/index.html
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
Post change: deleting page_on_front and_page_for_posts pages.
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//.
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/om-daniel/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/foton/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/blogg/2010/04/10/har-last-pluggar-kommer-studera/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/blogg/2010/05/04/alltid-ar-det-da-nagot/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/utbytesaret/index.html
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
Still deletes the whole cache (in this case 5 more pages).
CASE 3: Adding a comment:
wp_cache_post_change: comment detected. only deleting post page.wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/2010/05/07/sma-ting/.
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/2010/05/07/sma-ting/index.html
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
Removes the cached copy of that post. Works as intended.
CASE 4: Move the comment to the trash:
Moderated comment deleted or spammed. Don't delete any cache files.Post change: deleting cache files in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/2010/05/07/sma-ting/.
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//blogg/2010/05/07/sma-ting/index.html
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
Post change: deleting page_on_front and_page_for_posts pages.
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com//.
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/om-daniel/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/foton/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/blogg/2010/04/10/har-last-pluggar-kommer-studera/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/blogg/2010/05/04/alltid-ar-det-da-nagot/index.html
rebuild_or_gc: deleted /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/utbytesaret/index.html
wp_cache_post_id_gc clearing cache in /home/soilhear/public_html/wp-content/cache/supercache/www.soilheart.com/page/.
Deletes the whole cache once again.
On the whole: Works better, still deletes whole cache on comment or published posts editing, which in my opinion it should not do.
And actually the log states it only deletes the “page on front” (with all subfolders, i.e. the whole cache) and not the “page for posts”.
If I do a quick edit of posts, I don’t lose the cache, but if I do a FULL edit and save, I’m still loosing the cache. Ugh.
hi Donncha,
i upgraded WP to 2.9.2 and installed the new wp-super-cached plugin. All is working fine, but i can not set a Refresh preloaded time. When i put in for example “2” and press “Update Settings” my input is resetet to 0. Other inputs like setting up the Expired Time are working. I checked all pressmissions on the server. Seems all looking fine. The sites are cached correctly. Do you have an idea?
Greets from Germany.
ok, find the mistake, the intervall must be “minimum 30 minutes”
@Donncha: I’ve build a plugin that removes the front page from the cache every 15 minutes. Now I need to rebuild it as soon as it has been deleted. Is it possible to use a function in your plugin to acchieve that?
Niska – no, just fetch the front page using wget, curl, fsockopen, wp_remote_get() or whatever suits your script. The plugin will recreate the supercache page.
I have a question:
When i have “the preloaded mode” on with a time range of for exmample 30 min, did that mean that
a) All chached files are recached every 30 min? or
b) all files which are not chached (this last cron), because they are new, are chached every 30min?
Thanks for answer
All the preload files (single posts) are refreshed every 30 minutes in your example. The files are deleted and recreated. During the refresh it will now delete the cache of the single post before creating it again. All the cache isn’t deleted in one go.
This morning I added code to clear out supercache files older than the refresh time after the cache is refreshed. The only files this should affect are tags and archive pages created by visitors to your site.
Hi Donncha,
thanks for this important information.
Think i find a bug. When i change a chached posts or created a new posts, the plugin delete all chached Tags und Cat Pages. Think this is a bug, because the plugin should only delete the tags and cat pages which are assosiated with the post. The chached single posts Pages are fine.
Can anyone confirm this?
Is that in half-on mode or full?
BTW – please stop changing your email on every comment, I’m hardly going to spam you! 🙂
I’ve downloaded the latest dev version from yesterday (latest entry in changelog is 2010-02-08 14:24). I’ve got “WP Cache and Super Cache enabled = ON” and “Super Cache Compression = ON”. ALL other settings are unchecked. Expire time is set to 0 seconds (never expire).
Current Cache Contents:
WP-Cache (609.46KB) 147 Cached Pages 0 Expired Pages
WP-Super-Cache (655.89MB) 32766 Cached Pages 0 Expired Pages
My problem is that everyting seems to be working fine when logged in. I’m reloading a post several times. Everytime footer reads
Page loads very fast. It seems to be cached!
Now. I log out. I go to the same page, and get this error in footer.
I’ve checked the source code of wp-cache-phase2.php, and it seems as if line 388 @wp_mkdir_p( $dir ); is the source of the problem. It returns false, wich means that the directory wasn’t created.
I have two questions about this:
1) When I’m logged in I get the feeling that the cache is working. However, when manually checking the existanse of the cache using FTP, it isn’t found in the “wp-content/cache/supercache” directory. Why is that?
2) How come the directory wont be created? Could it be that there is too many files in the directory? I currently have 32766 files…
The html comments are not visible in comment above. I try again:
When logged in I get:
Dynamic page generated in 2.137 seconds.
Cached page generated by WP-Super-Cache on 2010-05-11 11:05:55
Compression = gzip
When logged out I get
Dynamic page generated in 2.014 seconds.
File not cached! Super Cache Couldn’t write to: wp-content/cache/supercache/www.domain.com/my-post.html/2599829494be928f16172d0.14739459.tmp
I’ve now removed the cache completely. And now I get cached pages when logged out. But I’m still confused.
Was there too many files in the supercache directory?
Why did I get the message that a post was cached when I was logged in, although it wasn’t?
Hi Donncha,
@Is that in half-on mode or full?
I run WordPress 2.9.2 normal version and using the full mode, with super-cache compression
@Donncha: I’m testing the dev version today again. The problem (described above) seems to affect only cat pages and not tags pages.
-> I gave a cached posts with the quick edit function a new tag. Than i checked out the footer comments on differnt posts, cats and tags. All tags stayed in chache, but all cat pages where rechached after the short editing of the post. The post is only assoziated with one cat.
@Niska: are you sure, when you set expired time o 0 seconds that means never expire?
Yes. 0 = posts should never expire.
Had a question about the rewrite rules for mobile support. Link to mu forums