See that nice dip in the graph for this week? I started to preload the cache used by WP Super Cache last Sunday and it’s made a noticeable difference in the load on my server here. The big spike is the preloading process.
I’ve always discouraged users from preloading the cache (Askapache Crazy Cache will do this for any cache plugin), mainly because of the possible problems so many files will cause for hosting companies. If you have thousands of cache files, it’s going to take so much longer to recover from a disk crash.
On the other hand, Google will now be using speed as a metric for judging how “good” a website is. In the past this plugin ignored the pages visited by bots because the bots only visited each page once so caching a page after the fact was pointless. The page, all pages, have to be cached first before Google ever visits.
That’s what it looks like. Once you start preloading it launches a wp-cron job to fetch 100 posts, then schedules another job 10 seconds in the future to fetch another 100 posts until it finishes. It also disables garbage collection of old pages, but making comments or posts will still clear out the appropriate cached files.
It only caches single posts right now. It may not be worth caching archive or tag pages because many sites already tell bots to ignore those pages as the server is doing less work it will serve those archive pages more quickly anyway.
The preloading only works if you’re using the plugin in Supercache or “ON” mode. It’s still a work in progress but has worked fine here. As well as the preloader the development version of the plugin has:
- Better support for mobile plugins.
- A cache tester.
- Can be configured to only delete the page a comment is left on, rather than the front page and associated pages.
- Works in WordPress 3.0.
It also has a number of bug fixes and other features added too.
I need testers though, so grab the development version from the download page. Install it and please leave feedback here or preferably on the support forum.
I’d like to add to my previous comment that my disk has 16 GB free. Maby I had too many files in the cache directory? I’ve read on another place that “One of the things to watch for is that you can’t put too many files, more than a few hundreds, in the same directory on Unix or performance will suffer.”.
Do anyone else have problem with too many files? Maby the cache directory structure should be by using the first four characters in the filename? So that “12345.html” would go into /1/2/3/4/12345.html
Just an idea…
But why did I get the message that a post was cached when I was logged in, although it wasn’t (I manually checked the cache directory)?
Donncha, this feature sounds like just what I’ve been looking for – thanks! Would you consider adding support for tag, category and archive pages? I rely on those being fast (and not just as fast as WP can serve them with low server load).
I finally got around to doing more work on the plugin. Grab the latest dev version if you’re using a static front page. When you publish a post it won’t clear out the whole cache now!
+1 Kena’s request. I dunno about tag pages but I’d really like to see category pages cached, at least at the top level if the categories are nested.
If it’s too complex and stops you adding other cool stuff then it’s not a deal-breaker. If the category pages are that popular then the 1st one there can take the hit.
Thx. I missed that Preload-Feature a lot. Now I dont need to use a Link-Checker any more to get all my pages cached. 🙂
It is probably me, but I’m having a problem getting WP Super Cache Preload Cache to work. Every time I click ON and then Update, it disables Preload. Then, if I enable Preload and set the time to 30 minutes, and hit Update Settings, Preload appears to switch ON to HALF ON. Hence, only 3 pages have been cached (and I’m surprised and don’t know how that many got through). Note that I’m using WP 3.0.
When I clicked preload the first time it got stuck at caching posts 100-200 and only appeared to have cached 40ish pages. I am using wp as a cms and using pages for everthing. So I clicked it again and now it just reloads the page and seems to do nothing. Any ideas?
If you’re having problems try the development version. It’s on the “Other Versions” page on the WordPress.org homepage for the plugin. It may work better for you as I’ve made a number of improvements to it.
Ok I think I have a better idea of what is going on now. I came back the next day and it had preloaded something like 80 pages. So it was just taking a long time. My page load with out cache are over 30 seconds so I am guessing that wp super cache has to load the page in some way to create the html version. This being the case it probably does take over a minute per page.
So I let it run and it made it to 94 pages preloaded. It seemed like it was going alright but I eventually got an email saying that it may have stalled. When I went back to check on it there were no pages “super cached”. I have not installed the development version as yet since I don’t think my original diagnosis was correct. Any thoughts on what seems to be going on now? Should I just go ahead with the development version and see if that helps?
Thanks
Looks like it’s not preloading. Try the development version, it might help you.
Hi,
where exactly does it store preloaded files? Is it the same as “directly cached files”?
I have the same problem as Kris. I might try the development version now.
Also, I’m still getting this error when visitors come from Google Images:
PHP Fatal error: Call to a member function get() on a non-object in /wp-includes/cache.php on line 93
I really don’t understand why it does serve a dynamic page to the Googlebot when I used the webmaster tools to check how Google sees my page.
Anyway, great work, appreciate this plugin a lot.
sOliver – wp-content/cache/supercache/ is where they go. Not the same as those other files.
That get() function is not a Supercache problem, that’s another plugin trying to use the WordPress object cache after the object cache has been destroyed by PHP.
If Googlebot sees a dynamic page use the debug system in the plugin to figure out what it’s doing.
You said: “That get() function is not a Supercache problem, that’s another plugin trying to use the WordPress object cache after the object cache has been destroyed by PHP.”.
Are you absolutely certain of this? Can you explain why the error message describes your cache file on a line that actually has a get() function rather than the file of the other plugin that is using the WP object cache improperly?
What other plugin?
I am getting this error regularly and I have only a few plugins installed.
Yes, a recent version of Supercache (after this post was written) accidentally used get() in the PHP shutdown process when it used get_option() but that’s not the case in the current release. I think the problem may sometimes occur in WordPress core code too. There’s a trac ticket on the subject but I don’t have a url for it.
hello,
i like the preload function, however with qtranslate it doesn’t preload the languages! any idea to to fix?
Not really, you’d have to hack the plugin to change the language headers in the request sent to itself.
hi there,
thanks for the amazing work, i notice a great performance increase on my blog with more than 68.000 posts.
i have everything preloaded 😉 this took about a week or so and i’m happy with the result.
only thing is, i have a little update to be made in the footer of every page and i wonder what the behavior will be of “Refresh preloaded cache files every * minutes”.
will it grab 100 pages a time to be re-cached leaving the cache in place, or will it clear the complete cache in once and then starts caching the pages?
i have quite a few broken links on these pages and use “broken links remover”, this combined with the huge number of posts makes my page load very slow with the preloaded cache so hence my question.
thanks,
Randall
Randall – it leaves the cache in place and only deletes the cached page being requested so it can be cached again.
thanks for the fast reply.
so, if i set,
“Refresh preloaded cache files every 720 minutes”
it will re-cache a page if a page is
A) older then 720 minutes
and
B) the page is visited by either a visitor or a bot
question: doesn’t that make the requested pages slow?
No, it will do a full refresh of all preloaded files every 720 minutes. The refresh is done by a WP Cron job so it doesn’t rely on a page being requested by a visitor.
ok, clear
will give it a try and see how it works
thanks
was thinking,
if i set it to refresh every 720 minutes it shows it will start refreshing in 12 hours.
because i have 68.000 pages the refresh will take about a week so i’m guessing this could lead me to a problem, not sure if the second cron job will start again if the first is still running.
would it be beter to set it to a 2 week schedule to be safe (this would mean tough i would have to wait 2 weeks before the first refreshing starts.)
is it possible to initiate a single refresh?
sorry for all the questions, normally i don’t mind to just try out all the options but the shear number of pages in this case makes me a little nervous 😉 since i have been caching for the last month already (i keep changing things)
Just to let you know,
the pre-load schedule behaved perfectly.
i had 68000 posts pre-loaded and wanted to have the pre-loaded cache to be refreshed, was worried that setting the refresh every 720 minutes would give troubles since the caching of that many pages would take longer than those 720 minutes.
no problem whatsoever, the re-preloading took several days and the 720 minute schedule only initiated again after the complete run was finished.
Thanks again for the great plugin
You said: “Can be configured to only delete the page a comment is left on, rather than the front page and associated pages.”
How?
Check “Only refresh current page when comments made.” on the Advanced Settings page.
Please reword the check box label from:
“Only refresh current page when comments made”,
which means:
“Do not refresh current page until a comment is made”
to:
“Refresh only the current page when a comment is made”,
which means:
“when a comment is made do not refresh any page(s) except (other than) the current one”.
Lee – thanks for the great suggestions!
Request: Please change the description of the preload interval. Do not say “every N minutes”. Instead say “N minutes after the completion of the preceding reload”
Hi Donncha ,
Thank you for the very useful caching plugin. I have an issue right now. I have enabled preload mode in my blog (so it disables the cache expiration/garbage collection thing). I have confirmed in the /wp-content/cache directory. It did create those superstatic HTML files for all my blog posts.
I have set preload to refresh every 2880 minutes. But my problem is that after this period, some of my blog posts does not have an updated sidebar?
Take note that I have added posts daily to my blog however even after 2880 minutes, some of the blog posts did not have an updated sidebar (where it shows the most recent posts).
Is this an error or am I missing something. Please let me clarify how preloading works. As what I read, it is supposed to refresh the entire super-cache files?(all blog posts). Or only some of them?
I do not want to refresh manually by pressing “preload cache now” again. It seems it defeat the purpose of setting it automatically. Thank you for your clarification.
Turn off preload mode if you want supercache files to be updated when they expire. With the mode on they won’t be updated until preload happens again!
I am confused, would that mean that If I enable preload to refresh every 2880 minutes (with preload mode on), the preload cache would then be refreshed but not updated?
What is the difference between refreshed and updated in this case? Or the scenario below:
Preload mode on- refresh
vs.
Preload off- cache expired then updated
Do they have the different results? Sorry if for asking a very noob question, but I do not understand this either. So bear patience. Thanks.
refresh = update. When the preload process runs it updates all the supercache files and deletes expired ones. When the cache is refreshed it’s updated 🙂
Hi Donncha,
Good stuff – really save my server workload.
A couple of question regarding preload, in my site – the preload run very well if I initiate to “Preload Cache Now” manually. However, when I schedule the preload by 1 hour, it doesn’t work – since I can’t see a different in the cache result.
My question is :
1. How do we check if the cron job work / no ? Is there any menu or log that we can find ?
2. Is it possible to run a command to run this preload cache? Let’s say I will run a cron command to request a certain URL to initiate the preload cache activity.
Thanks a bunch for your response in advance.. 🙂
1. use the cronview plugin that I linked to from the settings page.
2. Yeah, the function wp_cron_preload_cache() in wp-cache.php does the preloading. You’ll have to run that.
I am very confused with this settings. I went trough all the coments to learn more but it somehow does not make sense to me:-( I have a small site with around 170pages and I upload new post only once a week. Do I need to do this?
Martina – no, there’s no need to preload if your site loads quickly enough.
Thanks for the post. For those who are experiencing preload stall issues please make sure your robot.txt is not set to block bots.