Why not just use WP Cache? You can use the latest comments plugin on any page but the comments will always be the same, given the same args to it.
When someone visits a page that WP Cache hasn’t cached the cached comments will help.
By default wp cache doesn’t cache every page either so the cached comments will reduce the load on those pages too.

It should be easy enough to invalidate the cache. Add an action on the comment publish hook and simply remove cache/latest_comments*. I didn’t add that because it’s not important to me.

You can change the time to live of the cache by searching the plugin for “3600” and change it to the number of seconds you want the cache to live for.