The Mastodon Onslaught on your blog

You might not be on Mastodon yet, but your blog could get a torrent of traffic from Mastodon, or other Fediverse network if it’s shared there.

If your website is mentioned there, it might be the “victim” of an inadvertent denial of service attack, as hundreds or thousands of servers request the URL in the 60 seconds or so afterwards. That is precisely what JWZ blogged about last month when his site was taken down by Mastodon servers.

Every time I do a new blog post, within a second I have over a thousand simultaneous hits of that URL on my web server from unique IPs. Load goes over 100, and mariadb stops responding.

JWZ on Mastodon Stampede.

JWZ has over 8,000 followers. Every time he shares a post on Mastodon, the instances (servers) where those followers live will send a request to his blog to generate a preview. Actually, two requests will be sent:

• A request for the wp-json embed for the page.
• A request for the page that was shared.

Eventually, he blocked the Mastodon user agent. That stops previews of his website showing up on Mastodon posts, but resolves the problem for his website.

Yesterday morning, I decided to see what effect sharing a link on my Mastodon account would have on my server. My Mastodon account has 1.8K followers. A far cry from the number of followers JWZ has, but still enough to test my server.

I wanted to test several scenarios:

• Caching the post before sharing.
• Changing Apache configuration.
• Sharing without caching on my server.

My server is at Linode. I pay an average of $24/month to run this site and my photoblog is on it too where I share a daily photo + link on Mastodon. It’s not a heavy-duty server that can withstand a huge amount of traffic.

If you’d like to skip the details, my server coped fine with sharing a URL from here to Mastodon. The load average went up for about 20 seconds, topping out at the max for about 5 seconds before things calmed down. It was responsive the whole time. Install a full-page caching plugin like WP Super Cache, Jetpack Boost and WP Rest Cache and your site will probably be fine. Jetpack Boost and the Jetpack Image Accelerator will help when human visitors arrive.

The first test resulted in:

• 261 requests for the page embed.
• 359 requests for the page itself.
• 1 minute load average topped out at 1.34 for 5 seconds.

The page was cached by WP Super Cache, but I had set the garbage collection TTL to 60 seconds and I believe it expired halfway through the test, so it had to generate the cache again. Once I adjusted that, and set the TTL to 600 seconds, the second test performed better. The page remained cached throughout:

• 273 requests for the page embed.
• 289 requests for the page itself.
• 1 minute load average topped out at 0.71 for 5 seconds.

The main points of my Apache configuration:

• Keep alives are disabled.
• 5 start servers
• Minimum 10 spare servers

When I reduced the start and minimum spare servers to 1, the next test took longer to complete, and the load average rose to 1.24, even on a fully cached page. This was expected as the server didn’t have the spare capacity to deal with the sudden traffic.

After reverting the changes to Apache, I disabled caching on my blog and shared another URL. The load average only rose to 1.12 for a very short time. I was pleased with that. While caching does help, my server could cope with that traffic.

I suspected that there was one hit per Mastodon instance on my site. I checked my logs and was proved right. For all the accounts that follow me on mastodon.social, only one request was made. That does mean the onslaught of requests isn’t as bad as it might be. Instead of 1,800 requests for a page, there were far fewer. I did notice that a Friendica instance requested one of my test URLs several times.

Mastodon and other Fediverse servers will start requesting a preview within a second of you sharing your post on the network. It helps if your server is running some sort of caching.

If you have many Mastodon followers or if you’re worried about a DDoS from Mastodon, the following will help:

• Make sure Apache/Nginx has the spare capacity to grow quickly and respond to a sudden torrent of requests.
• Install a caching plugin like WP Super Cache.
• Use “expert caching” in WP Super Cache which serves the cached page using mod_rewrite. That will mean your blog post is served almost as fast as requesting a text file from the server. No PHP is executed at all.
• Install WP Rest Cache as it will soon cache the embed page request.
• Install Jetpack and enable the Image Accelerator and Jetpack Boost for human visitors who come later.

This problem has existed for a long time. Popular blogs had the same issue when they published new content and people following their blogs (through RSS feed readers, remember them?) hit the server looking for the new post. At least with Mastodon, you can load the post in a private browser window and cache it before sharing it. I want to write a WP Super Cache add-on plugin that allows the site owner to preload a new post as it’s published. That will ensure the new content is ready for sharing. I haven’t started work on that yet, so don’t ask when it’ll be done. Maybe someone else will beat me to it and claim all the credit!