For those managing servers: what kind of concurrent request count spikes do y’all prepare for?
Looks like the Locust load testing tool’s “first test” starts with 100 concurrent users: https://docs.locust.io/en/stable/quickstart.html
9 replies
Found a pretty hefty blog post about concurrency benchmarks using the Nginx web server.
https://blog.nginx.org/blog/testing-the-performance-of-nginx-and-nginx-plus-web-servers
Looks like the slowest metric (1 CPU, HTTPS) handled 428 connections per second.
@zachleat Yeah, even if all ~30,000 mastodon instances hit the server within 60 seconds (unlikely), that's still only 500 requests per second. That seems pretty manageable? At least for public content that's identical for every user. A real DDoS by an attacker can be *millions* of requests per second.
@williamoconnell yeah, that third blog post notes a hypothetical of 6.7k simultaneous requests—which seems like a lot!
@zachleat It says they're supposed to be spread over 60 seconds though. In practice probably more, since it should take some time for all the instances to pick up the post in the first place. The author says "I'm yet to see that work for me" but I'm unclear of that means they have evidence that it isn't happening or their site is just still crashing regardless. The lack of specific data makes it hard to draw conclusions.
@williamoconnell I’d assume that is just a timing issue between writing of the blog post and the rollout of the 60s “jitter” addition—a temporary mitigation for sure.
Mastodon *could* upload the preview as an asset sidecar’d with the post, similar to any image upload.
@zachleat I would think with session reuse you wouldn't pay that TLS connection cost constantly. Also I imagine OpenSSL/BoringSSL has improved in the last seven years, and more cryptographic operations are done in CPU now then back then.
That said I don't think I'd run anything without a caching server in front anymore, unless it was very server interactive and would miss constantly. Varnish did amazing things at Flywheel.
@jmhobbs great insight, thank you!
As a side note I think I’ve seen some of these caching defaults play out at the serverless platform level recently: https://www.zachleat.com/web/serverless-cost/ Some platforms are born from use cases that are intensely personalized, not heavily cached.
@zachleat I've never been lucky enough to have something that was this popular so I can't really speak from experience, but I'm a little confused by why it's causing problems for so many sites. I would think any CDN should be able to solve this, even a really short TTL like 30 seconds would reduce the flood to just a few requests.
Usually when testing an application I'm happy if it can handle 100-200 requests per second, but for public content I would always have a CDN in front.
