Performance of Open RSS feeds impacted after DDoS attack
Resolved
-
Server fell offline and all feeds were unresponsive.
-
After investigating, the server kept getting knocked offline, due to a huge number of requests made from different IP addresses all at the same time. Most requests were to RSS feeds. But some attempts were to pages that didn't exist, in what looked like a search for an admin login panel. Jokes on them, though—we don't have one!
The IP addresses shown in logs didn't resolve to any particular host and not much information could be determined about the source.
We've resolved the server issues and all feeds are back online. However, we're working on a huge performance improvement that we've been putting off that will prevent attacks like this from happening again. We'll update this issue as we're working...
-
Most of our work to resolve this is complete. The fixes we've implemented so far has got our servers pretty healthy and resilient against any potential attacks in the future.
We're seeing good performance gains in RSS feeds and on the website overall. We need another day or two to finish up and run some more tests, but we should be done very soon!
Also, we're aware that some RSS feeds are still problematic. We'll be tackling those as soon as we close out this issue. Thanks for your patience while we work through all this. Having your RSS feeds back up and running is our top priority.
-
We're pretty much done with the new implementation of all feeds and performed quite a bit of testing.
Fixed a lot of the broken feeds. There's a small amount of less-active feeds that are still delayed in showing new content. We've identified them and making some minor changes to resolve those. Aiming to wrap this all up by tomorrow.
Other than delays (which we're addressing), if there's a feed that isn't working for you or has some other problem that you want us to prioritize, you already know what to do 🤗.
-
Resolved the delays for all feeds and service has been running smoothly over the last few days. Testing is complete. So closing this issue out and removed the site-wide DDoS announcement banner from all feed preview pages.
Please note that any outstanding issues with feeds are likely unrelated to this issue and are already on our radar. We'll get back to working through them and include any fixes in our Changelog as we do normally.
We can't thank you all enough. The amount of support we've received getting past this stage in our organization has been overwhelming. We appreciate your patience in getting through this!