Websites blocking abusive AI bots are blocking RSS users too

Website owners have been rightfully concerned with the abusive behavior of aggressive automation and malicious AI bots that can compromise their server and make their website inoperable for other real, human users.

To prevent this abusive activity, website owners are looking to security tools and services like Cloudflare, Akamai, Amazon CloudFront, and many others to block non-human activity from accessing their website entirely. But these tools don't just block malicious bots. They end up inadvertently blocking RSS users from accessing their website as well.

When a website owner enables a security protection feature on a website, it becomes the gatekeeper of all website traffic. This means that, instead of navigating to the website directly, every visitor gets routed through a bot detection flow first.

If the flow detects that the website visit is by anything other than a real human, it goes into "bot fight" mode and begins showing prompts that a human must perform to bypass. But, of course, an RSS reader can't enter any details to verify that it's a human, because it isn't one. So it gets blocked, and the RSS reader user can't read or view any of the website's content.

The RSS protocol was created to allow internet users to access website content through an RSS reader—an app that the user controls—instead of accessing the content directly on the website itself. Users prefer using RSS readers over going directly to a website for updates for many reasons.

1. They may have a disability or level of blindness that makes the website difficult to use
2. They have special needs that require them to use assisted technology (like screen readers) that may not work when using the website
3. They don't want to create an account and give up personal details or private information that can be compromised if the website is hacked
4. They are concerned about their mental health and want to see website content chronologically and in its natural form, instead of being manipulated by algorithms, biased curation, or so-called "recommended" content

When websites take measures that block malicious bots from their content, they're also blocking RSS readers from accessing the content. This, in effect, blocks the users who rely on RSS readers as well.

The internet was created more than 50 years ago for scientists and researchers to share data and information with one another. Today, we rely on the internet practically for everything in our lives because of that very same reason: viewing and sharing information. But when website owners take measures that block RSS readers and good bots, they're preventing the sharing of this information by making their content inaccessible to users who rely on those tools.

In addition to sharing existing information, we've all come to rely on the internet to search and discover new information. If bots didn't go around visiting websites and scraping up their content, Google search wouldn't be able to populate its search engine and no one would know how to find what they're looking for. Finding the best restaurant in a new location you're visiting would be unheard of. The forums, websites, and apps we all use and rely on would slowly die, because no one would be able to find them. Discovering new areas of the internet would be impossible.

With the exception of paid content, the internet only works when content is free and openly accessible for users to view, share, and discover.

Unfortunately, despite all the downsides, there will still be website owners who continue to use these security tools in a way that blocks RSS users from accessing their content. But rest assured, Open RSS will continue to work vigorously to ensure public content on all websites remain open and freely accessible to our users through our RSS feeds.

If you are with us, please continue to use our feeds and send us feedback if you find that a feed isn't working for you. We also appreciate donations of any amount. The funding helps us improve our infrastructure and compensate our staff for their hard work to provide you with high-quality RSS feeds. Thank you so much for your support!