Yarr

It's common for feed applications to grab content from some unconventional location on a website to generate a feed because the website doesn't provide it. However, even though the site already tells this app where feeds are located, the app is still attempting to extract content from other areas on the website that clearly aren't feeds.

Why it's a problem

When a feed app doesn't request content from the right location or tries getting content from areas on a site not designated for feed consumption, the activity looks suspicious. This is likely to cause website owners to block this app from accessing its content.

What it means for users

Feed content obtained from the wrong location can be unpredictable and may not display correctly in the app. If websites block this app because of the behavior, which is likely, feeds won't work at all.

How to fix it

When the app visits a webpage to retrieve feed content, it should:

1. Get the website's feed location from the autodiscovery link element of the page and use that location for all subsequent feed requests
2. Update any attempts by its users to subscribe to the incorrect location in the application to the correct new location without making any further requests to the website for this info
3. Ensure the request isn't made to a location on the website that has been explicitly disallowed in its robots.txt file

This app is unnecessarily re-requesting feed content from websites without waiting until there's actually new content published.

Why it's a problem

The unnecessary traffic wastes a website's resources that can be used for its other site visitors.

What it means for users

Websites are likely to limit this app from accessing feed content, causing feeds to not update properly or stop working entirely.

How to fix it

The application needs to be updated to wait until the max-age of Cache-Control header expires before re-requesting feed content or properly implement conditional requests.