Requesting feed content unnecessarily severe
This app is unnecessarily re-requesting feed content from websites without waiting until there's actually new content published.
Why it's a problemThe unnecessary traffic wastes a website's resources that can be used for its other site visitors.
What it means for usersWebsites are likely to limit this app from accessing feed content, causing feeds to not update properly or stop working entirely.
How to fix it
The application needs to be updated to
wait
until the max-age of
Cache-Control header expires before re-requesting
feed content or properly implement
conditional requests.
Too many requests to non-existent content critical
A large number of requests are being made by this application for feed content across websites where no feed content exists.
Why it's a problemIt causes unnecessary strain on websites and can negatively impact their performance. Because bad bots often behave in this same way, websites will likely block the app for being associated with this activity, regardless of the application's intent.
What it means for usersWhen a website blocks this application for this behavior, feeds will stall or stop working entirely.
How to fix itThe application should reduce the number of requests being made to pages on a website that don't exist, and consider using the website's site map or robots.txt file, if available.
Not accurately identifying itself critical
Sometimes feed apps provide very little or inaccurate information about themselves to websites when requesting their content, which is what this application is doing. This is usually done to try to blend in with other traffic on the website, stay less noticeable, and decrease the chance websites will classify it as a bad bot trying to bypass a website's restrictions.
Why it's a problemFlying under the radar may be beneficial at times, but when done by a feed reader, it can look very suspicious. So websites are likely to block the application from obtaining feed content for not being identifiable.
What it means for usersIf a website blocks this application for being unidentifiable, which is likely, feeds used in the app won't work.
How to fix itWhen requesting feeds from websites, this app should properly identify itself using a unique User Agent header, with sufficient information so that websites always know what it is.
Not using updated feed locations critical
When a feed from a website is moved to a new location, the website informs the application where the feed has been moved to. However, instead of using the new locations of feeds, this application continuously tries to request feeds from their old locations.
Why it's a problemIt causes websites to use more computer resources than necessary to forcibly redirect the application to the rightful place over and over again.
What it means for usersFeeds will stop working or become fully inoperable when a website blocks the application due to this behavior.
How to fix itWhen a feed has been relocated to a new URL, the application should use the new URL and cease attempting to request feed content from the old one.
Requesting feed content from the wrong location critical
It's common for feed applications to grab content from some unconventional location on a website to generate a feed because the website doesn't provide it. However, even though the site already tells this app where feeds are located, the app is still attempting to extract content from other areas on the website that clearly aren't feeds.
Why it's a problemWhen a feed app doesn't request content from the right location or tries getting content from areas on a site not designated for feed consumption, the activity looks suspicious. This is likely to cause website owners to block this app from accessing its content.
What it means for usersFeed content obtained from the wrong location can be unpredictable and may not display correctly in the app. If websites block this app because of the behavior, which is likely, feeds won't work at all.
How to fix itWhen the app visits a webpage to retrieve feed content, it should:
-
Get the website's feed location from the
autodiscovery
linkelement of the page and use that location for all subsequent feed requests - Update any attempts by its users to subscribe to the incorrect location in the application to the correct new location without making any further requests to the website for this info
- Ensure the request isn't made to a location on the website that has been explicitly disallowed in its robots.txt file