What To Do If Robotted Out Pages Rank In SERP?

Posted On : April 2nd, 2019   By : Durga Prasad Kundu   To : Technical SEO

Webmasters use robots.txt file to inform web robots about which pages not to crawl. But, if the robotted out pages of a website rank instead of pages having content in it, then what webmasters should do?

Recently on Twitter, Google’s John Mueller was asked about this and in its reply John said that “If a robotted page from your site ranks instead of a page with content on it, for queries that users use to find your site, then I think you have work to do :-).”.

If a robotted page from your site ranks instead of a page with content on it, for queries that users use to find your site, then I think you have work to do :-).

— 🍌 John 🍌 (@JohnMu) March 28, 2019

Well, if a page is disallowed in robots.txt, that doesn’t mean Google won’t show it in search results. In fact, if Google finds the page in external sources like links of that present in other sites, then depending on the query, Google may show it in search results.

But if you’ve similar page with content present in the site and still Google showing the robotted page instead of the page with content, then either there is critical issues with the page like poor quality content or duplicate content or that page doesn’t have any links from external sources.

I think that was what John wanted to convey when he said that you have to work to do.