Different Reasons Gary Illyes of Google explains
Google’s Gary Illyes spoke about the causes of the annoying “crawled but not indexed” problems in May at the SERP Conf 2024 in Bulgaria. The interview was conducted a few months ago, but until Olesia Korobka brought it to light in a recent Facebook post, it was mostly forgotten.
The insights are still relevant and helpful even if the interview was conducted a while ago.
Knowing That It’s Crawled, Not Indexed
Google Search Console’s “Crawled – Currently Not Indexed” statement indicates that while a page has been crawled, Google has chosen not to index it. One questioner asked whether this may be because the page is too similar to information that has already been indexed during the live interview.
This is a legitimate question since Google’s documentation doesn’t explain why a website may be crawled but not indexed.
Gary Illyes acknowledged that one explanation might be because the information is comparable to what already exists. He did point out that other elements may be involved.
He clarified:
Yes, it may be one of the causes. Internal data structures make it difficult to divide this category into more focused sections, even if it is the ideal approach. One such instance is duplicate content, when we decide not to index a page since there is already an identical version with superior signals in our index. However, there could also be a lot of other variables.
Site Quality’s Effect on Indexing
Illyes emphasized that Google’s choice to crawl but not index certain pages may depend on the general quality of the website. He stated:
How many ‘crawled but not indexed’ URLs you see in Search Console may be greatly influenced by the overall quality of the website. A large quantity of these URLs may be a sign of poor quality. There have been times when our evaluation of a website’s quality has changed, resulting in various indexing choices.
Additional Crawled Not Indexed Factors
Illyes went on to discuss other possible explanations for URLs that are crawled but not indexed. These might be modifications to Google’s evaluation of the website or problems with the website’s functionality.
He clarified:
If you see an increase in the quantity of “crawled but not indexed” pages, it may indicate that Google no longer views the website as authoritative. As an alternative, there might be technological flaws that prevent indexing, including the website delivering the same content under several URLs.
Important lessons learned
Gary Illyes listed a number of possible explanations for why Google could crawl a website but not index it:
- The content of the already-indexed pages is too identical.
- There is exact identical information on a different website with stronger indications.
- General problems with the quality of the site
- Technical problems
Illyes did not specify what he meant by “better signals” on another website, but it is probable that he was referring to situations in which Google finds syndicated information on one website to be more effective than on the original, hence not indexing the original.
Author Bio
With more than ten years of expertise in the digital marketing field, Jane Smith is a seasoned SEO writer. Jane is an expert in producing high-quality content that increases traffic and engagement, and she knows how to properly optimize for search engines. Her current area of expertise is assisting local companies improve their internet presence by offering affordable SEO services in Dunedin FL. Because of her broad experience in a variety of industries, such as healthcare, banking, and e-commerce, Jane is a flexible writer who can adjust to the demands of diverse clients. When Jane isn’t writing engaging content, she likes to explore Florida’s stunning beaches and keep up with the newest SEO developments.