Google Search Quality Rater Guidelines
In this article, you will learn what Google Rater Guidelines are and how the document is structured and updated.
You will also learn what no one tells you about that document: Google's Search Quality Rating Guidelines are general guidelines and there is a lot more between those lines than what you get from most SEO articles on the topic.
Finally, you will know where to find the best resources for training and practicing your knowledge of the guidelines, as well as expert consultancy on search evaluation principles applied to SEO.
What are Google's Search Quality Rater Guidelines?
A bit of context and history.
Google's Search Quality Evaluator Guidelines is the document with instructions for Google raters (aka search engine evaluators) on how to rate the quality of web pages and the relevance of search results.
It's possible to find online references to Google's search quality rating program and guidelines that dates back to 2005, and even 2003, which proves that the company has been using this type of human feedback on search quality for over 15 years! But for a long period of time, the raters' guidelines were kept strictly confidential.
Despite Google's attempt to keep those guidelines secret, so to say, they would always end up being leaked.
In 2013, for the first time, Google decided to share their Search Quality Rating Guidelines publicly to the world. A big step, but with caveats.
The public version of the document was only 43 pages, whereas the leaked, non-public version was 161 pages. They gutted the guidelines for public release!
It was only in 2015 that the full version of the document was finally made publicly available by the company.
Ever since, in numerous opportunities and especially after the release of major updates, Google is actively recommending web publishers to read the quality rater guidelines.
"If you understand how raters learn to assess good content, that might help you improve your own content. In turn, you might perhaps do better in Search.", states Google on its recent guidance post on core algorithm updates.
How are the Guidelines Structured?
The document is divided into three parts.
Page Quality Rating Guideline
This is where Google specifies the criteria associated with page quality rating. Raters are instructed to observe several factors that are important as quality indicators, with special attention to Expertise, Authoritativeness and Trustworthiness (E-A-T). Examples are provided to illustrate each rating.
Understanding Mobile User Needs
To determine the relevance rating of a search result, the first step is understanding the needs of the user. This is where Google provides guidelines on what is important to consider for understanding user needs, particularly with mobile users in mind.
Needs Met Rating Guideline
After understanding the needs of the user, it's possible to assess how helpful a result is to satisfy their intent (i.e. meet the user needs). This is where Google provides additional guidelines on what is important for relevance rating. Part 3 is highly based on rating examples.
The Updates
There is not a predefined update frequency to the quality rater guidelines. Apparently, Google updates the document on an ad hoc basis.
According to historical data, we can expect two or three updates per year on average.
The changes are usually small and subtle. For example, they might include or remove examples, rephrase some sentences here and there, bring more clarity to what could be considered vague or confusing, provide additional instructions to avoid any bias, etc.
In other words, there are very little, if any, changes to the core of the document's content -- that is, on the guidelines as to how to evaluate web page quality and search result relevance. What changes is content that can be considered supplementary to the core content.
Either way, these updates always resonate heavily within the SEO community. And there is a reason for that: changes in the guidelines may signal that Google could be adjusting their algorithms to target those amendments.
The latest updates to the Search Quality Rater Guidelines took place on October 14, 2020, December 05, 2019, and September 05, 2019.
What no one tells you about the Quality Rater Guidelines
Until now.
The level of detail that many SEOs and marketers put into analyzing the Google rater guidelines is truly impressive. Few documents in the history of SEO and content marketing have been scrutinized to such an extent.
But there's a big but here. Most consultants and agencies applying Google rater guidelines principles for SEO have no actual experience with search quality evaluation!
How would it be asking for legal advice from someone who is very knowledgeable of the guidelines for consumer protection but has never worked as an attorney? That would be legitimate, of course, but only a professional would be able to know how that document works in the context of a legal process.
This exact analogy applies to what is going on, over the last years, in the SEO industry regarding the search quality rating guidelines. People drawing all sorts of conclusions from the document without being familiar with the search quality rating process itself.
So, let's clarify a few things.
Google's Quality Evaluator Guidelines are general guidelines, which means that in many cases the document is broad, vague and ambiguous, especially in the Needs Met section.
For example: how to evaluate the influence that the position of the target content on the page has on its relevance rating for the query? In our SEA Model methodology, we call this Extraneousness Issue.
And what about those extra clicks that may be requested from the user, or too much elapsed time before they are able to find the target content? What kind of influence do such things have on the relevance rating? In the SEA Model, that's what we call Effort Issue.
Experienced, top-performing raters also know that a result that addresses a less likely intent of the query cannot be rated in the higher end of the relevance scale, no matter the amount of content it provides for that intent. This is not something you will ever see addressed by those who know the guidelines but don't have enough or any experience as a rater.
You see, even with their 175 pages, the guidelines are far from specific enough to address many of these fairly common rating situations. Hence the term General Guidelines.
Google Quality Raters Guidelines for SEO and Content Marketing
SEO consultants and marketers who neglect the raters' guidelines (or claim that the document is overrated) probably fail to comprehend how impactful Content SEO can be.
In this context, the Search Quality Evaluator Guidelines are a treasure for SEOs and Content strategists, especially if used in conjunction with techniques known only to experienced, professional search quality raters.
The main reasons that make the document a gold mine of SEO and Content information are as follows:
It comes directly from Google
It provides details on what high quality pages should look like
It explains how the relevance of search results should be assessed
It shows where Google wants the search algorithm to go, according to Ben Gomes, the man in charge of Google Search
It shows what Google considers great content, which is a fundamental requirement for doing well in Search, as pointed out by Google Search Liaison Danny Sullivan
Where to get Google Rater Guidelines training and consultancy?
At the Search Evaluator Academy, there is a course based on quizzes created from the content of the guidelines. This can be useful for SEOs and marketers looking to ensure their understanding of that document.
There is also the SEA Model course, which is a complementary methodology for search evaluation to be used in conjunction with the general guidelines. The SEA Model is designed to make you think like an experienced rater, even if you have absolutely no experience in this field.
The Search Evaluator Partners program is a consulting service tailored to website owners, publishers, SEO and digital marketing professionals. It provides an opportunity for them to learn how search evaluation can help with SEO and Content, all while we evaluate their own websites or their client’s.