Imagine the impact that an increase of 4 positions would have to your search performance. How much more traffic could you steal from your competition? How much more revenue would you those additional views translate into? How much would you save on paid search by being present on page 1 of Google’s results?
By understanding the intent behind your target customers searches, and tailoring your content to match, the prizes are immensely compelling. Across the following article, I will explain the theory behind it, why it’s more important than ever post Florida 2 (Google’s update of March 12th 2019), and most of all, what you should do about it, now.
What does ‘search intent’ mean for SEO?
Search Intent is literally the intention behind the search query i.e. the intrinsic want or motivation behind the words typed into the search engine. People use different search queries even though they seek to satisfy the same intent, therefore search intent usually has a one to many relationship to keywords
The impact is the difference between being on page 1 (a ranking with the propensity to earn traffic) and page 2 of Google’s search engine results, especially since the Florida 2. The impact is financial and determines your visibility. Understanding how people use search engines for different things and anticipating their expectations is a large part of winning the search intent game.
We think the best approach to SEO makes good use of consumer psychology and takes account of what is informing the search. You need to know what’s motivating that keyword before you optimise for it. Get this right, and you can expect higher rankings and conversion rates. Thankfully there is a machine learning approach to do this well and at scale.
Understanding those various dynamics, and striking a happy balance between delivering too much content (more content than the user expects to see when they land on the page to satisfy their query) and too little content, (the content they’d expect to see is spread over several pages and needs more effort to find) is the key to getting search intent right.
Search intent is about relevance and how close you are to satisfying the user’s definition of relevance. You could create new landing pages to unlock additional intent for example.
Content that fails to satisfy search intent isn’t ineffective for SEO
- The highest ranking content addresses the intent, not just the keywords. It’s highly unlikely that someone trying to find out about “equity release” expects or wants to find an “equity release calculator” on the same page. That is not to say that customers may be interested in navigation to the calculator from the Equity Release page, as they qualify themselves. There’s not enough overlap so no point hedging your bets on that front.
- Your users have lots more work to do. Let’s assume a parallel universe where someone is interested in both finding out what equity release is and calculating how much equity release they would get, and let’s assume you somehow create content that ranks for both intents. Is that content going to please the end-user? No, because there will be too much content to work, especially on a mobile screen through after reading about how equity release schemes work and the different ones available. Assuming every single user matters, and you need to be laser targeted on satisfying that user with a compelling offer and making it seamless for them to convert. The content will either be too thin on the info they need, or to heavy on the info they don’t need.
- Too many of your site pages will rank for the same keyword. It might sound good to flood Search Engine Result Pages (SERPs) with your site, but it isn’t when it’s on page 2. Google doesn’t like cannibalisation as it is forced to choose between content. If it’s showing loads of your URLs for the same keyword, it’s a sign that the all the relevant content is either split or duplicated across all of those pages. The impact of cannibalisation constrains the maximum ranking position achievable, which results in a dilution of PageRank (i.e. users and the press can now link to more than 1 landing page).
- All this can lead to lower user satisfaction and higher bounce rates. Your page authority won’t look good after all that….
What the relationship between search intent and keywords looks like
Think of search intent as a cluster of keywords, below we’ve visualised keywords that share and don’t share the same search intent. We looked at the data which showed sites organised like a shopping catalogue perform worse than sites structured according to how users actually search for information and products. This affects page content as well as how pages link to each other.
We mapped out how keywords are or aren’t connected by search intent. You can see that below.
If we know the distance between keywords in terms of search intent (in blue), we can see which keywords belong with each other as they are connected by lines. We can also see the search intents that don’t match up at all, they’re the ones with big gaps between them.
Now we know people searching “equity release”, “what is equity release”, and “how does it work” have a similar search intent and expect to see this content all on the same page.
On the other hand, we can see “equity release interest rates” deserves it’s own landing page. Other pages using the same target phrase and content will perform worse and cannibalise the search results. On the other extreme, not having any targeted content for equity release rates will result in weaker rankings. That’s really useful info to have before you start writing content and linking between pages.
If you’re ignoring search intent, your leaving money on the table.
How we hacked search intent (for a high volume of keywords)
Quite simply, comparing content in the search engine results between two keywords can tell us whether two keywords share the same search intent — or not.
That is because Google has already taken into account the multiple meanings of the search query, the content of the available pages on the internet. Most importantly, Google has user data as to how successful content satisfies the search query and what content the user expects to see. This has all been distilled and is reflected in the ranking position.
So, if the SERP content is similar then the two keywords share the same search intent, because if the results for two keywords are similar enough, the chances are that the search intent for both keywords are the same. The differences are likely to be variation in the way the title tags are formed.
If it’s dissimilar the two keywords and content should be mapped to different landing pages.
The great thing about this method is that it’s accurate and it works for any language. So if you’re a global head of SEO that doesn’t speak every language you cover, then this method will work for you.
Try it yourself now. Load some of your keywords and compare the search engine results. You’ll get some priceless insights on how well (or badly) mapped user intent is to your landing pages.
But doing this at scale would be prohibitively time-consuming and error prone. 100 keywords would involve 4,950 search result comparisons (based on 100 * 99 * 0.5). If you’d like to work out how many search intent comparisons between keywords, the formula is:
N * (N — 1) * 0.5
Where N is the number of keywords.
We have an API that does this at scale.
To prove it, consider the following use cases
1. Cannibalisation of the SERPs
The SERPs for ‘quilling’ and ‘quilling paper’ are shown below:
The results similar enough such that they both have the same video results, a common People Also Ask (PAA) question, and 4 common ranking URLs. These according to the API and Google share the same search intent.
Now take the case where the site below is ranking twice for the same queries:
It would appear Google is struggling with Denise (the site above) and she should therefore consider de-optimising her title tags for “quilling paper” and optimising it for “Quilling Paper Strips 1/16” instead to match its H1, and optimising the quilling paper page for, well, quilling paper!
Other cases, could be to merge content or to canonicalise one page to the other as appropriate. The API algorithm tells you which keywords align together, it’s up to the SEO specialist to decide how they are best aligned.
Search intent optimisation goes beyond SERP cannibalisation
2. Content Overload
These two pages from German search results have different search intents — malaria tablet cost, and cholera tablet cost. Lots of relevant stuff comes up for both, but the results are very different.
The screenshot (below right) is optimised for travel vaccinations. The second one is a landing page specifically for malaria tablets.
No prizes for guessing which is more useful for the searcher.
If the site on the left was yours, you’d need to use new or existing content to turn it into a landing page specifically for the malaria tablets search intent. Basically, it should look a lot more like the page on the right. Otherwise, your searcher’s probably going to assume this page isn’t for them and go back to Google.
The downstream effect is that should enough people will stop selecting your domain even if it’s ranking high, then the UX signals will be passed back to Google. The statistics will show that the URL isn’t satisfying the search query and the entire domain way be negatively impacted following an algorithm update.
Here comes the science
If we model the rankings by search intent, we get the following beta values for the regression model:
google_rank = a + b0*content_overload + b1*cannibalisation
(Intercept) : 6.415069
content_overload : 1.150652
The model output shows a rank gain of 4.7 positions gained on average for each excessive cannibalised URL removed. For content overload, the rank gain is 1 position for new landing page created.
The p-values for all 3 are less than 0.01, which means both types of search intent meet the statistically significant threshold of 99%. It’s less than 1% that you’ll see a random change in ranking because of multiple URLs or intentions.
Here’s a visual version of all that:
Over to you
It’s only when you map out the keywords and their interrelationships (where they exist) that you realise how close and distant some search intents really are. You might think you’re giving your site visitors what they want, but that’s to make two incorrect assumptions; firstly that you know better than your customers and secondly, that you are entirely free of any bias of your own.
The whole point of SEO is to make sure the content that users find first gives them what they want in a way they expect.
The data has shown that a reliable approach to SEO makes good use of consumer psychology (using Google’s search engine results based on their user feedback) and that takes account of what is informing the search.
You need to know what’s motivating that keyword before you optimise for it. Get this right, and you can expect higher rankings and conversion rates. The easiest and fastest way to do this is by comparing the search engine results of keywords to see how similar the ranking content is between them. If you use the Artios Intent API, this can be done in minutes for large keyword datasets in any language.
Artios help brands profit from their paid Search and SEO data using data science. Artios are based in London.