If you’ve been a fan of the SEMtuts blog for a while now, then you probably would’ve realised that I’m a very big fan of Screaming Frog. Investing £149 into this tool is probably the best decision I’ve made, it’s saved me a lot of time and it’s automated the process of doing manual, borin’ work – which is always good.
Screaming frog can be used for a wide variety of reasons, whether it’s to identify mixed content errors on your site whilst transitioning from HTTP to HTTPs or to find internal linking opportunities. Its extraction feature is a golden gem that can be used for almost anything, as long as you know how your website is setup and coded.
How To Find Quick Internal Linking Wins With Screaming Frog
Assuming that you have a copy of Screaming Frog, and that you have a basic understanding of how to use the software, we can then use it to identify quick wins to our websites internal link structure. I’m not talking about any old link on your website, in this article, I’ll outline how you can find internal linking opportunities on the pages that matter, the pages that are capable of transferring juice to your other pages, it’s just that your not capitalising on this already. You will also need a license of Majestic to be able to identify what pages are worthy of building links on.
I assume you don’t have the time to go through and make amendments to every page, as the chances are your site is already established and it has hundreds or even thousands of pages. Therefore, to reduce your workload and the time associated with doing this, we can identify the pages that are powerful and have external links pointing to them already. This will increase the amount of juice being passed via your internal links, otherwise, you will spend a lot of time on pages that have 0 links – it isn’t worth it. I’m sure you have better things to be doing.
1) Determine what pages you’re looking to push and what keywords they rank for
Before any internal link building task, you need to determine what keywords you’d like those pages to rank for, otherwise, you’re probably just wasting your time. Firstly, you’ll need to identify what pages you’d like to start building links to and the keywords that are driving the most traffic to those pages. If one of your pages seems to be generating traffic from a keyword called ‘Web Design Services’ and it’s ranking 8th for that query – it’s worthwhile trying to build internal links using that anchor text, as this will signal to Google that this page is relevant to that query. This may result in that specific page ranking higher for that search term, in time.
To get an idea of what search queries are generating traffic, you can use a keyword research tool like SEMrush, or you can check your current webmaster tools setup to see the performance of your pages in organic search. Firstly, you need to get together a list of pages and what keywords you want them to rank for.
2) Use the extraction feature to fish out the pages that are referencing your desired keywords
The extraction feature can be used for almost anything, you will just need to know the XPath query (the tricky part). Here are some examples from the Screaming Frog team that may be useful. In this case, we’re looking to identify pages that have referenced a certain term in its content, some bloggers will default to the custom filter configuration feature to do this – but that’s wrong, doing so will pick up mentions of the keyword in all sorts of places, including your footer, header, sidebar and so on. At the moment, we are just looking for contextual opportunities within the main section of your content.
Therefore, I’d suggest using the custom extraction feature and an XPath query of something along these lines:
//li[@id="main"]//p[contains(text() ,'web design services')]
This XPath query is telling Screaming Frog to collate the URL’s that have referenced ‘web design services’ in its paragraph tags within the <li> section that has an ID of “main”. The chances are this query won’t work for your site, you will have to tamper with the query to get it to fit the structure of your site. The reasoning behind why this query works so well for the site I’m working on is because the keywords that I’m looking to create links on are referenced in the header and footer of the site, this query excludes those sections and only focuses on certain areas of my website’s code, otherwise the crawler will think that the keyword is on every page.
However, the below query should work for most sites, but this may still include sidebars, footers, headers and so on if a paragraph tag is being used in these areas.
//p[contains(text() ,'web design services')]
You can input your custom XPath query by going to Configuration > Custom > Extraction.
3) Run the extraction feature and refer to the ‘Extraction’ filter in the ‘Custom’ Section
Once you’ve got to grips of the XPath query that you’re going to use, as well as the pages you’re looking to push and their associated keywords, it’s time to run Screaming Frog. Once it’s completed, refer to the ‘Extraction’ filter in the ‘Custom’ section of the overview tab, if you’re happy, export that data into a spreadsheet by clicking the ‘Export’ button.
4) Bulk upload the exported URL’s to Majestic
Once you’ve exported the URL’s from Screaming Frog, it’s time to see if any of these URL’s are worthwhile building internal links on. We don’t want to spend time on adding internal links to posts that have 0 external backlinks, therefore we can use the Bulk Backlink Checker by Majestic to collect the number of backlinks/referring domains per URL.
Once submitted to Majestic, we can request a CSV export, one of which we can edit to filter out all of those URL’s that don’t have any external links. We only want the URL’s that have some sort of popularity outside of our site.
5) It’s time to get to work and add those internal links
Once all of the above steps have been replicated and we have a shiny new spreadsheet from Majestic containing all of the URL’s that reference our target keyword + have links pointing to them, it’s time to build those links. Go through the spreadsheet and ensure that you add your link to each of the mentions of your keyword, tick them off as you go and request a re-index of that page using Google’s Fetch as Google tool, or you can even use One Hour Indexing if you’re keen. Happy building!