Covid19 update: WE ARE OPEN... it's business as usual and we look forward to hearing from you.

Competitor Content Audit

So, You Don’t Have a Content Strategy Then?

When we start working with a new client, one of the first tasks we carry out is a thorough content audit, to find out what level of content they are putting out there and how well it resonates with their audience. More often than not, they won’t have any strategy in place for content creation, never mind a full blown editorial calendar.

We need to get them moving, and fast, so we build out a content strategy for them – starting with ideation.

The Trouble With Niches

It’s impossible to be an expert in every niche. We have no magic formula to generate hundreds of brilliant content ideas out of thin air.

Luckily we don’t have to.

Instead, we simply steal content ideas from competitor sites, and this post explains how we use URL Profiler and Majestic SEO to speed up the process.

The first thing we need to do is identify a few competitor sites with decent blogs.

Ideally we’ll try to find at least 10 competitor sites, by looking for posts that seem to be getting shares and comments – we’ll quantify it all later.

Find The Sitemap of (Almost) Any Website

We need to grab all the URLs from our competitor sites, and the easiest way to do this is to use their sitemap. Also, the sitemap is where site owners put all of the content they want people to find, so it is perfect for our requirements.

Whilst there is no standard for the location of a sitemap, you can probably guess them 90% of the time by going to

For almost every other instance you’ll encounter, you can find it in the robots.txt file:


Or by getting creative with Google site searches:


Mountains of Data

Armed with their sitemap, we can now pull back data to help us identify our competitor’s best content. This is where URL Profiler comes in; fire it up and right click on the white URL box, then select ‘Import from XML Sitemap’


URL Profiler is a bit of a swiss army knife, and to get the best performance out of it you really need to think about which datapoints you are interested in. In our case we are interested in link metrics and social share counts on a URL level. Turn everything else off.


The software comes with an in-built free Majestic API, but if you already pay for a subscription it is well worth adding your Majestic API key as it makes the data retrieval almost instantaneous.

Set the Profiler running and go make yourself a nice cup of tea (and perhaps a chocolate digestive… or 5) and by the time you come back you’ll have a nice spreadsheet full of data waiting for you.


We now know, for every URL on the website; external link counts, CitationFlow, TrustFlow, and social shares from all the main social networks.

Pulling All The Data Together

We need to rinse and repeat this process across the other competitor websites we originally identified, then combine the data into a master spreadsheet.

This can either be done manually or by using a simple script to automate it (Mac version).

We are only really interested in blog content, so we can also sanitise the data by using a text filter in Excel:


Skimming Off The Cream

We’re only interested in the best content, and we can slice the data up in a couple of different ways to find it.

To get the social data in order, we’ll first need to sum across all the social shares we have collected, creating a new column called ‘Total Shares’.


Then we just need to sort by social shares, then copy the top 50 or so results into Sheet 2 of the Excel document.


This gives us the most socially popular content.

Now, to get the content which has been most popular in terms of links, sort by ‘URL Majestic Ref Domains’, then copy the top 50 of so results into the Sheet 2 as well.


There will obviously be a lot of overlap, so we can dedupe by URL to produce a final list, probably around 60-75 URLs in total.

The Building Blocks to a Content Plan

With this, we have a list of content which has a proven track record – either in terms of linking domains or social shares (or most likely, both).

This removes the guesswork for us. We know that if we can create quality content around similar topics, there is a good chance it will resonate with our target audience.

To dig into what this content actually is, we can pull page titles and meta descriptions from our list. Simply upload the list back into URL Profiler and select ‘Readability’ from the ‘Content Analysis’ section:


By analysing the ‘Title’ column we’ll get an idea of the topics that have proven popular in this niche.


The next stage will require a bit more manual work, in crafting post titles that are along the same lines but also seek to add something new to the conversation.

I’m a fan of Brian Dean’s (AKA Backlinko) methodology around this, which he calls the Skyscraper Technique, and basically involves finding the best content in a niche (we already did this above!) and then creating a much, much better version of the same thing.

As an example, we might look at the list above and decide to do a ‘Definitive Guide to ImportXML’. You don’t necessarily have to go to such extreme lengths, but you will clearly need to add a fresh perspective on the topic if you want it to gain traction.

This research acts as a starting point for our initial content ideas for our clients, and we often turn back to it several months in if we need more inspiration.

Magnifying Outreach

Once you have determined what your content will be and started producing it, you can again make use of the research data when it comes to outreach.

Take all of the URLs which inspired your content piece, and drop each one into Majestic to find referring domains – each of which can be pitched to see if they are interested in your piece as well.

Similarly, we can find any influencers who shared the inspiration post(s) by dropping URLs into Topsy:


You could build these into a Twitter Custom Audience if you were planning on doing paid social, or simply reach out to them individually to see if they might be interested in taking a look at your content.

Only Half the Battle

Competitor analysis remains an important skill in the modern SEO landscape, as it offers the opportunity to learn from other people’s experience.

This type of research is great for giving you a clear overview of what has worked in the past (and therefore what is likely to work in the future).

However, coming up with the idea is only half the battle; you still need to execute on both the content creation and the outreach.

About alastair

Self-proclaimed Lord Adwordz, I manage our client’s PPC campaigns and SEO accounts. Google is a huge part of my work, even more so as a recent Android convert. When not digging through data, you’ll find me beating the office at puzzle bubble or raiding villages on Clash of Clans.

Give us a call on 01460 281865 or email us to see how we can help you!