During any successful SEO campaign there is always a focus on link building, whether that focus goes to creating some killer content that is going to shake the world, an info-graphic, outreach or any other tactic that is going to bring in the links.
As a business grows their website generally follows suit. Web pages get created, updated, dropped and new ones get created in their place. The website gets a redesign, switches CMS platform and whole sections of pages that used to be there no longer exist. Events pages get created and promoted and then when the party is over the pages get taken down. All of this can result in a ton of juicy links pointing to your domain that result in an error 404 ‘Page Not Found’.
Unless the site you are working on is brand spanking new then finding and fixing orphaned links can be a quick and easy way of picking up lost links and can earn you some early brownie points.
This process literally takes 10 minutes and it can identify a ton of links that are pointing to your domain but going to 404 errors.
1. Export a .csv file of all of the backlinks pointing to your domain.
a) Sign up for Majestic SEO – http://www.majesticseo.com
b) Enter your domain in to Majestic Site Explorer, click the ‘Use Historic Index’ radio button then click search.
c) Select the ‘Backlinks’ tab and select the ‘Remove Deleted Backlinks’ radio button
d) scroll to the bottom of the page and click ‘Download CSV’
2. Extract a list of URLs using Excel
All we need from this .csv file is a list of site ULRs that have links pointing to them so we need to do a bit of doctoring in Excel.
a) Open the .csv file in Excel and delete out all columns except for the ‘Target URL’ column. Also delete the top row which says ‘Target URL’.
c) There will be a lot of duplicates in your list so remove these by selecting all of your URLs and using Excel’s ‘Remove Duplicates’ function. On my (Mac) version of Excel there is a button on the ‘Data’ tab but different versions may vary so a trip to google may be needed.
d) Save your list of URLs as a text file (.txt)
3. Check for orphaned links using Screaming Frog – http://www.screamingfrog.co.uk/
a) Fire up Screaming Frog and switch it to ‘List’ mode
b) Click ‘Select File’ and find the .txt file that you just created
c) Click Start
d) Once Screaming Frog has done its work click on the ‘Status Code’ column header which will group together pages with the same status code. If all of your response codes are returned as ‘200 OK’ or ‘301 Moved Permanently’ then this is the end of the road.
The common status code for orphaned links is ‘404 Not Found’ and these are the URLs that need to be redirected but you also need to check any URL that returns a status code of ‘302 Moved Temporarily’. Many content management systems and eCommerce platforms employ their own custom Error 404 pages which display a message along with some suggested links to get a site visitor back on track and these pages often result in a status code of ‘302 Moved Temporarily’.
If you have any ‘404 Not Found’ or ‘302 Moved Temporarily’ results then click the ‘Export’ button to save your Screaming Frog results as a .csv file.
Open up your exported .csv file from Screaming Frog in Excel.
4. Create a 301 redirect for any pages with a response of 404 Not Found
Taking recent Google updates in to account, it is worth while looking at any orphaned links in detail and treating any spammy-looking links with caution. Only redirect links that you are sure are good links.
Rather than just redirecting all of these ‘404 Not Found’ URLs any offending ‘302 Moved Temporarily’ URLs to the Home page of your site you should try to work out where these links should be pointing to and redirect them appropriately.
Implementing a 301 redirect is pretty easy but how you do it depends on what type of server your site is hosted on and how your website is built. You can find the different solutions here – http://www.webconfs.com/how-to-redirect-a-webpage.php
Majestic isn’t always amazing at picking up all links and you can sometimes see links appearing in the ‘New Links’ report that have been around for a long time. For this reason it is worth while checking for orphaned links every few months.