When you’re launching a new website, it’s crucially important that your “301 redirect” strategy is in place correctly to avoid unnecessary headaches and frustration down the road.
If you’re changing your URL structure or moving to a new domain name, 301 redirects should be used to redirect the “Old URLs” to the “New URLs”.
However, more often than not, many websites have too many redirects implemented (3, 4 and even 5 at times) which negatively affects organic search performance by not passing enough link juice from redirect to redirect, to redirect and so on.
Luckily, there is now an easy solution to investigate these chains and clean them up so that you can steer clear of the giant mess that unnecessary redirects create.
Which Tool Can This be Accomplished With?
We will be using a tool called Screaming Frog SEO Spider that gives us the ability to turn a rather confusing process into something quite simple.
Having the right SEO tools for big projects like launching a new site is vital when it comes to ensuring a smooth transition. This could mean the difference between running a successful site launch or frantically scrambling to regain search rankings/organic traffic.
Let’s take a look below at the steps we need to take with Screaming Frog SEO Spider in order to pull a proper redirect chain report.
Change Your Spider Configuration
First things first, we need to change our settings in the “Configuration” section of Screaming Frog. What you will want to do is navigate to the “Advanced” tab in the “Spider Configuration” settings and check “Always Follow Redirects”. See the screenshot below:
Make sure you go through this first step as the default setting in Screaming Frog does not have this box ticked. It tells the software that we want to follow every redirect for the given URLs that we plug into the system.
This not only saves us a ton of time, but it also saves us the manual work of having to scrape and re-scrape our list of URLs more than once (you know what this means if you’ve used this method before). Luckily, by selecting “Always Follows Redirects”, this takes care of the whole process is one shot!
Make Sure “List” Mode Is Selected
Next, you will need to switch to “List” mode by navigating to the “Mode” tab and manually selecting “List” in between “Spider” and “SERP”.
This is different from the commonly used “Spider” mode as it allows you to plug a set list of URLs into the Screaming Frog software (instead of just the Home Page of the website like we do in Spider mode).
It’s important to note that “List” mode is also an excellent method to use even when you’re not auditing redirects and just have a specific set of URLs you want to scrape out.
Paste/Upload the Old URLs Into the Box
From there you will want to click on the “Upload List” button and select “Enter Manually” or whichever option fits your needs. It’s vital that you paste/upload the oldest set of URLs you have of the website when you are auditing redirects.
For example, if you were to upload the most recent URLs, you obviously wouldn’t get any redirect chains since you would already be uploading the “final destination” URL. Sounds like common sense, but you wouldn’t believe how many people get this step wrong.
Remember the idea here is to see the entire history of URLs/redirects so you can take the necessary action steps to clean this up or make recommendations. Therefore, you need to dig up your original scrape of the website and use the old URLs to get the full benefit from this approach.
Let’s Now Start The Crawl
Simply hit the “Start” button and let the fun begin! The Screaming Frog Spider will now start to crawl each one of the URLs you uploaded to the software.
You will quickly start to see the data crunching in the main box in Screaming Frog. The tool is now working on tracking every single redirect from start to finish, which is very valuable information when launching a new website (and after a site launch).
The only other way to get this information is to scrape the old URLs first, then scrape their redirects, then scrape their redirects and so on and so forth. This method would only be understandable if you didn’t know the right approach to auditing redirects. However, we now have a much more simplistic way of getting all the data we need when it comes to diagnosing redirect chains.
What should also be mentioned is that the time this scrape takes depends primarily on how many URLs you plug into Screaming Frog. Obviously, the more URLs you input, the longer it will take for this report to run and vice versa.
Either way, the process only takes a couple of seconds or minutes at the absolute most.
Download The Final Redirect Chain Report
Downloading the final report is as simple as going to the “Reports” tab and selecting “Redirect Chains”.
This will allow you to save the file in Microsoft Excel so you can refer back to it when necessary. This also makes it helpful when you have to make edits based upon the report you pulled. The information is neatly organized in Excel and easily digestible.
Many people don’t realize that you have this type of reporting capability in Screaming Frog and only use it for the general spider crawl of the website.
This really is a game changer for the more advanced SEO specialists out there and quite honestly can take things to another level.
Analyze the Report & Make Recommendations
Now that you’ve run, downloaded, and opened the report using Microsoft Excel, it’s time to dig in and analyze this information.
Generally speaking, you do not want to see too many redirects when reviewing your report. If this is the case, the idea here is to clean it up so you have the “Old URL” that 301 redirects to the “Final Destination/New URL”.
This way you do not lose as much link authority as you would with having 4 or 5 redirects in place. Far too many people have no idea what they’re doing when auditing redirects and either spend hours upon hours researching, or just leave a website ‘as is’ with multiple redirects in place (while taking a major hit to their SEO).
Now that you know what to look for, you will be able to make recommendations and take the necessary steps to clean things up.
That about wraps it up when it comes to auditing redirects for a website or the launch of a new website. Next time you have a sense that there’s too many redirects in place on your website or your client’s website, you will know exactly how to diagnose the issue.
Furthermore, you won’t become paralyzed by the complexity of having to dig through links upon links just to follow a redirect chain. Instead, you are armed and ready with a new solution to this problem and will be able to make the correct recommendations and/or even fix the changes yourself.
Please note that you should make this an SEO best practice moving forward, regardless what stage a website is at. You may be surprised at what you find.
If you are currently use Screaming Frog, I’d encourage you to dive in a bit deeper and see what other SEO benefits you can extract by testing this software. These types of techniques are what separates average SEO specialists from the more advanced SEO specialists and give you a major leg up on the competition.
This is the type of in-depth research ensures your site will stay in tip top shape from an SEO perspective. Sometimes these technical SEO tactics can seem extremely overwhelming and even boring, but I assure you that they are worth it.
Making sure redirects are in place properly can make a major difference and really helps you (and your website) stand out from the crowd…
That being said, how do you currently audit redirects for a website?
Note: The opinions expressed in this article are the views of the author, and not necessarily the views of Caphyon, its staff, or its partners.