How to find all 301 redirect for URL xyz.com/products (internal and external)?
-
This is what we are thinking:
- Get all URL of the xyz.com/products using XENU software.
- Search those URL on google (site;xyz.com url ) to find out if they are crawled by google, do the same on bing (as currently google shows 4k URL and bing 11k )
- Use opensiteexplorer (301 redirect ) and using (internal external) to get the desired result.
Is this the right approach? If not, what is the best way to find the correct result?
All suggestions are welcome.
-
You are welcome to work with the URLs in search results. I am unsure what numbers you are attempting to match.
-
Thanks @Ryan
I was referring https://www.google.com/#q=site:domain.com using site:domain.com , I know it doesnt give me all urls but isnt what we care about, at least matching numbers I mean?
-
You can check the redirects by uploading the LIST of URLs to Screaming Frog, which will then crawl the list and inform you of any header responses (301, 302, etc)
I am unclear on your second question. You previously stated the site involved is not a client and you do not have access to Webmaster Tools. What exactly are you asking or suggesting?
-
Thanks Ryan and everyone else, amazing answers So here is my understanding:
For Internal 301:
- Use Xenu or Screaming to scan url and create the list. I hope we can get a clean report from screaming.
For External:
-
Use Ose to find all backlinks and save them on excel
-
Use AHREFs to find all backlinks and save them
-
Use Majestic to find all backlinks and save them
-
Combine all url and remove duplicate(I guess manually we gotta do that)
Questions
a) How do we find out which one is 301 redirected beside checking each of them?
b) for backlinks we should check what google and bing crawled?
-
I agree - screaming frog is an awesome tool for finding the internal 301s. You can export a spreadsheet of all the pages that contain the 301s etc and so it makes creating a task list to work through (or pass on) pretty straight forward.
-
Your original question expressed a desire to locate "all" redirects for a given URL. It is highly unlikely to locate all such links without access to the link data in Google WMT and Bing WMT, along with the referrer data in GA.
The best you can do to find external URLs is to build a comprehensive backlink report using data from multiple providers (OSE, AHREFs, Majestic, etc). You should know from the start you will not cover all the links unless you are working with pages which have a low number of links.
-
Hi Ryan,
Thanks for your quick response.
Yes we heard about the screaming frog but never used it, will give it a try.
We do not have access to google/bing webmaster tool or analytics. We are more like a third party company working on this project.
Any other ideas?
-
I used to use Xenu until shortly after Dr Pete shared this blog (http://moz.com/blog/crawler-faceoff-xenu-vs-screaming-frog) which introduced me to Screaming Frog. Both will work, but you will find XENU is more like using DOS whereas Screaming Frog has a very nice interface.
For internal 301s, you can clearly crawl the site and export a list of all 301 redirects to the target page.
For external URLs, there is not a simple method. I suggest two tactics:
1. Examine your analytics for referring URLs
2. Examine your backlink reports for links to the page
You can then crawl the list of URLs and determine which pages are being redirected. With the above understood, the primary concern should be your internal URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How can a page have high Google/ organic traffic but show no ranking keywords in Moz?
We have a page on our website with a higher than average number of pageviews, 85% of which came from Google organic search. When I research this page by entering the URL into the "exact page" keyword research tool, Moz says it has no ranking keywords. How can a page be earning organic traffic without ranking for any keywords?
Moz Bar | | baystatemarketing0 -
I get this on every product, but i have put the keyword in the H1 tag makes now sense
Why it's an issue:Although using targeted keywords in H1 tags on your page does not directly correlate to high rankings, it does appear to provide some slight value. It's also considered a best practice for accessibility and helps potential visitors determine your page's content, so we recommend it. Over-using keywords, however, can be perceived as keyword stuffing (a form of search engine spam) and can negatively impact rankings, so use keywords in H1 tags two or fewer times. To adhere to best practices in Google News and Bing News, headlines should contain the relevant keyword target and be treated with the same importance as title tags
Moz Bar | | Carlsimp0 -
Issues with Page Grader using Wordpress / Visual Composer?
HI I know there is a flagged issue with page titles on the Visual Composer - but it is telling me there is no test on one page completely even though there is?? And it works on the others?
Moz Bar | | KellyDSD860 -
On Page Grader - URL not accessible
We have tried to use the On Page Grader today and it is coming back with URL not accessible for all pages on our website. We previously used the On Page Grader on Friday 10th Nov for a couple of product pages with no issues. Since then, the only changes we have made on the websites is updating some downloadable documents. We have done this several times before and it has never affected Moz. We have not changed the page URLs, and therefore do not know why it is now not working. The pages are working fine on the website with no issues. A link to one of the pages is below. http://www.processinstruments.co.uk/products/dissolved-oxygen-monitor/ Any help would be greatly appreciated.
Moz Bar | | PiMike0 -
Www.site.com linking to pages www10.site.com
The root domain of the website in question is www.site.com but all subpages are on the subdomain www10.site.com (I'm pretty sure it's a subdomain, at least, used for load balancing?). A funny thing happens on this site with the moz toolbar. I visit a subpage, www10.site.com/articles/articletopic1 That page has a lot of links on it, all of them visibly going to the subdomain www10.site.com. However, the moz toolbar shows some of them as Internal links and most of them as External links. As far as I can tell, there is no real rhyme or reason to the difference between the links that are highlighted as Internal vs. External. The link structures vary greatly: Some are properly structured www10.site.com/blogs/category
Moz Bar | | Motava
And some are poor like www10.site.com/articles/show_articles.php?section=category1 So a couple questions here: Does this subdomain www10 have a detriment on the rankings of subpages?
What could possibly cause the internal links on these subpages to be highlighted as external pages with the moz toolbar?1 -
On Page Grader can't access my URLs
HI- I am trying to grade some specific pages for keywords with the on page grader but it keeps telling me "Sorry, but that URL is inaccessible. " I can reach them via the browser and they are not https. Any thoughts? Here is a sample: www.bulkcandystore.com/kosher-candy Any help is appreciated. Ken
Moz Bar | | CandymanKen0