How to find all 301 redirect for URL xyz.com/products (internal and external)?
-
This is what we are thinking:
- Get all URL of the xyz.com/products using XENU software.
- Search those URL on google (site;xyz.com url ) to find out if they are crawled by google, do the same on bing (as currently google shows 4k URL and bing 11k )
- Use opensiteexplorer (301 redirect ) and using (internal external) to get the desired result.
Is this the right approach? If not, what is the best way to find the correct result?
All suggestions are welcome.
-
You are welcome to work with the URLs in search results. I am unsure what numbers you are attempting to match.
-
Thanks @Ryan
I was referring https://www.google.com/#q=site:domain.com using site:domain.com , I know it doesnt give me all urls but isnt what we care about, at least matching numbers I mean?
-
You can check the redirects by uploading the LIST of URLs to Screaming Frog, which will then crawl the list and inform you of any header responses (301, 302, etc)
I am unclear on your second question. You previously stated the site involved is not a client and you do not have access to Webmaster Tools. What exactly are you asking or suggesting?
-
Thanks Ryan and everyone else, amazing answers So here is my understanding:
For Internal 301:
- Use Xenu or Screaming to scan url and create the list. I hope we can get a clean report from screaming.
For External:
-
Use Ose to find all backlinks and save them on excel
-
Use AHREFs to find all backlinks and save them
-
Use Majestic to find all backlinks and save them
-
Combine all url and remove duplicate(I guess manually we gotta do that)
Questions
a) How do we find out which one is 301 redirected beside checking each of them?
b) for backlinks we should check what google and bing crawled?
-
I agree - screaming frog is an awesome tool for finding the internal 301s. You can export a spreadsheet of all the pages that contain the 301s etc and so it makes creating a task list to work through (or pass on) pretty straight forward.
-
Your original question expressed a desire to locate "all" redirects for a given URL. It is highly unlikely to locate all such links without access to the link data in Google WMT and Bing WMT, along with the referrer data in GA.
The best you can do to find external URLs is to build a comprehensive backlink report using data from multiple providers (OSE, AHREFs, Majestic, etc). You should know from the start you will not cover all the links unless you are working with pages which have a low number of links.
-
Hi Ryan,
Thanks for your quick response.
Yes we heard about the screaming frog but never used it, will give it a try.
We do not have access to google/bing webmaster tool or analytics. We are more like a third party company working on this project.
Any other ideas?
-
I used to use Xenu until shortly after Dr Pete shared this blog (http://moz.com/blog/crawler-faceoff-xenu-vs-screaming-frog) which introduced me to Screaming Frog. Both will work, but you will find XENU is more like using DOS whereas Screaming Frog has a very nice interface.
For internal 301s, you can clearly crawl the site and export a list of all 301 redirects to the target page.
For external URLs, there is not a simple method. I suggest two tactics:
1. Examine your analytics for referring URLs
2. Examine your backlink reports for links to the page
You can then crawl the list of URLs and determine which pages are being redirected. With the above understood, the primary concern should be your internal URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I can't find multiple titles
Hello, Moz showed a lot of multiple titles on our website but, when I go to fix them, there isn't any multiple titles. I was wondering if anyone knew where these issues could be coming from? Many thanks, Olivia
Moz Bar | | SigneerHFS0 -
Is it possible to find every reference to my company name across the internet?
I am trying to find every single website that mentions my company name (attempting to identify some negative posts.) It is a rather unique name but I suppose that kind of search would be difficult. Any ideas?
Moz Bar | | pcthrpy0 -
Moz Top Pages and Tool Bar Not Crawling Internal Pages and Links
Hello, We’re having two issues with our Moz tools and we’re not sure what’s causing them and whether they are related. The Moz Bar isn’t highlighting some of our internal links (including navigation links). The Top Pages Report in Open Site Explorer is only picking up the homepage and a couple error pages (none of the internal pages). The full Crawl Report is picking up everything though. Could a potential cause of both these issues be the Title attribute in some our links? – We use <a <="" span="">title="Example" href="link"></a> <a <="" span="">Or is this most likely from something else blocking the crawler from accessing our links/pages? Google Search Console does seem to be picking up the links in the navigation and everything is indexed/rendered correctly so we also didn’t know if this is something that could be issue. Any insight or help would be appreciated. Please let us know if there are any details we could provide that might help. Looking forward to hearing from all of you! Thank you in advance. Best,</a>
Moz Bar | | Ben-R0 -
Learn how to use MozBar to find local citations. Get your Daily SEO Fix!
Citations are mentions of your business and address on webpages other than your own such as an online yellow pages directory or a local business association page. They are a key component in search engine ranking algorithms so building consistent and accurate citations for your local business(s) is a key Local SEO tactic. In today’s Daily SEO Fix, Tori shows you how to use MozBar to find citations around the web. Check out the video and go give this tip a try! Don't have MozBar? No problem! You can download it for free here. This video wraps up The Moz Daily SEO Fix--Moz tool tips and tricks in under two minutes. Thank you SO much to everyone who has been following this campaign, we really appreciate your support. To watch all of our "Fixes," visit The Moz Daily SEO Fix playlist on YouTube. And, be sure to keep an eye out tomorrow for our wrap-up blog post...there may be a pretty hilarious blooper reel! 😉
Moz Bar | | kellyjcoop3 -
How to find and reduce more of 100 internal links on Prestashop
hi, i'm working on a site endrena.com who has more of 466 links, but i don't know what they are, i know only the links from menu and subcategories but they are not 466. the moz onpage grader say me the category receive a A grade and it don't say me any advice like more link but if i use the rankranger onpage it found more of 466 links! Can you help me to find all this links? 166m3gz.png
Moz Bar | | seopalermo0 -
Why can't On-Page Grader grade any Hilton hotel URLs?
I'm receiving the "Sorry, but that URL is inaccessible." for every hilton hotel webpage I check when using On-Page Grader. Is Hilton blocking Moz's On-Page Grader or is something else going on? Here are a few "inaccessible URLs" from different brands within Hilton's portfolio: http://doubletree3.hilton.com/en/hotels/new-york/doubletree-by-hilton-hotel-metropolitan-new-york-city-NYCDTDT/index.html http://home2suites3.hilton.com/en/hotels/tennessee/home2-suites-by-hilton-nashville-vanderbilt-tn-BNAHTHT/index.html http://hamptoninn3.hilton.com/en/hotels/florida/hampton-inn-and-suites-destin-DSINEHX/index.html http://hiltongardeninn3.hilton.com/en/hotels/georgia/hilton-garden-inn-atlanta-downtown-ATLDOGI/index.html Thanks in advance.
Moz Bar | | Just-Me0 -
Where to find one off crawl report
Hello, I don't know if I am being a bit daft but I don't seem to be able to find the area where I can request a one off crawl report anymore (rather than setting up a campaign). Can someone let me know where this is now? Thanks!
Moz Bar | | RikkiD220 -
Moz not crawling opencart product pages
hi, i have waited for over 2 weeks now and the crawler only got 8 pages, and is not getting all the open cart pages and products. any idea of what can be wrong? im using joomla 2.5.11 and mijoshop 2.0.5 (which uses opencart 1.5.5.1). thanks
Moz Bar | | marlvass10