The best way to crawl a site is Screaming Frog. That should give you the information you are looking for.
Best posts made by Tylerj
-
RE: Best Site Architecture tool for analysis my & competitors site?
-
RE: Assistance with High Priority Duplicate Page Content Errors
What are the pages that are showing duplicate?
-
RE: Why isn't the Moz crawler getting all of my item pages?
Meghan from Moz pointed out I had Disallow: /*? in the robots.txt and that appears to have been the problem. It is the default Magento 2 setting, so not sure why they had that in there.
-
RE: Subdomains, blogs and redirects, oh my!
1. I do not have experience with a Hubspot blog, but I don't think a subdomain will hurt your efforts. I have always read that the best way to do it is in a subdirectory, but the difference between that and a subdomain isn't that big. You can setup a Wordpress blog pretty easy.
2. Google does know how to associate a redirect's link juice and domain authority to the page it redirects to. Gary Illyes said- all redirects pass PageRank now https://www.seroundtable.com/google-redirects-not-seo-22777.html
-
RE: Can't generate a sitemap with all my pages
Looking at your site how sure are you that you need 70,000 pages?
For the sitemap I would stop trying to use a website and do it yourself. It looks like you are running IIS. They have a sitemap generator that you can install on a server easily and run it there. It looks like you have GoDaddy, they catch a lot of crap but I have always found their technical support to be top notch. If you can't figure out how to do it on the server I would give them a call.
-
RE: I earned some high-quality backlinks, but the target page's ranking is not improving. Why?
Did you check to see if the links are follow or no follow? How old is the domain?
-
RE: Why do I have so many extra indexed pages?
It ended up being my search results. I was able to use the site operator to break it down.
-
RE: Putting rel=canonical tags on blogpost pointing to product pages
It might work, but eventually Google will catch on. If the site gets penalized for it, trying to get the ranking back up is a pain.
-
RE: Google WMT/search console showing thousands of links in "Internal Links"
Most of the time this is because it is in the main menu or in the footer. Is it in either of those locations? If it is a blog post do you show recently added posts anywhere? Or related posts?