SEOMOZ Crawling Our Site
-
Hi there,
We get a report from SEOMOZ every week which shows our performance within search. I noticed for our website www.unifor.com.au that it looks through over 10,000 pages, however our website sells less than 500 products so not sure why or how so many pages are trawled? If someone could let me know that would be great. It uses up a lot of bandwidth doing each of these searches so if the amount of pages being trawled reduced it would definitely assist.
Thanks,
Geoff
-
That's a question best answered by a programmer and/or someone who is familiar with the code of your site. Sorry I can't help more!
-
Hi Adam,
Thanks - how do you change the navigation so each page is linked with a single URL?
Thanks
-
If SEOmoz is crawling all those duplicate/unnecessary URLs, the search engines probably are, too.
Best solution: Change the navigation on your site so that each page is linked to using a single URL.
-
Thanks Adam.
Have had a look and it looks like the crawlers are cycling through numerous unnecessarily crawled pages for each product based on the top and bottom nav bar on our website (i.e. it adds FAQ and Shipping Info for each product + all the categories we have created on the website). Attached is an image link of some lines from the CSV file.
I imagine this is what is making it chew up bandwidth. Any insight on how to avoid / change this would be much appreciated.
Thanks
Geoff
-
In addition to what Adam said, you can just search your site on Google and I come up with over 14,000 results:
Both ways should give you a good insight into what's being found.
-
Hi Geoff,
There are multiple reasons your site could have "extra" pages: tag pages, category pages, print this pages, duplicate pages, etc.
To see what pages SEOmoz is crawling:
- Go to http://pro.seomoz.org/campaigns
- Click View This Campaign for your site
- Click Crawl Diagnostics
- At the top right, Export as CSV
- You'll get a spreadsheet listing all the URLs that SEOmoz crawled
Hope that helps!
~Adam
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEOMoz Langage ?
Hello, Is there any way to change the lagage of the interface ? Especially for reporting tool ? Thanks in advance for your answer, Alex
Moz Pro | | ACORED0 -
Order of urls in SEOMoz crawl report
Is there any rhyme or reason to the order of urls in the SEOMoz crawl report, or are the urls just listed in random order?
Moz Pro | | LynnMarie0 -
Crawl Disgnosis only crawling 250 pages not 10,000
My crawl diagnosis has suddenly dropped from 10,000 pages to just 250. I've been tracking and working on an ecommerce website with 102,000 pages (www.heatingreplacementparts.co.uk) and the history for this was showing some great improvements. Suddenly the CD report today is showing only 250 pages! What has happened? Not only is this frustrating to work with as I was chipping away at the errors and warnings, but also my graphs for reporting to my client are now all screwed up. I have a pro plan and nothing has (or should have!) changed.
Moz Pro | | eseyo0 -
Site Ranking Report
Hi guys, My site ranking report says that I've gone from being 1-20 for a variety of keywords in Google UK to not in the top 50. When I do a search myself I see that my site remains where it previously was (between 1-20). How reliable is the site ranking reporting on a weekly basis? Is it best to look at it monthly?
Moz Pro | | columbus0 -
Is there anything like SEMRush in SEOMoz?
I'm new to SEOMoz and trying to find something like SEMRush for finding what keywords competitors use. Is there anything like that within SEOMoz?
Moz Pro | | seanuk0 -
SEOMOZ reports the statistics.
SEOMOZ reports the Statistics, but where do i manage & improve??? Simply Statistics is all about SEOMOZ??
Moz Pro | | webicers0 -
Dismiss crawl diagnostics error
Hello everyone, Is there a way to dismiss some errors in the Crawl Diagnostics tool so they don't appear again? It happens so that some of the errors are never going to be fixed because of their nature. For example, 'Title too long' errors that point to some of the threads on my forum - it doesn't make sense to change the title of a thread posted by user just for the sake of the error disappearing from the 'Crawl Diagnostics' tool. 🙂 Otherwise the CD interface gets a little bit cluttered with errors which I will never fix anyway. I wonder how others deal with this problem. Thanks.
Moz Pro | | MaratM0 -
Site not showing data in Open Site Explorer
I have run open site explorer for a couple of sites but I am getting the "no data available for this URL" message. I have read the reasons listed on the error message but none of these apply to the particular sites I am looking for data from. Any thoughts as to why this might be? Thanks, Kwan
Moz Pro | | SocialKwan0