My crawl is only crawling only the 1st page. What’s going wrong?
-
When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .
Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./
http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx -
Hi there!
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.
Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.
I hope this helps, and if you have any other questions, please reach out to us at help@moz.com! We'll be able to take this conversation offline and answer any other questions for you from there!
Cheers!
Erin
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Correct GMB page is not being pulled to validate in Moz Dashboard
Hi, I have a client who will be purchasing a Moz Local package. They have 1 main location and 11 off-site locations throughout the city. I am trying to validate the last location through the dashboard but the correct Google My Business page is not being pulled. I went to the backend of the Google My Business page and there is a banner saying, "Your business is verified. Listings may be reviewed for quality and can take up to 3 days to be published." This banner has been present for almost 3 weeks now. The Google Business pages were bulk verified and only this one page is pending review. Does anyone have insights as to why the page is still pending review? If the page is verified, why is it not being pulled to Moz Dashboard?
Moz Local | | FSCInteractiveLLC0 -
How long do Facebook and Google pages have to be published before Moz can verify them?
Hi there. I've checked the forums and have seen similar questions, but couldn't find an answer in them. I have a listing that I imported from my Google My Business account, but it comes up as unverified for both Facebook and Google. Both pages exist, but have only been published for about 3 weeks. Both sites contain the accurate business information and are the appropriate types of pages, so the best I can determine from what I've read in the other posts is that "newly" published Facebook and Google pages may not be able to be verified by Moz Local. So the question is, what is considered "newly published"? The Facebook business page is: https://www.facebook.com/JTBentonConsulting/ The Google+ page is: https://plus.google.com/b/114469171569907148968/+JTBentonConsultingAdamstown?pageId=114469171569907148968&hl=en The name registered for Google My Business is: J T Benton Consulting - Mobile Notary Public Any ideas?
Moz Local | | monicastuckey0 -
Trying to connect my moz local to my facebook page - says it isn't found
Trying to connect my moz local to my facebook page - says it isn't found even though it is. I don't want to create another listing for this. Cheers Claire
Moz Local | | blenkiseo0 -
Has anyone found a way to change the Google page linked to a Moz local account AFTER it was purchased?
Before realized that the Moz listing should have the correct GMB account connected to it, a few local listings were purchased. Is there a way to change the GMB page it is pulling data from and sending out to directories? Thanks!
Moz Local | | BlueCorona2 -
What's Going On with Moz Social
Why can't I add my social info to the Moz campaign that I'm currently running?
Moz Local | | maxcarnage0 -
Facebook page has disappeared from moz local
My facebook page is very active and very much complete. This was shown on moz local up until a week ago. I have changed nothing on facebook but now moz local shows that it is 0% complete and only gives the option of creating a listing. How do i rectify?
Moz Local | | Sassandgrace0 -
How can I add the county (I'm in the UK) to my Facebook page
I am trying to setup a Moz Local campaign but I am unable to change the county (I am in the UK). In the Moz report, my 'state' is listed as 'Cnwll', but it should be 'Cornwall'. My Google + page has the correct information and is verified. My Facebook page has the correct information, apart from the fact that I can't add add a county. Does anyone have any suggestions on where I could go from here? Thanks.
Moz Local | | Nanuq0 -
I updated my Google My Business page weeks ago and the name of my business still has not changed in Moz Local. Of course it won't allow me to change the name. Please help.
I updated my Google My Business page weeks ago and the name of my business still has not changed in Moz Local. Of course it won't allow me to change the name. Please help.
Moz Local | | NicoleDeLeon0