Site structure: Any issues with 404'd parent folders?
-
Is there any issue with a 404'd parent folder in a URL? There's no links to the parent folder and a parent folder page never existed. For example say I have the following pages w/ content:
/famous-dogs/lassie/
/famous-dogs/snoopy/
/famous-dogs/scooby-doo/But I never (and maybe never plan to) created a general **/famous-dogs/ **page. Sitemaps.xml does not link to it, nor does any page on my site.
Is there any concerns with doing this? Am I missing out on any sort of value that might pass to a parent folder?
-
Yeah - there is various speculation about how signals or authority traverse folder structures (see for example this whiteboard Friday ) but I haven't seen anything suggesting it's permanent - all of this may be an argument for adding /famous-dogs/ at some point, but I wouldn't personally stress about it not being there at launch.
-
Yeah. I'd just leave it as a 404 in that case
-
In my scenario, considering I might add a parent "famous dogs" page at some point, it'd probably best to leave robots.txt alone, right?
-
Thanks for the response. This is what I expected.
I swear I read somewhere that Google may pass some form of value from a child to a parent. i.e. "/famous-dogs/lassie/" could pass some value to "/famous-dogs/", absent any links. Can't find the source, but I suppose I'm a bit worried that I'd permanently lose out on some value if the parent does not exist initially. Considering I may add a "famous dogs" parent page at some point.
-
PS - if you're worried about the crawling, you could always block it in robots.txt if you really wanted (but unless it's a huge site I wouldn't bother). Note - if you do go this route, do it carefully so as not to block all contents of the folder at the same time!
-
The short answer is that there should be no harm going with your proposed approach.
Longer version: I believe there are cases where Google has tried to crawl a directory like "/famous-dogs/" in your example purely because it appears as a sub-folder in the paths of other pages even though there are not any direct links to it. But even if it does crawl it, if you don't have or intend to have a page there, a 404 is a perfectly valid response.
In general, while there could be a case that it's worth creating a "/famous-dogs/" page if there is search demand you can fulfil, until or unless you do, there is no harm in it returning a 404 response.
-
Seems odd that indexers would care if a parent directory page exists or not. Is there any proof that Google will attempt crawl parent folder pages that aren't in sitemaps.xml and aren't linked to anywhere else?
Perhaps I'm slowly building out my site. Depending on the material/approach, it might make sense to release a page talking about a sub-category (lassie) before releasing content about a parent category (famous dogs). Or maybe "famous dogs" is such low search volume that it doesn't make sense to spend time creating a parent "famous dogs" page.
If I'm understanding correctly, with the above you're effectively telling me to:
1. Build a parent category page. If I don't plan on investing much time/effort into the parent page, noindex it.
2. Reorganize my site folder structure.
Neither seem like a great option.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What's the average rank update time after site and/or backlink changes?
What's currently the typical time, ON AVERAGE, it takes to see ranking changes when significant improvements are made to significant ranking signals on a long-established (as opposed to brand new) website? Does the rank update associated with on-page optimization happen sooner than addition of quality backlinks?
Intermediate & Advanced SEO | | JCCMoz0 -
How Best To Accommodate A Site's Changing Subject Matter?
Hi, I'm dealing with a several year old site that has had a lot of success in organic search around one particular subject and is now evolving into other subjects. Would like your experience on how best to handle this. Here's what we have so far: First, the site was about niche craft carpentry. Then, it added training. Then, it added training in other subjects in smatterings, like plumbing, electrical, etc. Now it's considering adding training in subjects even further from niche craft carpentry. So, interior decorator training, landscaping training, etc. Nearly all of it's organic search traffic (about 200,00 per month) comes from blogs, articles and discussions related to the original topic of niche craft carpentry... not training. As we've branched out from carpentry into carpentry training and then other subject training, have not had great success in organic with these new less related topics. We've had some for carpentry training type terms, but not much else. If the site owners are hell bent on expanding into these other training subjects for business reasons other than search, how would you structure it? For instance, would you go originalsitename.com/landscaping or landscaping.OriginalSiteName.com or what? I understand that a landscaping.originalsitename.com is for all intents and purposes a new domain name and won't have the authority of the original. However, would it have more chance of breaking free of how Google has pigeon-holed the original site's subject matter as niche carpentry-relevant only? Or, would you just keep adding subjects to the original domain name and figure that one of these days google is going to see it as the Lynda.com of an expanding galaxy of home improvement? I should add that the future of the site is training, so landscape training or interior design training is pretty far from high end niche carpentry stuff. What do you think? Thanks!
Intermediate & Advanced SEO | | 945010 -
My site shows 503 error to Google bot, but can see the site fine. Not indexing in Google. Help
Hi, This site is not indexed on Google at all. http://www.thethreehorseshoespub.co.uk Looking into it, it seems to be giving a 503 error to the google bot. I can see the site I have checked source code Checked robots Did have a sitemap param. but removed it for testing GWMT is showing 'unreachable' if I submit a site map or fetch Any ideas on how to remove this error? Many thanks in advance
Intermediate & Advanced SEO | | SolveWebMedia0 -
Site Redesign Inconsistent Trailing Slash Issue
I'm looking at a site that has implemented trailing slashes inconsistently across multiple pages. For instance:
Intermediate & Advanced SEO | | GrouchyKids
http://www.examplesite.co.uk/ (WITH)
http://www.examplesite.co.uk/product-range (WITHOUT)
http://www.examplesite.co.uk/product (WITHOUT)
http://www.examplesite.co.uk/blog/ (WITH)
http://www.examplesite.co.uk/blog/blog-article/ (WITH) The blog was created later in Wordpress which is one of the reasons why this issue exists. Looking at the inbound links unsurprisingly the lions share go to the home page but lots of other pages have links as well, particularly the product pages, no to many to the blog pages. This pattern is similar in terms of which pages rank, the home page ranks well for a variety of phrases, the product pages also do quite well. I know that ideally the URL's should be identical to the existing site, or if you have to you should 301 redirect old to new. The client wants to switch the whole site over to Wordpress which will be default implement a consistent URL structure across the board, thereby changing at least some of the URL's no matter what I do. I remember a Matt Cutts video that stated that even a 301 redirect will loose a clicks worth of link juice see: https://www.youtube.com/watch?v=Filv4pP-1nw The existing site has a poor UX compared to the new proposed design so this should help us. Has anyone got any experience with a similar issue or any advice about how best to proceed?0 -
Penguin recovery, no manual action. Are our EMD sites killing our brand site?
Hi guys, Our brand site (http://urban3d.net) has been seeing steady decline due to algorithm updates for the past two years. Our previous SEO company engaged in some black-hat link building which has hurt us very badly. We have recently re-launched the site, with better design, better content, and completed a disavow of hundreds of bad links. The site is technically indexed, but is still nowhere in the SERPs after months of work to recover it by our internal marketing team. The last SEO company also told us to build EMD sites for our core services, which we did: http://3dvisualisation.co.uk/ http://propertybrochure.com/ http://kitchencgi.com/ My question is - could these EMD sites now hurting us even further and stopping our main brand site from ranking? Our plan is to rescue our brand site, with a view to retiring these outlier sites. However, with no progress on the brand site, we can't afford to remove these site (which are ranking). It seems a bit chicken and egg. Any advice would be very much appreciated. Aidan, Urban 3D
Intermediate & Advanced SEO | | aidancass0 -
Was anyone hit by BOTH the 'Phantom' update as well as Penguin 2.0?
I'm interested to know if Phantom was just a "pre-Penguin" 2.0 or if it was a completely different update. Thoughts?
Intermediate & Advanced SEO | | nicole.healthline0 -
Refocusing a site's conent
Here's a question I was asked recently, and I can really see going either way, but want to double check my preference. The site has been around for years and over that time expanded it's content to a variety of areas that are not really core to it's mission, income or themed content. These jettisonable other areas have a fair amount of built up authority but don't really contribute anything to the site's bottom line. The site is considering what to do with these off-theme pages and the two options seem to be: Leave them in place, but make them hard to find for users, thus preserving their authority as an inlink to other core pages. or... Just move on and 301 the pages to whatever is half-way relevant. The 301 the pages camp seems to believe that making the site's existing/remaining content focused on three or four narrower areas will have benefits for what Google sees the site as being about. So, instead of being about 12 different things that aren't too related to each other, the site will be about 3 or 4 things that are kinda related to eachother. Personally, I'm not eager to let go of old pages because they do produce some traffic and have some authority value to help the core pages via in-context and navigation links. On the other hand, maybe focusing more would have benefits search benefits. What do think? Best... Darcy
Intermediate & Advanced SEO | | 945010 -
A Client Changed the Link Structure for Their Site... Not Just Once, but Twice
I have a client who's experiencing a number of crawl errors, which I've gotten down fo 9,000 from 18,000. One of the challenges they experience is that they've modified their URL structure a couple times. First it was: site.com/year/month/day/post-name
Intermediate & Advanced SEO | | digisavvy
Then it was: site.com/category/post-name
Now it's: site.com/post-name I'm not sure of the time elapsed between these changes, but enough time has passed that the URLs for the previous two URL structures have been indexed and spit out 404s now. What's the best/clean way to address this issue?I'm not going to create 9k redirect rules obviously, but there's got to be a way to address this issue and resolve it moving forward.0