Cgi-bin folder
-
Is there cg-bin folder of any site related to SEO in anyways and why we always block Cgi-bin folder in robots.txt.
-
As eyepaq says - it's for holding executable "program" files. search engiens don't need access so it gets blocked. If it's empty you can even remove it.
-
No. No correlation with SEO.
However the same way you block your temp or admin folder, there is no need in letting any bot crawling your cgi-bin folder.
Cgi-bin folder holds executable file - you don't want external sources to see it anyway - it's "very" private from security reasons. You need more then knowing what's inside the folder to do harm but keeping it hidden is the wise call. Better safe then sorry.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I want to move some pages of my website to a folder and nav menu in those pages should only show inner page links, will it hurt SEO?
Hi, My website has a few SaaS products, to make my website simple i want to move my website some pages to its specific folder structure , so eg website.com/product1/features
Technical SEO | | webbeemoz
website.com/product1/pricing
website.com/product1/information and same for product2 and so on, the website.com/product1/.. menu will only show the links of product1 and only one link to homepage (possibly in footer). Please share your opinion will it be a good idea, from UI perspective it will be simple , but i am not sure about SEO perspective, please help thanks0 -
Cross Domain Canonicalization for Site Folder
Hello colleagues! I have a client who decided to launch a separate domain so they could offer their content translated for other countries. Each country (except the US/English) content lives in its own country folder as follows: client.com/01/02/zh
Technical SEO | | SimpleSearch
client.com/01/02/tw etc. The problem is that they post the content in US/English on this domain too. It does NOT have its own folder, but exists righth after the date (as in the above example) Oh, and the content is the same as on their "main" domain so google likes to index that sometimes vs. the original client on the domain where we want the traffic to go. SO, is there a way to say "hey google, please index the US content only on the main domain, but continue to index the translated content in these folders on this totally separate domain?" Thank you so much in advance.0 -
URL Structure & SEO - Should we be using sub-folders?
Hi all, As part of our content marketing efforts we have run a number of initiatives in the past and created pages on the website to go along with them (also where the links for these particular projects point to). However, the URL structure isn't actually a reflection of where the pages sit on the site. Unfortunately I'm unable to provide a URL for reasons I won't bore you with, but here's an example: We recently ran a competition that was very successful in generating links. The URL for this is www.domain.co.uk/competition. However, the page actually sits within the About Us section - which is where all of our news and content marketing pages go - and uses a URL override. How much of an issue is this in regards to A) Our SEO in general?; and B) Ensuring we receive as much equity from the links we earn as possible? A brief explanation of what URL overrides actually are would also be useful! (We have a digital marketing agency who handle most of our SEO) Thanks in advance guys! John
Technical SEO | | NAHL-14300 -
SEO problem if homepage is 2 folders deep?
We are currently looking at a site for a client, where instead of featuring standard file structure, every folder is being buried two folders deep by the CMS. So the homepage is: www.domain.com.au/folder/folder And a subpage is: www.domain.com.au/folder/folder/subpage Is this necessarily and SEO problem? Will it be positive for rankings to pull out the two redundant folders? Any insights are appreciated! Cheers
Technical SEO | | MarketingResults0 -
Google indexing directory folder listing page
Google somehow managed to find several of our images index folders and decided to include them into their index. Example: websitesite.com/category/images/ is what you'll see when doing a site:website.com search. So, I have two-part question: 1) Does this hurt our site's ability to rank in any way?
Technical SEO | | invision
Because all Google sees is just a directory listing page with a bunch of links to images in the folder. 2) If there could be any negative effect, what is the best way to get these folders out of Google's index?
I could block via robots.txt, but I'm afraid it will also block all the images in that folder from being indexed in Google image search. I could also turn off directory listing in cpanel / htaccess, but then that gives is a 403 forbidden. Will this hurt the site in anyway and would it prevent Google from indexing the images in the directory? Thanks,
Tony0 -
Keywords in file names vs folder names
We understand the value of a keyword phrase included in the URL. Is there more value to having that phrase in the folder name of the URL or the file name or does it matter? Example: http://www.biztoolsone.com/website-design.php or http://www.biztoolsone.com/website-design/ Which is best? Thanks, Wick Smith
Technical SEO | | wcksmith0 -
Which is better? Remove folder or fix the links and wait?
Dear Mozers, Recently I added a new language to one of my sites in a new folder (www.site.com/es/) but for some reasons many links got broken or were simply sending to a wrong page. This caused a bad indexing and it also showed a lot of duplicate content. I know how to fix it but my question is this: Is it better to remove the second language folder, fix it and then put it back up after a few months or just fix it now as it is and wait for G to come back and index the new links?
Technical SEO | | Silviu0 -
Crawling image folders / crawl allowance
We recently removed /img and /imgp from our robots.txt file thus allowing googlebot to crawl our image folders. Not sure why we had these blocked in the first place, but we opened them up in response to an email from Google Product Search about not being able to crawl images - which can/has hurt our traffic from Google Shopping. My question is: will allowing Google to crawl our image files eat up our 'crawl allowance'? We wouldn't want Google to not crawl/index certain pages, and ding our organic traffic, because more of our allotted crawl bandwidth is getting chewed up crawling image files. Outside of the non-detailed crawl stat graphs from Webmaster Tools, what's the best way to check how frequently/ deeply our site is getting crawled? Thanks all!
Technical SEO | | evoNick0