Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How to check Robots.txt File
-
I want to know that how we are going to check that the Robots.txt File of the website is working properly. Kindly elaborate the mechanism for it please.
-
My site also have this problem please help.
-
Hi,
After you've read the article Nozzle recommended, use the https://technicalseo.com/tools/robots-txt/ tool to test whatever you put together.
-
Here is a great resource to answer your question: https://moz.com/learn/seo/robotstxt.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Top 10 SEO Experts in the World
Here are some of the top SEO experts in the world known for their contributions to the field, thought leadership, and innovative strategies: Rand Fishkin - Co-founder of Moz and SparkToro, widely known for his insights and contributions to SEO. Neil Patel - Co-founder of Crazy Egg, Hello Bar, and KISSmetrics, renowned for his SEO and digital marketing expertise. 3.. Brian Dean - Founder of Backlinko, famous for his advanced SEO strategies and detailed guides. Rafay Waqar - Co-founder of SEOServices and a LinkedIn influencer, he provide valuable insights into search engine algorithms and updates. Barry Schwartz - Founder of Search Engine Roundtable, known for his in-depth coverage of SEO news and trends. Aleyda Solis - International SEO consultant and founder of Orainti, recognized for her expertise in technical SEO and international SEO strategies. Bill Slawski - Director of SEO Research at Go Fish Digital, known for his deep understanding of search engine patents and algorithms. Vanessa Fox - Creator of Google Webmaster Central and author of "Marketing in the Age of Google," known for her expertise in technical SEO and analytics. Ann Smarty - Founder of Viral Content Bee and a well-known figure in the SEO community for her content marketing and link-building expertise. Cyrus Shepard - Former Head of SEO at Moz and founder of Zyppy, known for his comprehensive SEO knowledge and actionable insights.
International SEO | | cupll.rs11 -
Who is the best SEO expert in the World?
Hey everyone, i am creating a blog post on Top SEO Experts in the World. I need your recommendation who is in the top 10 list? Your suggestions is highly appreciated for me. Thanks!
International SEO | | gxpl090 -
Moving from single domain to multiple CCTLDs
Hi, I have a website targeting 3 markets (and therefor 3 languages). I was currently using a single domain with each market being targeted in the following format: www.website.com/pl
International SEO | | cellydy
www.website.com/de
www.website.com/hu It's clear to me by looking at organic results, that in my industry (Real Estate) Google is putting a large emphasis on local businesses and local domains. Top 10 organic results for all my keywords in all markets have country specific CCTLDs. I decided to migrate from a single domain strategy to a multi domain strategy. I own the domains. The new structure is www.website.com/pl -> www.website.pl
www.website.com/de -> www.website.de
www.website.com/hu -> www.website.hu All the website have been added to google search console and 301 redirects are in place and working correctly. The pages are all interlinked and have rel=alternate to each other. The sitemaps are all done correctly. My question is how do I tell Google about this. The change of address feature only works for changing one domain to one other domain. It's been a week and the old www.website.com domain is still showing up (even considering 301 redirects). Or do I just need to be patient and wait it out? Any tips?0 -
Baidu Webmaster Tools: How to setup in "Site Properties" the field "Affiliate subject"?
Hi ,
International SEO | | lcourse
finally I managed to setup my site in Baidu Webmaster Tools with the help of a freelance staff member in China. Site is verified and sitemap submitted. In section "Site Properties", field "Affiliate subject" I can't figure out after extensive search what I need to setup here for a foreign company without any presence and without company registration in China. Anybody can help? When I click on this field, it indicates "Site association subject is a necessary link for mobile resources to enter search." so my site will not show up in mobile results without it? Grateful for any tips on how to resolve this piece of puzzle of the baidu setup.
Thanks0 -
"Duplicate without user-selected canonical” - impact to SERPs
Hello, we are facing some issues on our project and we would like to get some advice. Scenario
International SEO | | Alex_Pisa
We run several websites (www.brandName.com, www.brandName.be, www.brandName.ch, etc..) all in French language . All sites have nearly the same content & structure, only minor text (some headings and phone numbers due to different countries are different). There are many good quality pages, but again they are the same over all domains. Goal
We want local domains (be, ch, fr, etc.) to appear in SERPs and also comply with Google policy of local language variants and/or canonical links. Current solution
Currently we don’t use canonicals, instead we use rel="alternate" hreflang="x-default": <link rel="alternate" hreflang="fr-BE" href="https://www.brandName.be/" /> <link rel="alternate" hreflang="fr-CA" href="https://www.brandName.ca/" /> <link rel="alternate" hreflang="fr-CH" href="https://www.brandName.ch/" /> <link rel="alternate" hreflang="fr-FR" href="https://www.brandName.fr/" /> <link rel="alternate" hreflang="fr-LU" href="https://www.brandName.lu/" /> <link rel="alternate" hreflang="x-default" href="https://www.brandName.com/" /> Issue
After Googlebot crawled the websites we see lot of “Duplicate without user-selected canonical” in Coverage/Excluded report (Google Search Console) for most domains. When we inspect some of those URLs we can see Google has decided that canonical URL points to (example): User-declared canonical: None
Google-selected canonical: …same page, but on a different domain Strange is that even those URLs are on Google and can be found in SERPs. Obviously Google doesn’t know what to make of it. We noticed many websites in the same scenario use a self-referencing approach which is not really “kosher” - we are afraid if we use the same approach we can get penalized by Google. Question: What do you suggest to fix the “Duplicate without user-selected canonical” in our scenario? Any suggestions/ideas appreciated, thanks. Regards.0 -
Help! Choosing a domain for a European sub-brand when working as a partner in North America
Background: Let's say there's a European company ABC.com, they have some presence in the US already for a lot of product brands in a certain space (let's say they make widgets). ABC Co gets 1,600 searches a month and all of that volume centers around the widgets they are known for. ABC Co purchases a company that makes gears, let's call it Gears Inc (gears.com). Gears Inc. was known for making gears in Europe, but their brand is not known in the US (search volume 0). Ideally, I would keep the Gears Inc. brand and build up the presence in the US, separating it from ABC Co. ABC Co wants to maintain their brand and eliminate Gears Inc. But we've received permission to keep the Gears brand for bringing that product to the US ... we will have an uphill battle building up the brand recognition, but at least it won't get lost in what ABC Co is already known for in the US. (ie: we don't want calls for widgets). Domain Situation: ABC Co. has redirected gears.com (DA 1) to a subdomain: {gearmakers}.abcco.com (DA 66) ... they have agreed to place a landing page under that 301 that links to the regional domains (theirs in the EU and ours in the US/North America). They are unwilling to let us use or purchase gears.com OR 301 gears.com directly to our domain. What we're trying to do: build Gears Inc. as a recognizable brand when someone searches "gears inc", this domain would rank first create a simple "brand domain" that a less-tech-savvy users could easily navigate to needs to have recognition in US, Canada and Mexico
International SEO | | mkretsinger
I don't know if this helps or provides anything more? The question is what do we use as our domain name? Any feedback is appreciated!0 -
Blocking domestic Google's in Robots.txt
Hey, I want to block Google.co.uk from crawling a site but want Google.de to crawl it. I know how to configure the Robots.txt to block Google and other engines - is there a fix to block certain domestic crawlers? any ideas? Thanks B
International SEO | | Bush_JSM0 -
When I upload my app in chrome web store, it shows this error - "no manifest found in package please make sure to put manifest at the root directory of the zip package". Please explain me the process of putting manifest file.
Chrome Web store give error "no manifest found in package please make sure to put manifest at the root directory of the zip package"
International SEO | | SameerBhatia0