Site Map Problems or Are They?
-
According to webmaster tools my Sitemap contains urls which are blocked by robots.txt
Our site map is generically generated and encompasses all web pages, whether I have excluded them using the robots.txt file
As far as I am aware this has never been an issue until recently. Is this hurting my rankings and how do I fix it?
Secondly, webmaster tools says there is over 5,000 error/warnings on my site map. But site map is only 1,400 or so pages submitted. How do I see what is going on?
-
Thanks for the robots.txt help - kind of what I figured it's just never said anything about it before.
Now for the 5000+ errors/warnings for the site map, it only allows me to see a few of them - how do I see the rest? the issue with the robot.txt problem is all i can see right now
-
Usually, you would only put links you want indexed into the sitemap.xml file. GWT is just letting you know some of those links you posted are being blocked by robots (and wont be indexed). Shouldn't hurt your rankings if you don't want to index those pages anyways.
Check out the Optimization > Sitemaps > # errors link in GWT for more details.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moving site from html to Wordpress site: Should I port all old pages and redirect?
Any help would be appreciated. I am porting an old legacy .html site, which has about 500,000 visitors/month and over 10,000 pages to a new custom Wordpress site with a responsive design (long overdue, of course) that has been written and only needs a few finishing touches, and which includes many database features to generate new pages that did not previously exist. My questions are: Should I bother to port over older pages that are "thin" and have no incoming links, such that reworking them would take time away from the need to port quickly? I will be restructuring the legacy URLs to be lean and clean, so 301 redirects will be necessary. I know that there will be link juice loss, but how long does it usually take for the redirects to "take hold?" I will be moving to https at the same time to avoid yet another porting issue. Many thanks for any advice and opinions as I embark on this massive data entry project.
Technical SEO | | gheh20130 -
Sitemap.xml Site multilang
HI all, I have some questions about multilang sitemap.xml. So, we use the same domain subdirectories with gTLDs example.com/pt-br/
Technical SEO | | mobic
example.com/us/
example.com/es/ How should I do the sitemap.xml in this case? I thought of three alternatives: Should I do a sitemap_index.xml to each lang and make categories for these sitemaps? Examples:
http://www.example.com/pt-br/sitemap_index.xml
http://www.example.com/en/sitemap_index.xml
http://www.example.com/es/sitemap_index.xml Should I do only one sitemap_index.xml covering all categories of all languages ? Examples:
http://www.example.com/sitemap_index.xml
http://www.example.com/pt-br/sitemap_categorias_1.xml
http://www.example.com/es/sitemap_categorias_1.xml
http://www.example.com/us/sitemap_categorias_1.xml Should I do a sitemap setting all multilang? <url><loc>http://www.example.com/us/</loc>
<xhtml:link <br="">rel="alternate"
hreflang="es"
href="http://www.example.com/pt-br/"
/>
<xhtml:link <br="">rel="alternate"
hreflang="us"
href="http://www.example.com/us/"
/>
<xhtml:link <br="">rel="alternate"
hreflang="pt-br"
href="http://www.example.com/pt-br/"
/></xhtml:link></xhtml:link></xhtml:link></url> Thanks for any advice.0 -
Is a micro site the way to go?
Hello, a client has asked us today to quote for how much it would cost them to get a micro site built. A Google employee has told them that because their current URL doesn't include .co.uk or.com it is simply: brandname.word that it will be harder for them to get their website to rank. My understanding is that micro sites aren't a good solution for any problem as Google doesn't like them. Would it be better for them to buy a .co.uk (they are a UK company) url and then redirect the url to their current website or is there a better solution? Many thanks
Technical SEO | | mblsolutions0 -
What do you think about my new site?
Hi everyone, I'm looking for a review for my new site www.interlive.it Could you please let me know what do you think about the work that I did for my site. I'll be very happy to receive your suggestions. Regards, Mike
Technical SEO | | salvyy0 -
How can i do SEO For Ecommerce site
I am doing SEO for my WP blog but now I am starting my recently launch an eCommerce site where I am selling electronics products. I want to know how can I do the SEO so at least I can top 10 position for my google India. Second how can i avoid duplicate content about copying manufacture contents. Please help
Technical SEO | | chandubaba0 -
Site maps, Is there any benefit?
I have a relatively simple and small site (60 pages). All of it is crawlable and there is nothing I want non follow. So, is there any real benefit to a sitemap since Google can get to all the site anyway? Do they give the site more credence or something because it's there? I guess as an aside, are there any favorite sites that will generate a site map? Thanks!
Technical SEO | | Banknotes0 -
Will rankings for my micro site rank better if I 301 redirect it to my main site?
This is my first time asking so I will try to be as clear as possible. Ok, I have a micro site that is an (exact match domain) and the domain is a couple 3-4 years old and ranks very well for several search terms. The main two terms it ranks for are like this. houses for rent in XXXXX XXXXX homes for rent (XXXXX equals a city name) The issue is this site has no backlinks, zero advanced SEO, I only did basic optimization to it when i set the site up. Even site structure, url structure all are not good.
Technical SEO | | Robbie8299
The only page I have ever even seen rank is the main root url. But with all that the site does really good in the top 1-2 results for key search terms. Now, I have a main site that is a very big site that has steadily been climbing in search terms every month with great backlinks, optimized for the city and all.
It currently ranks on second page for the listed search terms listed above. What I want to do is 301 redirect this microsite to my city page on my main site that is much better optimized for the key city terms.
The 301 redirect would point this "root domain" (mymicrosite.com) to my city page that looks like this. www.mymaindomain.com/city/XXXXXXX If I do this will Google rank my main URL city page as well as it ranks this microsite with zero links, seo, etc, etc. What happens if it does not? Will I be able to turn off the 301 redirect and keep the microsite rankings? My main reason for wanting this is I want this city page to rank well and I only want to optimize one site instead of both. Any help would be great!0 -
Google and QnA sites
My website has a QnA site - a bit like this one except it's not private to premium members. It is a page with a left colomn for category links and it has a list of recently asked questions, each question is a link to view the full question and answers etc. Does google know this is a QnA ? Or will it say - hey, there are far too many links on this page, tut tut. Is there anything I can do to help it understand what the page is.
Technical SEO | | borderbound0