Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Robots.txt Tester - syntax not understood

2

3

1749

Log in to reply

JamesHancocks1 last edited by

I've looked in the robots.txt Tester and I can see 3 warnings:

There is a 'syntax not understood' warning for each of these.

XML Sitemaps:
https://www.pkeducation.co.uk/post-sitemap.xml
https://www.pkeducation.co.uk/sitemap_index.xml

How do I fix or reformat these to remove the warnings?

Many thanks in advance.
Jim
1 Reply Last reply
Reply Quote 0
JamesHancocks1 @Martijn_Scheijbeler last edited by

I'm to give that a go Martijn.

The text "XML Sitemaps" is in there and flagas as an error. Does this need to be reformatted as well or deleted?

Kind regards,
James.
1 Reply Last reply
Reply Quote 0
Martijn_Scheijbeler last edited by

Hi James,

The right syntax is:

Sitemap: https://www.pkeducation.co.uk/post-sitemap.xml
Sitemap: https://www.pkeducation.co.uk/sitemap_index.xml

When you retry it should show up as working.

Martijn.
1 Reply Last reply
Reply Quote 2

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Robots.txt allows wp-admin/admin-ajax.php

Hello, Mozzers!
I noticed something peculiar in the robots.txt used by one of my clients: Allow: /wp-admin/admin-ajax.php What would be the purpose of allowing a search engine to crawl this file?
Is it OK? Should I do something about it?
Everything else on /wp-admin/ is disallowed.
Thanks in advance for your help.
-AK:
Technical SEO | | AndyKubrin

2
I have two robots.txt pages for www and non-www version. Will that be a problem?

There are two robots.txt pages. One for www version and another for non-www version though I have moved to the non-www version.
Technical SEO | | ramb

0
Good robots txt for magento

Dear Communtiy, I am trying to improve the SEO ratings for my website www.rijwielcashencarry.nl (magento). My next step will be implementing robots txt to exclude some crawling pages.
Does anybody have a good magento robots txt for me? And what need i copy exactly? Thanks everybody! Greetings, Bob
Technical SEO | | rijwielcashencarry040

0
2 sitemaps on my robots.txt?

Hi, I thought that I just could link one sitemap from my site's robots.txt but... I may be wrong. So, I need to confirm if this kind of implementation is right or wrong: robots.txt for Magento Community and Enterprise ...
Sitemap: http://www.mysite.es/media/sitemap/es.xml
Sitemap: http://www.mysite.pt/media/sitemap/pt.xml Thanks in advance,
Technical SEO | | Webicultors

0
Is sitemap required on my robots.txt?

Hi, I know that linking your sitemap from your robots.txt file is a good practice. Ok, but... may I just send my sitemap to search console and forget about adding ti to my robots.txt? That's my situation: 1 multilang platform which means... ... 2 set of pages. One for each lang, of course But my CMS (magento) only allows me to have 1 robots.txt file So, again: may I have a robots.txt file woth no sitemap AND not suffering any potential SEO loss? Thanks in advance, Juan Vicente Mañanas Abad
Technical SEO | | Webicultors

0
Is there a limit to how many URLs you can put in a robots.txt file?

We have a site that has way too many urls caused by our crawlable faceted navigation. We are trying to purge 90% of our urls from the indexes. We put no index tags on the url combinations that we do no want indexed anymore, but it is taking google way too long to find the no index tags. Meanwhile we are getting hit with excessive url warnings and have been it by Panda. Would it help speed the process of purging urls if we added the urls to the robots.txt file? Could this cause any issues for us? Could it have the opposite effect and block the crawler from finding the urls, but not purge them from the index? The list could be in excess of 100MM urls.
Technical SEO | | kcb8178

0
Can I Disallow Faceted Nav URLs - Robots.txt

I have been disallowing /*? So I know that works without affecting crawling. I am wondering if I can disallow the faceted nav urls. So disallow: /category.html/? /category2.html/? /category3.html/*? To prevent the price faceted url from being cached: /category.html?price=1%2C1000
and
/category.html?price=1%2C1000&product_material=88 Thanks!
Technical SEO | | tylerfraser

0
Robots.txt File Redirects to Home Page

I've been doing some site analysis for a new SEO client and it has been brought to my attention that their robots.txt file redirects to their homepage. I was wondering: Is there a benfit to setup your robots.txt file to do this? Will this effect how their site will get indexed? Thanks for your response! Kyle Site URL: http://www.radisphere.net/
Technical SEO | | kchandler

0