Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Robots blocked by pages webmasters tools

Intermediate & Advanced SEO

2

7

637

Log in to reply

mihoreis last edited by
- a mistake made in software. How can I solve the problem quickly? help me.
XTRjH
1 Reply Last reply
Reply Quote 0
mihoreis @GastonRiera last edited by

Ok,

Updated.

I need to wait 1 week then

Thank You, Gaston
1 Reply Last reply
Reply Quote 0
Gaston Riera @mihoreis last edited by

That sitemap line place it at the end of the file.
like this:

User-agent: *
Disallow:

Sitemap: https://site.com/sitemap.xml

Best luck!
1 Reply Last reply
Reply Quote 1
mihoreis @GastonRiera last edited by

OK,

I did,

can you check from the picture.

Thank You.

zZd9P
1 Reply Last reply
Reply Quote 0
Gaston Riera @mihoreis last edited by

its better to set it as I've discribed before:

User-agent: *
Disallow:

It would take over a week, depending in scale of the website.
1 Reply Last reply
Reply Quote 1
mihoreis @GastonRiera last edited by

Yes,

in the shape of the photo,

is it correct? re-scanning takes a long time?

c4ZCy
1 Reply Last reply
Reply Quote 0
Gaston Riera last edited by

Hello there!

Have you identified all pages that you want to be blocked?
If you just do not want any page to be blocked, just edit you robots.txt as:

User-agent: *
Disallow:

For more information, check these resources:
Cómo bloquear URLs con robots.txt - Google Webmasters
About /robots.txt - robotstxt.org

Best Luck!
GR
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Structure of HTML Page

Hello, Is is true that search engine give more value to some part of the page than other ? Is only the main content considered ? or are the other also given weight but very small weight ? If I have div in the main content as those considered par of the main content or no ? Thank you,
Intermediate & Advanced SEO | | seoanalytics

1
Ecommerce category pages

Hi there, I've been thinking a lot about this lately. I work on a lot of webshops that are made by the same company. I don't like to say this, but not all of their shops perform great SEO-wise. They use a filtering system which occasionally creates hundreds to thousands of category pages. Basically what happens is this: A client that sells fashion has a site (www.client.com). They have 'main categories' like 'Men' 'Women', 'Kids', 'Sale'. So when you click on 'men' in the main navigation, you get www.client.com/men/. Then you can filter on brand, subcategory or color. So you get: www.client.com/men/brand. Basically, the url follows the order in which you filter. So you can also get to 'brand' via 'category': www.client.com/shoes/brand Obviously, this page has the same content as www.client.com/brand/shoes or even /shoes/brand/black and /men/shoes/brand/black if all the brands' shoes happen to be black and mens' shoes. Currently this is fixed by a dynamic canonical system that canonicalizes the brand/category combinations. So there can be 8000 url's on the site, which canonicalize to about 4000 url's. I have a gut feeling that this is still not a good situation for SEO, and I also believe that it would be a lot better to have the filtering system default to a defined order, like /gender/category/brand/color so you don't even need to use these excessive amounts of canonicalization. Because, you can canonicalize the whole bunch, but you'd still offer thousands of useless pages for Google to waste its crawl budget on. Not to mention the time saved when crawling and analysing using Screaming Frog or other audit tools. Any opinions on this matter?
Intermediate & Advanced SEO | | Adriaan.Multiply

0
Drop in Indexed pages

Hope everyone is having an Awesome December! I first noticed a drop in my index in the beginnings of November. My site drop in indexed pages from 1400 to 600 in the past 3-4 weeks. I don't know the cause of it, and would like the community to help me figure out why my indexing has dropped. Thank you for taking time out of your schedule to read this.
Intermediate & Advanced SEO | | BSC

0
Why Is Google Webmaster Tools Pulling Zero Keyword Data?

I just linked a Google Webmaster Tools account to Google Analytics for a client, and Search Engine Optimization reports are showing up in Google Analytics as enabled, but there is zero keyword data, landing page data, etc., in the reports themselves. Has anyone encountered this?
Intermediate & Advanced SEO | | yoursearchteam

0
I've seen and heard alot about city-specific landing pages for businesses with multiple locations, but what about city-specific landing pages for cities nearby that you aren't actually located in? Is it ok to create landing pages for nearby cities?

I asked here https://www.google.com/moderator/#7/e=adbf4 but figured out ask the Moz Community also! Is it actually best practice to create landing pages for nearby cities if you don't have an actual address there? Even if your target customers are there? For example, If I am in Miami, but have a lot of customers who come from nearby cities like Fort Lauderdale is it okay to create those LP's? I've heard this described as best practice, but I'm beginning to question whether Google sees it that way.
Intermediate & Advanced SEO | | RickyShockley

2
Will I mess with Authorship if I setup multiple client websites under my Webmaster tools login?

I currently have a dozen client websites or so that I have setup under my Webmaster tools login. Should I put them each separately under their own webmaster tools, then just add me as a user? Is the way I'm doing it now messing with Authorship?
Intermediate & Advanced SEO | | daviddischler

0
Articles | Posts or Pages

I'm looking to add a number of feature rich articles to educate, promote best practice and provide useful all round advice. My reasearch to date on the pros and cons have drawn me to writing articles as a page, as per
http://yoast.com/articles/wordpress-seo/#pagesvsposts My Question, etc.
I would like to get an outside opinion on the above. Additionally, how can you find out the site structure of a website.
i.e. what are blog posts or static pages
Intermediate & Advanced SEO | | Mark_Ch

0
My landing page changed in google's serp. I used to have a product page now I have a pdf?

I have been optimizing this page for a few weeks now and and have seen our page for up from 23rd to 11th on the serp's. I come to work today and not only have I dropped to 15 but I've also had my relevant product page replaced by this page . Not to mention the second page is a pdf! I am not sure what happened here but any advice on how I could fix this would be great. My site is www.mynaturalmarket.com and the keyword I'm working on is Zyflamend.
Intermediate & Advanced SEO | | KenyonManu3-SEOSEM

0