Website not crawled
-
i added website www.nsale.in in add campaign, it shows only 1 page crawled. but its working fine for other sites, any idea why it failed ?
-
This page references several reasons why only one page may be crawled. If it isn't any of these reasons, please contact our help desk. Thanks!
-
just looked at the other site you say is working... it has a much simpler robots file and no delay. just sayin' //
-
That's one of the bigger robots files I've seen in awhile..
You sure that crawl delay isn't messing things up? Not sure but that'd be my first guess.
-
its from day one. i have other domains on the same server www.litetab.in, its crawling properly. not sure why www.nsale.in alone doesnt work.
-
How long has it been up? Also, did you add the site from its root?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Permalink structure of an excising website
Hey everyone, We have a website - a business directory. It is a large directory with lots of B2B listings. So currently our URL structure for the categories look like this: dispensarystack.com/category/professional-services/ **We want to rank for the KW "dispensary professional services". ** Is it enough that we have the word dispensary in our brand name and in this way in the URL structure or is it better to make the slug: **dispensarystack.com/category/**dispensary-professional-services/ Or even closer to the root of the website: **dispensarystack.com/**dispensary-professional-services/ Thank you for your time and answers!
Technical SEO | | Nobody16154975014230 -
I can't crawl the archive of this website with Screaming Frog
Hi I'm trying to crawl this website (http://zeri.info/) with Screaming Frog but because of some technical issue with their site (i can't find what is causing it) i'm able to crawl only the first page of each category (ex. http://zeri.info/sport/) and then it will go to crawl each page of their archive (hundreds of thousands of pages) but it won't crawl the links inside these pages. Thanks a lot!
Technical SEO | | gjergjshala0 -
Changes to website haven't been crawled in over a month
We redesigned our website at http://www.aptinting.com a few months ago. We were fully expecting the crawl frequency to be very low because we had redesigned the website from a format that had been very static, and that probably has something to do with the problem we're currently having. We made some important changes to our homepage about a month ago, and the cached version of that page is still from April 2nd. Yet, whenever we create new pages, they get indexed within days. We've made a point to create lots of new blog articles and case studies to send a message to Google that the website should be crawled at a greater rate. We've also created new links to the homepage through press releases, guest blog articles, and by posting to social media, hoping that all of these things would send a message to Google saying that the homepage should be "reevaluated". However, we seem to be stuck with the April 2nd version of the homepage, which is severely lacking. Any suggestions would be greatly appreciated. Thanks!
Technical SEO | | Lemmons0 -
Canonicalization on my website
I am kind of new to all this but I would like to understand canonicalization. I have a website which when you arrive on it is www.mysite.com but once inside and flicking back to the homepage it reverts to www.mysite.com/index.html. Should I be doing something re canonicalization? If so what? Will the link juice be diluted by having two home page versions? Thanks
Technical SEO | | FCAbroad0 -
Website Redirects
Background information: We have a website (devicelock.com) which is currently our corporate website. The company use to operate under (ntutility.com) which is now being redirected to devicelock.com via a DNS Forward - 302 Redirect. The IT admin (a founder of the company) is reluctant to change it to a 301. The current flow is ntutility.com redirects to protect-me.com then redirects again to devicelock.com. When i search up Devicelock on google, it shows up as ntutlity.com. There is no devicelock.com homepage on google search. Question: Are there any negative implications about this? Is this hurting our SEO in any way? When i do link building, will this have any negative affects? Will my links for devicelock be attributed to devicelock.com?
Technical SEO | | Devicelock0 -
Index quickly a website? (Google,Bing..)
Hi, I would like to know what are the best practices in 2012 to index our website in less than 24 hours? (or less..) Thanks for your answer 😄
Technical SEO | | Probikeshop0 -
Redirecting website page to another
Hi there one of my old pages on my site is currently ranking for a phrases that I want to rank for on a new page I created. My old page from 1 year ago is ranking for 'Property Management Training' (it's a blog post dating 2011) I have cretaed a new main Page on my site and would like to rank for 'Property Management' as it's more relevant. What is the best suggestion to keep my ranking but have people go to my new page? 301 redirect old page to new page? Thanks,
Technical SEO | | daracreative1 -
Can Search engines crawl this page
Hi guys, To put a long story short we have had to make a copy of our site and put it on another domain so in essence there are 2 copies of our site on the web.What we have done is put a username and password on the homepage - http://www.ughhwiki.co.uk/ now i just want to be 100% sure that the search engines cannot crawl this? Thank you Jon
Technical SEO | | imrubbish0