Bot Crawling issues
-
Dear all, Is this cache:www.subhavaastu.com is not working now with Google. Why it is not showing when the site is crawled. What were the new algorithms which Google is adopted? in my searches of all of my site internal links of www.SubhaVaastu.com, I observed only 404 instead of Google visiting time and date. After observing this 404 of each of my site links, I understand Google stopped crawling my site.
Some more examples are shown below with other websites:
cache:www.vastuwebsite.com ("NOT" showing when Google visited this site)
cache:www.vastuconsultantusa.com ("NOT" showing when Google visited this site)
cache:www.shubhavaastu.com ("NOT" showing when Google visited this site)
cache:www.subhavastu.com (Showing when Google visited this site)
To my surprise, I noticed that Google crawled latest links in my site, which I added a new link (https://www.subhavaastu.com/remove-negativity.html) just 10 days back, this new link was clearly crawled by Google. I typed "remove negativity subhavaastu", I saw the results with this new page in the SERP, but on the same way when I typed "cache:www.subhavaastu.com/remove-negativity.html", it is showing again 404.
what is happening with Google, is Google is following any new algorithms now. Is google changed any new concept? or, is my site is penalized in any case, I think it may not be, because if my site is penalized, then Google should not visit the new links and should not show the results with my site. Coming with my site, its pure from viruses, no malicious codes. Indeed it's an article based site, which has a good reputation. This domain is taken in the year 2003. We never spam anywhere we never did any wrong methods.
If my site is penalized, then is it manual penalized or bot penalized. I thoroughly checked webmaster tools and google console, I never found any notices or any note from Google.
Require experts analyzation on this doubt. Thanks in Advance.
-
Dear SeoMAN,
Thanks for your answer. I just tried at google, its shows normal, I just checked google search console and webmaster tools, there is nothing serious shows. Can it be possible to try to find out what was the main cause.
Is I taken the right image, is I searched exactly what you are asking about?
Many thanks.
-
Dear Vijay, Thank You.
You are an expert in this field, I respect your thought, but I am thinking there should be some other reason, that might cause of this problem. I am MOZ PRO subscriber, if there is, canonical issues, I will get such notifications. Long back, there are some canonical issues, we fix them.
Site is totally free from malware and viruses, I just now also observed Google webmaster tools and checked other ways, there is nothing viruses found in my site.
Can it be possible to find any other issues which caused this problem.
Many thanks.
-
You are asking quite a lot of questions here, basically if your site has been around for a reasonable amount of time and your page is accessible and can be crawled by search engines you should be able to Google your domain name (including the TLD) like "mydomain.com", if that doesn't show up you either have a serious technical error or you have had a penalty.
As for telling whether it is bot or manual Google search console should give you a bit more information. Personally as far as I can see a penalty is a penalty and needs to be overcome one way or another, whether it is automatic or manual is a bit beside the point!
-
It can be due to a lot of SEO/technical issues, Here are a couple of observations/suggestions:
- Your site works with https:// and with http:// both and also with www. and without www. , this might be an issue for duplicate content. Please make sure that you have only one version with or without www. or https://
- Check your website for malware and viruses just as a precaution.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Images added to website automatically become URLs - is this an issue?
Hello Mozzers! I've just been trawling through a website and noticed all of the images had their own URLs. There's a bespoke CMS and that's how it works with images... So out of 1447 urls, 1314 are images. Firstly, is this an issue / problem from an SEO perspective. If it is, how should I deal with it? Thanks in advance, Luke
Algorithm Updates | | McTaggart0 -
Google Webmaster Tools: Quality Issues on http://www.enakliyat.com.tr/
Specifically, we detected low-quality pages on your site which do not provide substantially unique content or added value. Examples could include thin affiliate pages, doorway pages, automatically generated content, or copied content. We encourage you to make changes to your site so that it meets our quality guidelines. Once you've made these changes, please submit your site for reconsideration in Google's search results. Google Webmaster Tool send me this message I think the low-quality pages is like the this http://www.enakliyat.com.tr/detaylar/bursa-fethiye-ucevler-nakliye-5834 page and we have so many pages like this... Example 1: http://www.enakliyat.com.tr/detaylar/evden-eve-nakliyat-5906 **Example 2 : **http://www.enakliyat.com.tr/detaylar/cekmekoy-izmit-5905 **Example 3: **http://www.enakliyat.com.tr/detaylar/evden-eve-nakliyat-5906 What should I do to these pages HELP 😞
Algorithm Updates | | iskq0 -
Could we run into issues with duplicate content penalties if we were to borrow product descriptions?
Hello, I work for an online retailer that has the opportunity to add a lot of SKUs to our site in a relatively short amount of time by borrowing content from another site (with their permission). There are a lot of positives for us to do this, but one big question we have is what the borrowed content will do to our search rankings (we normally write our own original content in house for a couple thousand SKUs). Organic search traffic brings in a significant chunk of our business and we definitely don't want to do something that would jeopardize our rankings. Could we run into issues with duplicate content penalties if we were to use the borrowed product descriptions? Is there a rule of thumb for what proportion of the site should be original content vs. duplicate content without running into issues with our search rankings? Thank you for your help!
Algorithm Updates | | airnwater0 -
Need some Real Insight into our SEO Issue and Content Generation
We have our site www.practo.com We have our blog as blog.practo.com We plan to have our main site in a months time from now as www.ray.practo.com The Issues - I will then need to direct all my existing traffic from www.practo.com to www.ray.practo.com Keeping in mind SEO and also since I will be generating new content via our Wordpress instance what are the best ways to do this so that google does not have difficulty in find out content 1. Would it be good if I put the Wordpress instance as ray.practo.com/ blog(wordpress instance comes in here in the directory) / article-url 2.Would it be better with www.practo.com / ray / blog/article-url I am using wordpress to roll out all our new SEO based content on various keywords and topics for which we want traffice - primary reasons are since we needed a content generation cms platform so that we dont have to deal with html pages and every time publish those content pages via a developer. Is the above - what soever I am planning to do in the correct manner keeping SEO in mind. Any suggestions are welcome. I seriously need to know writing seo based content on wordpress instance and have them in the urls is that a good idea? Or is only html a good idea. But we need some cms to be there so that content writers can write content independently. Please guide accordingly. Thanks
Algorithm Updates | | shanky10 -
HI, i am pro member of Seo moz, i just want to know , How much time take Seomoz for Crawl Diagnostics.
HI, i am pro member of Seo moz, i just want to know , How much time take Seomoz for Crawl Diagnostics. Because last evening i have changes in my website pages as seomoz suggested but i am not getting any changes in Crawl Diagnostics.
Algorithm Updates | | jaybinary0 -
SERPS Issue. Why is this happening?
All of our pages have been indexed by Google and we are ranking very well thus far. Here is an example of a search where, based on my searches, we rank well. Search for "Indiana ERISA Bond". You can search for any state name followed by "ERISA Bond" and we rank really well (Top of page 1). The pages that should show up for you are here: 1. https://www.suretegrity.com/surety-bonds/indiana/erisa-bond 2. https://www.suretegrity.com/surety-bonds/indiana/erisa My issue is with a different product we offer. Search for "Georgia Mortgage Lender Bond". The pages I would hope to come up, in order of priority, are: 1. https://www.suretegrity.com/surety-bonds/georgia/mortgage-lender 2. https://www.suretegrity.com/surety-bonds/georgia/mortgage-broker-banker-lender When I do my searches I show up at the bottom of page 2 and the URL that is being found is #2 above and not #1. I have spent a LOT of time thinking about this and am in need of some direction. What am I doing wrong? Thanks, Alex
Algorithm Updates | | dbuckles0 -
Issue: Having to Fight Product Marketing to Use Keywords
This is sort of a "DUH!" moment to me. I know everyone has come across this at some point in time and am interested in hearing how others deal with this. A little background: I was researching keywords for new menus and pages. Sometimes, people (product marketing in my case) do not give me a heads up on changes they want to make to pages and it is always a fight with them to change it. This is pretty normal for me and I am use to it. It is one of those things that they don't want to discuss it with you because they know you are going to critique their work. and, yes, change it for the good of the company. I had a co-worker say to me:
Algorithm Updates | | SmartBear
"We may have to start making [pages] meaningful to the human visitor than satify the bot army". My response was:
"What better way to make it [web page] meaningful to users by knowing which terms they search on the most in our industry? Keyword research is not just for Search Engines, it is actual live data as to what most people are searching. That is why I put such a high precedence on it and report on trends. You can bet that if 100,000 people are searching for [keyword], that is what they want to see when they search for it." Anyways, that is how I handled this particular event. I have several responses when these comments pop up from time to time. Usually it is something to the fact that they are not the ones who will get fired if leads drop via organic search, so we better try this. But today, I was feeling kind of spunky and decided to take another route. What are some of your responses to these types of remarks? Hopefully this will make for a good discussion.0 -
Is URL appearance defined by crawling or by XML sitemap
I am having a problem developing a sitemap because I have long URLs that are made by zend. They go like this: http://myagingfolks.com/professionals/20661/social-workers/pennsylvania-civi-stanger Because these URL's are long and are fed by Zend when I try to call them all up, to put on the sitemap, the system runs out of memory and crashes. Do you know what part of a search result, in google, say, comes from the URL? Would it be fine for me to submit to google only www.myagingfolks.com/professionals/20661. Does the crawler find that the URL is indeed http://myagingfolks.com/professionals/20661/social-workers/pennsylvania-civi-stanger or does it go with just what the sitemap tells it?
Algorithm Updates | | Jordanrg0