How is my competition causing bad crawl errors and links on my site
-
We have a compeditor who we are in a legal dispute at the moment, and they are using under hand tactics to cause us to have bad links and crawl errors and i do not know how they are doing it or how to stop it.
The crawl errors we are getting is the site having two urls together, for example www.testsite.com/www.testsite.com and other errors are pages that we do not even have or pages that are spelt wrong or have a dot after the page name.
We have been told off a number of people in our field that this has also happened to them and i would like to know how they are doing it so we can have this stopped
Since they have been doing this our traffic has gone down by half
-
Hi no there is only me who deals with the site. I have put copyright notice on the site but i will read about authorship accorss the site.
-
Hi Diane,
I like the way Ryan thinks! ...but I am hoping we won't have to go to the length of having to resort to a content bomb.
The lesson in a situation like this is to realize that being a good white hat SEO unfortunately means needing an understanding of some of the tactics used by black hats or maybe just a little help from some friends
Since there is obviously an issue with your content being copied, the first thing I would do is to implement Authorship markup across your entire site. By doing this you ensure that any content that is "borrowed" is immediately "outed" to the search engines because it doesn't have an external link from your Google profile page, which acts as verification that you are in fact the author of the content. Matt Cutts and Othar Hansson have given a really easy rundown on implementation in this Google Webmaster Help video
For the moment though, it would be better if you can avoid making any changes to the site until we can identify all of the issues in play.
BTW ... I think I have an inkling of what might be going on here ... is there a programmer or designer who is or has been involved in the development of the site besides yourself?
In the meantime, I'm continuing with a diagnostic based on the information we have and will let you know as soon as I have confirmed my suspicions or otherwise.
Hang in there,
Sha
-
May I suggest planting a "bomb" in your content?
Most thieves are lazy. Rather then create content themselves they steal from others. Their laziness is dependable.
Take a look at their copies and determine what content is and is not being stolen. If they are copying everything including the HTML code and meta tags, you can add canonical tags to your site and other helpful code.
If they are not copying the meta tags, you can add fake content and use the noindex, nofollow tag to protect your site, but provide content which otherwise would cause a site to be removed from Google's index. When they steal the content, it wont have the noindex tag and the site would get nailed.
There are many other possibilities but I am confident you can outsmart them if you try. In addition, be sure to report the site(s) to Google: http://www.google.com/support/bin/static.py?page=ts.cs&ts=1114905
Another idea is to copyright your work, thereby protecting it and giving you legal proof the content is yours.
-
thanks for this, will send private message. we have had to redo the site so many times with new content because the content keeps on being stolen by a franchise group who then pass it on to their franchisees.
-
Hi Diane,
Ryan's response is spot on and his suggestions are excellent.
If you can provide the URL(s), then we can take a look and see exactly what is going on with the referring page(s).
If you don't want to share the information publicly in the Q&A, you can private message each of us through your SEOmoz profile page.
If you ever think that someone has access to edit pages on your site without your permission, the first thing to do is to check with your service provider whether there are any active ftp accounts that you are unaware of. I have seen situations before where people have managed to get a "back door" set up and then it is as simple as logging in and changing pages without your knowledge.
Given that this involves a legal dispute, if we can do a proper diagnosis and trace the source of the errors (or if there happens to be a back door in place), then you would be able to:
- issue a cease and desist
- better secure the server against unauthorized access
- Ban any ip addresses identified as malicious
Hope that helps,
Sha
-
You mentioned these are crawl errors. Are you using the SEOmoz crawl report? If so, please look at the "referrer" field. It will offer the page on your site which is providing the bad URL.
If you are willing to share the referring page, we can take a look and possibly provide more detail.
Anyone can create bad links to your site which can appear in Google or Bing WMT. Only someone with the ability to add content on your site can create crawl errors. Either your site is open to user generated content and someone created a bad link, or someone with access to your web server created the content.
-
will do thanks
-
If iit is happniong to many, then maybe its a reason not to suspect them, but like i say go to Bing WMT and have a look at where the links ae comming from.
-
the reason why i know they are behind it, is because other companies in the field of the website have had the same problems and after doing research for our legal team over this matter, we spent time speaking to over 40 people in the field of the website and found it happened to them aswell.
These people are cowboys but hopefully it will all be sorted out soon.
-
I would look in Bing WMT to see where the links are comming from for a start.
i must say also, If you dont know how they are doing it, then maybe you dont know if they are doing it., it may be somthing quite inocent.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Very wierd pages. 2900 403 errors in page crawl for a site that only has 140 pages.
Hi there, I just made a crawl of the website of one of my clients with the crawl tool from moz. I have 2900 403 errors and there is only 140 pages on the website. I will give an exemple of what the crawl error gives me. | http://www.mysite.com/en/www.mysite.com/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | | | | | | | | | | There are 2900 pages like this. I have tried visiting the pages and they work, but they are only html pages without CSS. Can you guys help me to see what the problems is. We have experienced huge drops in traffic since Septembre.
Technical SEO | | H.M.N.0 -
CSS background image links bad for seo?
On one of the websites I manage SEO for, the developers are changing how our graphical links are coded. They're basically coding in such away where there is no anchor text and no alt tag, so for example: So there's no anchor nor alt context for Google's crawler. How badly will this affect SEO, or is it extremely minimal and I shouldn't worry about? Thanks in advance.
Technical SEO | | JimLynch0 -
Can increase in crawl errors in GWT) be caused by input fields and jquery?
Dear Mozzerz We took over www.urgiganten.dk not long ago and last week we opened up for indexation, after having taken the old website down for a couple of months. One week after opening for indexation we saw a huge increase in crawl errors.Google is discovering some weird links to e.g http://www.urgiganten.dk/30-garmin-urremme/ which returns a 404. In GWT we are told that we are linking to this url from http://www.urgiganten.dk/garmin-urremme. But nowhere on http://www.urgiganten.dk/garmin-urremme will you find this link. However you will find the following script in the source code, which is the only code part that contains "/30-garmin-urremme/":Can it be true that google take the id and adds it to our tld to form a url? We have seen quite a lot of these errors not only on Urgiganten.dk but also some of our other websites!
Technical SEO | | urgiganten0 -
Unnatural links from your site
Hi, 24 February got this penalty message in Google webmaster tool. Google detected a pattern of unnatural, artificial, deceptive, or manipulative outbound links on pages on this site. This may be the result of selling links that pass PageRank or participating in link schemes. Already removed all the link on the blog and sent reconsideration request to Google spam team. But request is rejected. Please help me on this or share link with me on same case. Thanks,
Technical SEO | | KLLC0 -
Pointing Other URL to My Site? Good or bad for ranking.
A few years ago I purchased a few keyword rich domain names and set up some satellite sites. Spammy I now know. What should I do now? I own the domain names for at least another 3 years. Should I point them to my main site or would that hurt my main site ranking?
Technical SEO | | caisson0 -
Ajax Optimization in Mobile Site - Ajax Crawling
I'm working on a mobile site that has links embedded in JavaScript/Ajax in the homepage. This functionality is preventing the crawlers for accessing the links to mobile specific URLs. We're using an m. sub-domain. This is just an object in the homepage with an expandable list of links. I was wondering if using the following solution provided by Google will be a good way to help with this situation. https://developers.google.com/webmasters/ajax-crawling/ Thanks!
Technical SEO | | burnseo0 -
Trying to get google to know my site is a magazine site is this wrong
Hi, i have put a line to describe what my site is at the top of my site and i want to know if this is wrong or not. We have dropped frok being number one in google for lifestyle magazine to now number seven. Before we had to redo our site we were number one and then we dropepd to around number four when we finished the site and now we are number seven and i need to try and get back up there. To help google know we are a lifestyle magazine i have put a line at the top of the site and i want to know if this looks out of place and if i should take it down. i need advice on how to get google to know we are a lifestyle magazine and get back in the top five of google my site is www.in2town.co.uk any help would be great
Technical SEO | | ClaireH-1848860 -
Do we need to manually submit a sitemap every time, or can we host it on our site as /sitemap and Google will see & crawl it?
I realized we don't have a sitemap in place, so we're going to get one built. Once we do, I'll submit it manually to Google via Webmaster tools. However, we have a very dynamic site with content constantly being added. Will I need to keep manually re-submitting the sitemap to Google? Or could we have the continually updating sitemap live on our site at /sitemap and the crawlers will just pick it up from there? I noticed this is what SEOmoz does at http://www.seomoz.org/sitemap.
Technical SEO | | askotzko0