Huge number of crawl anomalies and 404s - non- existent urls
-
Hi there,
Our site was redesigned at the end of January 2020. Since the new site was launched we have seen a big drop in impressions (50-60%) and also a big drop in total and organic traffic (again 50-60%) when compared to the old site.
I know in the current climate some businesses will see a drop in traffic, however we are a tech business and some of our core search terms have increased in search volume as a result of remote-working.
According to search console there are 82k urls excluded from coverage - the majority of these are classed as 'crawl anomaly' and there are 250+ 404's - almost all of the urls are non-existent, they have our root domain with a string of random characters on the end. Here are a couple of examples:
root.domain.com/96jumblestorebb42a1c2320800306682
root.domain.com/01sportsplazac9a3c52miz-63jth601
root.domain.com/39autoparts-agency26be7ff420582220
root.domain.com/05open-kitchenaf69a7a29510363
Is this a cause for concern? I'm thinking that all of these random fake urls could be preventing genuine pages from being indexed / or they could be having an impact on our search visibility. Can somebody advise please?
Thanks!
-
Unlikely, as long as they're returning 404 errors you should be OK. Maybe update your disavow file and you should be good to go!
-
Thanks for your reply.
I’m new to the business and I’ve found that that the old website had a spam attack, all of these fake urls are from the old pages (as they have 301s).
There are 82,000 crawl anomalies from these fake/spam URLs and around 200 404s. None of the fake /spam urls have been indexed. Could this be having a negative effect of search visibility/DA or rankings?
Thanks!
-
It's tough to say without seeing the site. Overall it's unlikely if you don't use that string anywhere. We usually see it more for broken relative URLs. Maybe a third party site is using that string.
-
Thanks for your reply, would broken urls from the internal linking structure explain the random characters? e.g. root.domain.com/96jumblestorebb42a1c2320800306682
We've never had any page content/urls relating to 'jumblestore'.
Thanks!
-
From what I can tell, this probably isn't the reasons for the drops. I'd go back and ensure that any URLs that changed are 301 redirecting to the correct destination URL. I'd also ensure that no pages that were associated with high volume keywords no longer exist.
For your issue, Google is likely finding some broken URLs, possibly from your internal linking structure. Perform a crawl of the site and see if you can find "Inlinks" to those broken pages. If so, you can work with dev to eliminate the issue.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Use existing page with bad URL or brand new URL?
Hello, We will be updating an existing page with more helpful information with the goal of reaching more potential customers through SEO and also attaching a SEM campaign to the specific landing page. The current URL of the page scores 25 on Page Authority, and has 2 links to it from blog articles (PA 35, 31). The current content needs to be rewritten to be more helpful and also needs some additional information. The downsides are that it has an "bad" URL- no target keyword and uses underscores. Which of the following choices would you make? 1. Update this old "bad" URL with new content. Benefit from the existing PA. -or- 2. Start with a new optimized URL, reusing some of the old content and utilizing a 301 redirect from the previous page? Thank you!
Technical SEO | | XLMarketing0 -
Should we change our URLs for SEO benefit?
Hi, I'm currently covering a maternity marketing role at i-escape and one our main objectives is to increase organic traffic to the website. i-escape has a selection of hand-picked boutique hotels, villas, lodges, guesthouses and apartments for people to discover and book. At the moment each hotel page URL follows this structure: https://www.i-escape.com/hotelname We'd like to change this to include some searchable words in the URL dependent on the type of hotel. For example: https://www.i-escape.com/boutique-hotels/hotelname or https://www.i-escape.com/boutique-apartments/hotelname If we do go ahead, we know we need to make sure all old style URLs canonically redirect to the new style. Is having the keyword in the URL important enough for us to change over 1500 URLs on the website? We have quite a high quality links pointing to these hotel pages URLs. Also, will this help us with navigation/user journeys/crawls as there will be a /boutique-hotels/hotelname rather than just /hotelname? Thanks so much all! Clair
Technical SEO | | iescape0 -
To include / at the end of a URL or not
Hi I have recently noticed my site works with / and the end of a URL and without. I wanted to know if there is any SEO impact on this? Will it be seen as 2 different pages? if so what is the best option to go for www.mydomain.com/page/ or www.mydomain.com/page Thanks E
Technical SEO | | Direct_Ram0 -
Is it Detrimental to Repeat a Word in Our URL?
Hey guys! We run a tour company in Barcelona. Our company name is Barcelona Experience. We're customizing our URL's to include keywords which can be found in all the important areas on the page (title tage, meta descp., etc).
Technical SEO | | BarcelonaExperience
We want to change "www.barcelonaexperience.com/bike-tours" to "www.barcelonaexperience.com/barcelona-bike-tours"
We're worried the repetition of "barcelona" could be a bad thing. True, or not true? Thanks!0 -
Home page URL
Hi, I work on this site: http://www.towerhousetraining.co.uk/about-us. This is the home page URL. Should this be 301'd to: http://www.towerhousetraining.co.uk? I have created a site map, which I submitted to Google Webmaster Tools, which includes these URL's: /about-us, /training-we-offer & /contact-us. There are a total of 3 pages on the website. Webmaster tools has only indexed 2 out of 3 pages. I think this is something to do with the /about-us URL, as when I do a site: search, these pages appear: www.towerhousetraining.co.uk/, /training-we-offer & /contact-us. I am not sure why Google has indexed the home page as www.towerhousetraining.co.uk/ and not /about-us? Is it a bad idea in general not to have your homepage as your root domain? I added a to the homepage, but am wondering if this was the right thing to do? Any help would be appreciated.
Technical SEO | | CWseo0 -
Second URL
Hi We have a .com and a .co.uk Main website is .co.uk, we also have a landing page for the .com If we redirect the .com to the .co.uk, will it create duplicate content ... May seem like a silly question, but want to be sure that that the visitors cant access our website at both urls, as that would be duplicate content Thanks in advance John
Technical SEO | | Johnny4B0 -
URL structure
Hi, I am in the process of having a site created which will focus on the Xbox 360, PS3, Wii and PS3 Vita. I would appreciate some advice when it comes to the URL structure. Each category mentioned above will have the following subsections News
Technical SEO | | WalesDragon
Reviews
Screenshots
Trailers Would the best url structure be? www.domain.com/xbox-360/news/news-story-headline
www.domain.com/ps3/reviews/ps3-game-name Thanks in advance for your help and suggestions.0 -
Including spatial location in URL structure. Does subfolder number and keyword order actually matter?
The SEOMoz On-Page report for my site brings up one warning (among others) that I find interesting: Minimal Subfolders in the URL My site deals with trails and courses for both races and general running. The structure for a trail is, for example: /trails/Canada/British-Columbia/Greater-Vancouver-Regional-District/Baden--Powell-Trail/trail/2 The structure for courses is: /course/28 In both cases, the id at the end is used for a database lookup. I'm considering an URL structure that would be: /trail/Baden-Powell-Trail/ca-bc-vancouver This would use the country code (CA) and sub-country code (BC) along with the short name for the region. This could be good because: it puts the main keyword first the URL is much shorter there are only 3 levels in the URL structure However, there is evidence, from Google's Matt Cutts, that the keyword order and URL structure don't matter in that way: See this post: http://www.seomoz.org/q/all-page-files-in-root-or-to-use-directories If Matt Cutts says they aren't so important then why are they listed in the SEOMoz On-Page Report? I'd prefer to use /trail/ca-bc-vancouver/Baden-Powell-Trail. I'll probably do a similar thing for courses. Is this a good idea? Thoughts? Many thanks, in advance, for your help. Cheers, Edward watch?v=l_A1iRY6XTM watch?v=gRzMhlFZz9I
Technical SEO | | esarge0