404s in Google Search Console and javascript
-
The end of April, we made the switch from http to https and I was prepared for a surge in crawl errors while Google sorted out our site. However, I wasn't prepared for the surge in impossibly incorrect URLs and partial URLs that I've seen since then.
I have learned that as Googlebot grows up, he'she's now attempting to read more javascript and will occasionally try to parse out and "read" a URL in a string of javascript code where no URL is actually present. So, I've "marked as fixed" hundreds of bits like
/TRo39,
category/cig
etc., etc....But they are also returning hundreds of otherwise correct URLs with a .html extension when our CMS system generates URLs with a .uts extension like this:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.html
when it should be:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.utsWorst of all, when I look at them in GSC and check the "linked from" tab it shows they are linked from themselves, so I can't backtrack and find a common source of the error.
Is anyone else experiencing this? Got any suggestions on how to stop it from happening in the future? Last month it was 50 URLs, this month 150, so I can't keep creating redirects and hoping it goes away.
Thanks for any and all suggestions!
Liz Micik -
Hi Liz,
What I would do as well is go with a solution around your robots.txt to make sure the crawlers will respect it and don't go on a hunch trying to find new URLs that are embedded somewhere else. Usually it's something you shouldn't worry about too much, it's just the crawler doing a good job trying to find more content/URLs on your site.
Martijn.
-
Google Search Console 404s can be very irritating.
What does your robots.txt file currently say?
You can remove all instances of .html in your htaccess file to help solve for this issue. More on that here: https://alexcican.com/post/how-to-remove-php-html-htm-extensions-with-htaccess/
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to check which site performing well in google organic?
Hi All, Is it possible to check sites via any tool which sites performing good in google organic? Any site ... Is it possible via Alexa? My Concern is majorly for UK Ecommerce site... Thanks!
Algorithm Updates | | pragnesh96390 -
Weird Bing Search Results
Hi all, I'm hoping someone can explain what's going on here, because after hours of searching I cannot find anyone having the same problem... We use Bing search to provide the site search functionality on our website and recently for a particular keyword search, the results include several pages which are not on our site: they are on completely different domains! You can replicate it by going to bing.com and using the "site:" operator together with that keyword. Again, results from other domains appear in amongst the pages on our site. I cannot find any other keywords which produce this same behaviour: every other keyword I have tried shows only results from our site. However, I obviously haven't tested absolutely every possible keyword combination. Bing isn't "padding" out the results or anything like that, because we have more than enough pages referencing this term on the site, and I'm at a total loss as to why this is happening. So, I suppose my question is: has anyone ever had this happen to them? And if so, what did they do about it? Many thanks, Dan
Algorithm Updates | | clarkovitch0 -
Creating Content for Semantic search?
Need some good examples of semantic search friendly content. I have been doing a lot of reading on the subject, but have seen no real good examples of 'this is one way to structure it'. Lots of reading on the topic from an overall satellite perspective, but no clear cut examples I could find of "this is the way the pieces should be put together in a piece of content and this is the most affective ways to accomplish it". **What I know: ** -It needs to answer a question that precludes the 'keyword being used' -It needs to or should be connected to authorship for someone in that topic industry -It should incorporate various social media sources as reference to the topic -It should link out to authoritative resources on the topic -It should use some structured data markup Here is a great resource on the important semantic search pieces: http://www.seoskeptic.com/semantic-seo-making-shift-strings-things/ ,but I want to move past the research into creating the content that will make the connections needed to get the content to rank. I know Storify is an excellent medium to accomplish this off page, but only gives no follow attribution to the topic creator and links their in. I am not a coder, but a marketer and creating the backend markup will really take me out of my wheel house. I don't want to spend all of my time flailing with code when I should be creating compelling semantic content. Any helpful examples or resources welcome. Thanks in advance.
Algorithm Updates | | photoseo10 -
Drop in Page Indexing, Small rise in Search Queries
Hello, I have a news based website so i am creating multiple new posts daily. I changed a lot of the site and got rid of old potentially duplicate content back in feb and had a sharp drop in pages indexed. I know this was because I removed a lot of pages though. However I still have a good 20,000 + pages on my site and my indexing has dropped a further three times since then. From 9,000 to 2,000 a coupe of months ago and then slowly down since April to just 133. It doesn't seem to have affected my search queries yet but surely will if it continues. I am really confused as to how this might happen & how to turn it around. We dont use any dodgy SEO tricks either.
Algorithm Updates | | luwhosjack0 -
Google Page Rank not improving
Hi All, I have a site live with a homepage rank of 5, Ever since relaunching (on the same domain) 6 months ago the inner page rank has remained at NA. Its crawled pretty consistently, Can anyone think of a reason this may be happening? www.glowm.com
Algorithm Updates | | thebluecubeuk0 -
Why does Google Alerts call my website a blog?
Our company started a WordPress blog about 14 years ago. It has since added a third-party forum, a user-submitted photo gallery, and a huge database of searchable products. We also have almost 4000 posts. With all that said, Google Alerts often lists our content under blogs rather than websites. Sometimes it shows up in both? Does anyone know what criteria Google uses for determining the type of content, and how we can signal to them that we are a website?
Algorithm Updates | | TMI.com0 -
If you got hit by a google update what you do first?
I have not been known the Panda. My site statistics knows it. I manage to endure and get back where i felt everytime with change of design, ad placement, content... I am very curious: if i did nothing, my site will turn back? why i kicked in every big update? Simply if you get hit by update, without any act, could your site healed?
Algorithm Updates | | MaxCrandale0 -
Recent changes to suggested search algorithm?
Our company recently had a "company name + scam" listing as #2 in suggested search, and yesterday, it miraculously disappeared. Has anyone else noticed similar changes in suggested search results? I hope it stick, I'm just trying to understand exactly what caused that 1 listing to vanish.
Algorithm Updates | | CareerBliss0