Repeated mysterious 404's from ancient site structure killing my rankings
-
Several years ago I changed my site structure to go from a flash based site to a blog based wordpress site. After doing so I went from page 1 to page 30 for my relevant search terms. I have employed people to help me track down the problem and I believe that they have narroed it to the existance of 404's being created from some unknown internal source. I have been for years getting links like this...
<colgroup><col width="792"></colgroup>
||
......regularly showing in webmaster tools, (this is from a top pages report from MOZ where there are hundreds also shown).
When I do a moz crawl of the site, none of these links show up. Therefore I have no way of finding the source of these links (they also do not show me the source in WMT as they should).
We have completely cleared the site and rebuilt it and although it is still only a couple of weeks in it still does not appear to have stopped them.
Does anyone have any way of helping me find the source of these mysterious 404's?
-
Why bother trying to clean anything up? If somewhere out there there are links to your domain, and they're 404'ing, just 301 them to new pages on your site! Capture that link juice, don't let it run out
-
Thanks for your reply EEE3
The ancient link says it is linked from another non existent ancient page that no longer exists and it is always first crawled and last detected on the day that it arrives.
eg. last crawled 4/23/14, first detected 4/23/14
http://www.dfphotographer.com.au/brisbaneweddingphotographer/2011/03/st-kilda-wedding......
linked from
http://dfphotographer.com.au/brisbaneweddingphotographer/index.php/2011/03/st-kilda-wedding.....
and
http://dfphotographer.com.au/brisbaneweddingphotographer/2011/03/st-kilda-wedding....
-
Thanks for your response Keri,
Being staff can you please tell me where does the top pages data come from? Is it from crawling my site (like a google spider) or is it sourced from google or somewhere else. How often is that data refreshed?
In answer to your response, I have tried both screaming frog and xenu and my nice clean site structure is all it picks up. None of the ancient messy site structure appears.
Have been through the list of domains looking for an old sitemap or something similar that may have been scraped off my site but after a long and arduous task could not locate any reference to any of these links that show up in top pages and webmaster tools (which says they are linked from other ancient pages - which I will expand on below)
We have looked at all the usual suspects - old sitemaps, plugins and rebuilt the site just in case we missed anything that was lingering around. I have had really good people looking at it who continue to do so it just never seems to go away.
-
In Webmaster Tools, when you click on the 404 and the popup window appears, what is showing in the Linked from tab?
-
I edited the post so the URLs didn't run together. Still not perfect, but a little easier to read.
I'm not exactly sure where those links are coming from. You might run a tool like Xenu Link Sleuth or Screaming Frog on your site to see if there is an internal linking widget gone awry. The other thought I have is to look at Open Site Explorer to see what sites are linking to you and if they're linking to any of those pages.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
WP Events Calendar Creates URLs Too Long in Site Crawler
My travel/tourism site is on WP and using an Events plugin that ads a calendar of events to many pages. The MOZ crawler is indexing almost 46K links with a URL too long, but the site only has about 3.8K pages indexed in Google. I can tell MOZ is indexing the same pages over and over again but just adding a random calendar month and year. Here are some examples. https://www.visitcurrituck.com/four-day-stay/?full=1&long_events=1&country[0]=US&ajaxCalendar=1&mo=10&yr=2003 https://www.visitcurrituck.com/four-day-stay/?full=1&long_events=1&country%5B0%5D=US&ajaxCalendar=1&mo=10&yr=2034 https://www.visitcurrituck.com/beach-houses-family-time/?full=1&long_events=1&country%5B0%5D=US&ajaxCalendar=1&mo=1&yr=1873 Any advice on how to prevent MOZ from indexing this way? I don't believe that Google is seeing this also, but maybe they are. I just know my site has over 63K issues and I'm sure at least 75% or more is because of the way they are picking up on the events calendar. Thanks!
Link Explorer | | CinivaAgency1 -
Moz's new Link Explorer, including our revamped index and DA/PA scores is now open to everyone!
Hey Moz Community, Link Explorer is now open to the public! Everyone can access it via a subscription or a free Moz ‘Community’ account. As you may know by now, the brand-new Link Explorer tool is primed to replace Open Site Explorer as Moz’s link building and analysis tool. The Link Explorer project is the result of an incredible amount of perseverance and hard work by the team, and we’re proud to be able to finally share it with you — we know it’s going to revolutionize how you approach link building and make your job easier. You can read more about the tool here in Sarah Bird’s announcement post. Because Link Explorer improves on almost every aspect of Open Site Explorer, the metrics have improved, too. That means you’re likely going to see some Domain Authority and Page Authority discrepancies between OSE’s index and Link Explorer’s index. We definitely suggest you use the new DA/PA from Link Explorer, as they’re more accurate and refresh daily rather than monthly, as was the case with OSE’s index. However, we also realize that many of you use these metrics to report to your clients and colleagues, and a sudden change or fluctuation could potentially make your job harder. Which DA is the real DA? The new DA is based on a much larger index that has many improvements, several of which are designed to make the index more like Google’s than ever before. You should consider moving towards the new DA (and the old DA won’t be updated after April 26th 2018, so the sooner the better). While there will be fluctuations as we improve the model and add features to the index, we expect it to remain largely stable and to be a far more accurate picture of a site’s authority according to how it’s seen by Google. Why is Link Explorer’s DA/PA considered better than OSE’s, and which should I trust? The larger link index with improved crawl selection allows us to produce a stronger model that includes a much larger proportion of the web. That being said, DA and PA should always be considered in the context of your competitors. A drop in PA or DA relative to the old OSE is of little concern if your competitors saw similar movement. Is Domain Authority/Page Authority an absolute score or a relative one? Both DA and PA are relative to the Internet as a whole. If Facebook acquired a billion new links, everyone’s PA and DA would drop relative to Facebook. Because of this, it’s always best to look at PA and DA in comparison to your competitors. What does a drop/raise in DA mean in Link Explorer vs OSE? How can I explain this to my clients when I’m reporting it? DA and PA should always be considered in the context of your competitors. A drop or raise in PA or DA relative to the old OSE is of little concern if your competitors saw similar movement. Reporting that your site has moved from a DA of 45 to a DA of 42 doesn’t tell the whole story, but reporting that your site has a DA of 42 while your main competitor moved from a 43 to a 37 shows that, relative to the sites you’re competing against in the SERPs, your site has significantly more authority and ranking power. What’s happening to MozTrust and MozRank and why, and what should I replace those with? The improvements to our DA/PA and Spam Score metrics now now account for more important nuances in helping you determine one site’s ability to rank higher than another. Because they no longer correlate with Google’s ranking model as well as they used to, MozRank and MozTrust are being deprecated for better metrics. Users should rely on Page Authority, Domain Authority, and Spam Score to determine the importance and quality of pages, domains, and links. I have historical data I use to help my clients benchmark their progress. What do I do now that DA is calculated differently? You should annotate any KPI changes referencing the change in DA and PA. However, most importantly, you should compare those changes to your competitors, as this will best show how strong your site’s authority is relative to the sites you’re competing against in the SERPs. We take updating our metrics very seriously, and our last major update to the model was 7 years ago. Users of Domain Authority and Page Authority can expect us to continue to produce steady, reliable metrics for the long haul, and only make changes to these metrics when we believe the benefits dramatically outweigh the stability of the metric. Do you have any questions about the new metrics? Anticipating a tough time reporting changes to clients or bosses? Metrics, features or functionality missing that you would want to see? Let us know in the thread, and we’ll work to find a good answer for you. Hope you enjoy the new Link Explorer product and the amazing new link index powering it. We are very excited to provide this valuable data to our community and customers.
Link Explorer | | IanWatson9 -
I crawled my site, but an old crawl report still is visible
I crawled my site recently, but an old crawl report still is still all I can see
Link Explorer | | Bigjim0 -
Why isn't OSE showing any of my links?
My domain uses a redirect of all traffic to https. The site is https://www.tallslimtees.com. I've been working on it this year and know there are several good, topical links coming in. But OSE shows nothing. Any idea why this would be the case? How can I see all of my links and the data on them?
Link Explorer | | DanDeceuster0 -
Site Mark-up is Abnormally Small
My site www.brightonsoundsystem.co.uk has been optimised for speed so I have minimised the code needed. Now if I put it through the OSE spam analysis it has a flag for "Site Mark-up is Abnormally Small". What ratio of visible text compared to mark-up code is being used to trigger this flag. Also as this is the only flag I have is ti worth the time fixing.
Link Explorer | | Brighton-Soundsystem0 -
Open Site Explorer is finding old html Files that havn't been on my site in two years... even after a 301 Redirect. HELP!
Hello!
Link Explorer | | morganlindsaycole
My problem started when I became aware that when I checked my backlinks for the past two years, it states that no backlinks have been found. When I ran a site analysis on SEMrush - No backlinks are found on the URL, or Domain. There are 7 Backlinks on the Root Domain and those were configured in 2012. I made a second domain www.columbusweddingphotographersreviews.comwhere I linked to my domain at www.morganlindsayphotography.com so I could test that google had crawled both websites and after, still no backlink was found. I have also been published on a dozen or so wedding websites that has linked to my website where they are follow links and still nothing. (http://www.brendasweddingblog.com/blogs/2015/2/23/an-elegant-fall-wedding-in-ohio-with-morgan-lindsay-photography) **Website Background- **
In 2012 I had two separate websites - One for Seniors that was an HTML website I build in Dreamweaver at www.morganlindsayphotography/seniors and another for Wedding Clients found at www.morganlindsayphotography.com/Wedding - (wordpress) I had a Splash page wish was found atwww.morganlindsayphotography.com. Two years ago when I became aware splash pages were frowned upon in Google, I combined the two websites and stayed with the Wordpress which was www.morganlindsayphotography.com/Wedding
Because I did not want users to have to go to www.morganlindsayphotography.com/Wedding to view my url, Godaddy moved my wordpress site from thewww.morganlindsyphotography.com/Wedding directory towww.morganlindsayphotography.com When I ran the Open Site Explorer with Moz I found after runningwww.morganlindsayphotography.com the TOP pages on this domain according to Page Authority are old HTML files from my senior website, as well as old Posts from when my wordpress site was found atwww.moragnlindsayphotography.com/Weddings
No current pots or pages are showing up besideswww.morganlindsyphotography.com I do run a cache management system to speed up my system and recently cleaned out my .htcacess folder and still had no luck. This is difficulty something **Last night I made a 301 Redirect in my htaccess for all the old links pointing to the new links as best as I could. My htacess folder looks like this.. BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]</ifmodule> END WordPress Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /Wedding http://www.morganlindsayphotography.com/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /Wedding/ http://www.morganlindsayphotography.com/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /about.html http://www.morganlindsayphotography.com/about-morgan-lindsay/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /app.html http://www.morganlindsayphotography.com/blog/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /experience.html http://www.morganlindsayphotography.com/senior-sessions/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /index.html http://www.morganlindsayphotography.com/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /senior.html http://www.morganlindsayphotography.com/ohio-senior-photographer/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /seniorsconstruction.html http://www.morganlindsayphotography.com/ohio-senior-photographer/ Permanent URL redirect - generated by www.rapidtables.com Redirect 301 /Wedding/2012/06/22/brittany-reis-jason-mcclaflin-tiffin-ohio-wedding/ http://www.morganlindsayphotography.com/holy-family-church-columbus-wedding/ After I ran the open site moz explorer and the www.morganlindsayphotography/Wedding was still there..0 -
Hi guys. My site, www.x-mini.com attained more links and got better alexa ranking. However, my DA and PA dropped. How can I explain this?
My site, www.x-mini.com attained more links and got better alexa ranking. However, my DA and PA dropped. How can I explain this? Siz5lkp.png
Link Explorer | | Dineshr840 -
OSE Stats - Number of Unique C IP's and PA/DA keep going up and down. Why?
For the past 3 months (if not more) I have been checking my sites stats on OSE and have noticed drastic decreases and increases in unique C-IP's, ranging from 600 to 1200. I have also noticed the PA & DA going up and down by 5 points. We have a talented team that work hard to market our business ethically with strong content and PR relationships, white-hat link building all the way, active social media and a dedicated on-site technical team. The data from MajesticSEO seems to be a lot more steady and consistent. Why is OSE showing drastic changes in 1 to 2 week intervals? Many thanks Ross
Link Explorer | | David_Connor0