How to find 20 hidden 404s
-
Hello,
We have like twenty 404s left to find. How do you find these when:
1. They don't show up in Google Webmaster Tools
2. They don't have any other internal or external pages linking to them.
3. They don't show up in site:domain.com (We have 9000 pages and only 600 show up - I fixed those out of the 600).
4. They are probably causing high bounce rates.
5. They're not in the sitemap
Thanks!
-
You should be able to crawl the entire site without increasing the RAM, once you buy the paid version.
-
Right in the paid version wont stop! You can see in their website that the limit of urls will be remove once you buy the license. http://www.screamingfrog.co.uk/seo-spider/licence/
It should catch everything, maybe you can contact their support just to make sure, they are very good on support over twitter.
-
Yes, but if it only check 397 in the free version is it going to stop there in the paid version as well. Just making sure.
Also, will it catch everything?
-
Yes I just run a craw for 500 000 urls
-
The free version checked 397 URLs. Will the purchased version check all 9000?
-
You can try crawl the site using screamingfrog, make sure you check the box “Check Links Outside Folder” in the spider menu.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL Parameter Setting Recommendation - Webmaster Tools, Breadcrumbs & 404s
Hi All, We use a parameter called "breadCrumb" to drive the breadcrumbs on our ecommerce product pages that are categorized in multiple places. For example, our "Blue Widget" product may have the following URLs: http://www.oursite.com/item3332/blue-widget
Intermediate & Advanced SEO | | Doug_G
http://www.oursite.com/item3332/blue-widget_?breadCrumb=BrandTree_
http://www.oursite.com/item3332/blue-widget_?breadCrumb=CategoryTree1_
http://www.oursite.com/item3332/blue-widget_?breadCrumb=CategoryTree2_ We use a canonical tag pointing back to the base product URL. The parameter only changes the breadcrumbs. Which of the following, if any, settings would you recommend for such a parameter in GWT: Does this parameter change page content seen by the user? Options: Yes/No
How does this parameter affect page content? Options: Narrows/Specifies/Other Currently, google decided to automatically assign the parameter as "Yes/Other/Let Googlebot Decide" without notifying us. We noticed a drop in rankings around the suspected time of the assignment. Lastly, we have a consistent flow of products that are discontinued that we 404. As a result of the breadcrumb parameter, our 404s increase significantly (one for each path). Would 800 404 crawl errors out of 18k products cause a penalty on a young site? We got an "Increase in '404' pages' email from GWT, shortly after our rankings seemed to drop. Thank you for any advice or suggestions! Doug0 -
B2B site targeting 20,000 companies with 20,000 dedicated "target company pages" on own website.
An energy company I'm working with has decided to target 20,000 odd companies on their own b2b website, by producing a new dedicated page per target company on their website - each page including unique copy and a sales proposition (20,000 odd new pages to optimize! Yikes!). I've never come across such an approach before... what might be the SEO pitfalls (other than that's a helluva number of pages to optimize!). Any thoughts would be very welcome.
Intermediate & Advanced SEO | | McTaggart0 -
How would you suggest finding content topics for this site?
Hello, How would you suggest finding content topics for this site: nlpca.com The end goal is signups for training seminars in San Francisco, California and Salt Lake City, Utah. In the future the seminars will move more towards life coaching trainings but right now they are mostly about NLP. NLP is a personal development field. Just looking for ideas for the process of finding topics for the most link-bait-heavy fabulous content. The owners of the site are authorities in the field. This is for both blog and article content. Thanks.
Intermediate & Advanced SEO | | BobGW0 -
Search engine simulators are not finding text on my website. Do I have a problem with Javascript or AJAX?
My website text is not appearing in search engine simulators. Is there a problem with the javascript? Or perhaps AJAX is affecting it? Is there a tool I can use to examine how my website architecture is affecting how the site is crawled? I am totally lost. Help!
Intermediate & Advanced SEO | | ecigseo0 -
Does having small ticket items (say under $1) available for customers to find & buy help or hurt our site?
I feel really silly asking this question to begin with, but... as a music store, we have a lot of "smalls" for products, like a guitar pick. We sell picks for $0.50 each, or a single clarinet reed at $0.79. Some believe this is too small, finicky, and cumbersome to have listed for sale on our site. To me, I wholeheartedly disagree with the notiion of excluding "smalls" for a plethera of SEO, customer service, & online SALES reasons... Also we offer USPS shipping to offer low shipping costs on small goods. Can I really be wrong about this? Thanks, Kevin
Intermediate & Advanced SEO | | Kevin_McLeish1 -
Webmaster or analytics can we find pages that are 404
Hi, Webmaster or analytics can we find pages that are 404 404 landing pages? From which source to what page and from page i get the 404 error when someone accesses my webpage? Need to know which pages are live on sites that are broken in my site Thanks
Intermediate & Advanced SEO | | mtthompsons0 -
Randomly Displayed Text: Hidden text issue?
I want to add some script to my site so that a given page publishes a different paragraph of text every time the page loads. Something like randomly displayed testimonials (but with more text). So, when you look at the page source, you would see all the text (e.g testimonial-1, testimonial-2, etc.), but the user would only see one paragraph randomly. Would this be considered hidden text (one code for search engine, one for use)? Is there a safe number of words you can do this with without setting off red flags? I appreciate the help.
Intermediate & Advanced SEO | | inhouseseo0 -
How to find pages hardest hit
I have been hearing that panda can penalize a website for low quality pages. I have run duplicate content check and done my best to go through the whole website. I hear many people talking about deleting hardest hit pages, or fixing hardest hit pages. My question is how can I find which pages on our website are hardest hit? Is there anyway to check a website for pages that might score low. We do have a ecom section to the website which I am concerned might be considered low quality for each product page. Any advice would be a great help.
Intermediate & Advanced SEO | | fertilityhealth0