Is there a Modern GSiteCrawler
-
We have a site with 1000+ URLs. We'd like to index all of the pages capturing URL's, Page Titles, etc with 100% accuracy. Which SEOmoz tool would be best at this? PS - in the old days we used the likes of GSiteCrawler. Thanks for your direction.
-
Yes, actually the 'Crawl Diagnostics' tool that comes with SEOmoz Pro does let you export to CSV. There are just two limitations:
-
It isn't in real-time (and I've typically leaned towards instant gratification ).
-
It may not find every page (to give you a real life example, one of my many experimental sites has approximately 1,000,000 pages .... in checking the Crawl Diagnostics export just now, SEOmoz found 10,319). But for smaller sites that our agency works with, it's pretty much spot-on.
-
-
Thanks Corey. We'll investigate all of the helpful recommendations you made.
Also just confirming, in your experience doesn't SEOMoz's "SEO Web Crawler" do what we are looking for?
-
Sadly SEOmoz (or any SEO tool) never seems to solve more than 5% of the puzzle. But I think Screaming Frog will probably do what you're asking (plus, it's free). If not that, I like SEO PowerSuite's Website Auditor a lot, and always seems to pull back denser data than most alternatives (but isn't free).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My posts are ignored by google not sure why
I have been investigating this problem for some time now and there must be a technical problem. My posts seem to be ignored by google. For example, if i put this title of my article in google, other sites come up instead of my website Lincolnshire Caravan Owners Struggle To Compete With Haven https://www.in2town.co.uk/skegness-news/lincolnshire-caravan-owners-struggle-to-compete-with-haven/ I am trying to find out what technical problem I have that is stopping google displaying my post. Can anyone advise me on what tools to use and how to find out what is going wrong
Technical SEO | | headlinesplus0 -
Google Not Picking Up Posts
I am trying to work out why from March 4th Google is not seeing my posts. Our google impressions have dropped from 8,000 to 40. If you put in the full article name with speach marks it does not find it, and instead shows the home page in google. We have not had any warnings. We did have work done on our site but nothing else i could think of to cause this. Can anyone let me know what may have caused this. All articles are original
Technical SEO | | headlinesplus0 -
What Tools Do I USe To Find Why My Site No Longer Ranks
Hi, I made the mistake of hiring a freelancer to work on my website [in2town.co.uk](link url) but after having a good website things went from bad to worse. The freelancer was kicked off the platform due to lots of compliants from people and creating backdoors to websites and posting on them. It cost me money to have the back door to our site closed. I then found lots of websites were stealing my content through the rss feed. Two of those sites have now been shut down by their hosting company. With all these problems I found in Feb that the hundreds of keywords that I ranked for had vanished. And all the ones that were in the top ten for many years have also vanished. When I create an article which includes https://www.in2town.co.uk/skegness-news/lincolnshire-premier-inn-staff-fear-for-their-jobs/ they cannot be found in Google. Normally before all these problems, my articles were found straight away. If I put in the title name Lincolnshire Premier Inn Staff Fear For Their Jobs and then add In2town in front of it, then instead of the page coming up with the article, it instead shows the home page. Can anyone please advise what tools i should be using to find out the problems and solve them, and can anyone offer advice please on what to do to solve this.
Technical SEO | | blogwoman10 -
Surge in spammy links
Hi, Our website www.foodjet.com has recently seen a huge amount of spammy incoming links to non-exisiting URLS: They all target pages that lead to a 404 and which clearly do not exist on our website. Since they have started to appear our DA has plummeted. I have already disavowed some domains, but more re-appear just as fast. I have also checked if our site has been hacked, which does not seem to be the case. What am I missing? And/or what can I do?
Technical SEO | | FoodJEt0 -
FAQ Schema Markup
I was wondering which blog posts would qualify for an FAQ Schema markup. For instance, we have a blog post which is more like a Q&A interview with our customer where our product gets mentioned several times. Would we get dinged for including our product name in the markup? First of all does that kind of blog posts even qualify for the markup? Example of the blog post: https://www.revulytics.com/blog/qa-techsmith-snagit-strategy-lead-daniel-foster
Technical SEO | | revulytics0 -
Internal link structure for my loan website
Hi folks. I own a Norwegian consumer loan/financing website, which has been monetized with links. I've created various silos for my content, according to what I believe is most relevant to the user.
Technical SEO | | llevy
However, as a result each article now has a sidebar list, which in turn links to all other articles within the same category (silo). As you can see here, it has about 30 links in the sidebar: forbrukslån.no/beste-lån. With 30 articles in a silo, that corresponds to over 900 internal links, in just one silo alone. I wonder if this could be hurting me SEO wise? I know G cares a lot about relevance and user experience. So I have a feeling it could be interpreted as spammy. Reason I did this in the first place, is that the header links are also being repeated on all pages, without any issue. T4FHxHw0 -
Google will index us, but Bing won't. Why?
Bing is crawling our site, but not indexing it, and we cannot figure out why -- plus it's being indexed fine in Google. Any ideas on what the issue with Bing might be? Here's are some details to let you know what we've already checked/established: We have 4 301’s and the rest of our site checks out We’ve already established our Robots is ok, and that we are fixing our site map/it's in fine shape We do not see anything blocking bingbot access to the site There is no varnish or any load balancers, so nothing on that end that would be blocking the access We also don't see any rules in the apache or the .htaccess config that would be blocking the access
Technical SEO | | Alex_RevelInteractive0 -
DISQUS COMMENTS backlinks-good for seo? YES/NO?
DISQUS COMMENTS backlinks-good for seo? YES/NO? I have just started commenting on "powered by disquus" websites in the Disqus comments box and left a link to my website in the name field! Having googled whether Disqus comments backlinks are any good for seo purposes i have discovered that there is a 50/50 view on the subject with some people saying they are a "goldmine" for getting high PR backlinks and others saying they are a waste of time because googlebot cannot read Java. My own experience of commenting on Disqus powered websites is that wordpress blogs powered by disqus comments ARE INDEXED by GOOGLE and the "BACKLINK IS IN THE SOURCE OF THE PAGE" When i comment on normal websites using the Disqus comment system i have found that my Disqus comments ARE NOT indexed by Google and there IS NO BACKLINK in the page source! Has anybody got any views on whether Disqus comments backlinks are any good?
Technical SEO | | Freebetsuk2