Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
WP Events Calendar Creates URLs Too Long in Site Crawler
-
My travel/tourism site is on WP and using an Events plugin that ads a calendar of events to many pages. The MOZ crawler is indexing almost 46K links with a URL too long, but the site only has about 3.8K pages indexed in Google. I can tell MOZ is indexing the same pages over and over again but just adding a random calendar month and year. Here are some examples.
- https://www.visitcurrituck.com/four-day-stay/?full=1&long_events=1&country[0]=US&ajaxCalendar=1&mo=10&yr=2003
- https://www.visitcurrituck.com/four-day-stay/?full=1&long_events=1&country%5B0%5D=US&ajaxCalendar=1&mo=10&yr=2034
- https://www.visitcurrituck.com/beach-houses-family-time/?full=1&long_events=1&country%5B0%5D=US&ajaxCalendar=1&mo=1&yr=1873
Any advice on how to prevent MOZ from indexing this way? I don't believe that Google is seeing this also, but maybe they are. I just know my site has over 63K issues and I'm sure at least 75% or more is because of the way they are picking up on the events calendar.
Thanks!
-
Pretty sure it would be:
- User-agent: Rogerbot
- Disallow: /*?ajaxCalendar
- Disallow: /*&ajaxCalendar
That would pretty much handle OPs concern there
-
Hi there!
Thanks so much for the great question! Sorry for the trouble here. If you're looking to keep rogerbot from crawling and reporting issues on those calendar pages all together, you could block us from those pages using a disallow directive in your robots.txt file. It may look something like this:
User-agent: Rogerbot
Disallow: /?fullor whatever tag you're wanting to block us from crawling. I have a guide here that talks more about rogerbot and this technique that may be helpful, as well. I would recommend also checking your robots file in a robots.txt checker after making changes just so that no surprises come up!
If you'd like us to look into where we're finding these links or if you have any other questions, please feel free to email us at help@moz.com with the Campaign you're working.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long will it take for google to reindex 301 redirected url and refresh search results with new urls.
We have set up 301 redirects for some subdomains which we are planning to use, but then we changed our focus to main domain. Google has already crawled some of the subdomain pages. As it was duplicate content over subdomains and main domain, we have set up 301 redirects for all subdomains to redirect to the main domain. We have set up 301 redirects a month ago, but some of the search results still show old URLs. Do you need something extra for google search results to refresh with main domain and remove subdomain URLs from google searches? We need rankings to be passed new URLs.
Link Explorer | Jun 11, 2018, 8:35 AM | swapnil120 -
Open Site Explore page titles show as "No Title"
This has been asked many times but I cannot find an answer. On Open Site Explorer, most URLs I enter show as "No title" which means you have to hover over the URL to see which page is being referred to. I know you'd want an example, so here's your own site Moz.com 🙂 CWrkxZQ
Link Explorer | Mar 13, 2017, 8:13 AM | clifra0 -
Rogerbot and Dotbot blocked by WP Engine hosting
We recently discovered a problem with our MOZ account which led to checking our host's settings. Turns out that WP Engine (who we love to death) automatically BLOCK Rogerbot and Dotbot from crawling sites they host. What? Yeah exactly expect I was cursing a little too. So we went through the steps of getting that sorted one bot at a time but a week on and we're still getting 403 returns via OSE. This is more of a discussion than a question expect what I would like to see is a force crawl button so that we can test that we're no longer serving a 403 error from our host MOZ. I thought WP Engine were huge and I'm surprised this has not come up before but there's no info on it anywhere. Is there a way to force a mozbots crawl??
Link Explorer | Oct 5, 2014, 4:52 AM | wearehappymedia0 -
Open Site Explorer
I have entered my site (patioenjoyment.com) a few different times into the OSE and it never seems to find it, MOZ has found it because it has done the crawl test for me a few times, found some issues and I have been working on those. But I assume that if it is able crawl my site it should find it to run the OSE. It has come back with "**It looks like we haven't discovered established link data for this URL yet." everytime. ** Thanks, Julie
Link Explorer | Sep 12, 2014, 1:13 PM | patioenjoyment0 -
Open Site Explorer Recently Discovered Links Question
Hey Mozzers, I have a question regarding the Open Site Explorer Recently Discovered Links. I know that I have receive follow link from a few sites that have more DA than my site. I am wondering why it is taking so long for OSE to discover these links? I double checked the sites to make sure that they don't have NOFOLLOW (sorry for the double negative), and the sites have linked to my site for over a month now. Any ideas? I'm not concerned about it because Search Engines have discovered those links. I am just curious why this is, or if it is normal. Cheers!
Link Explorer | May 6, 2014, 11:39 PM | evan890 -
Why Open Site Explorer is Showing 0 Total Internal Links?
Open Site Explorer is reporting "0" value under "Total Internal Links" as well as "Internal Followed Links" metrics for my website. I have a good interlinking of pages throughout the site.
Link Explorer | Apr 4, 2024, 11:24 AM | vivekrathore
Is there an issue with OSE (Open Site Explorer) crawler ? OR There are some crawling problems with our Website's structure ?
Can anybody please give an idea or recommendations on this ?1 -
Open site explorer
Hi, I am a little confused how open site explorer updates (not sure its working as aspected). It shows the recent discovered links but these then disappear and do not show in the main list (Inbound links) after a few days. Plus the links in the inbound links tab has not changed for nearly 10 weeks, my other tools are showing the new links..... Does it update on a certain data, if so where can i check the status? Thanks
Link Explorer | Feb 11, 2014, 10:33 AM | 720digital0 -
301 Redirects: How long for Google to recognize? How long for Moz/OSE to recognize?
I redirected a few pages that had funky URL issues (capitals and underscores and useless words) to the same page renamed with proper URLs. The sitemaps were changed also, and re-submitted. Like this: mydomain.com/Not_Pertinent_Words.html >>> mydomain.com/good-words.html Google seems to have found them and changed the search engine results listing in about 8 days. But it's been about a month now and Moz and OSE still have not transferred all the strength and link data from the old URLs to the new ones. Question 1: How long does it usually take Google to transfer all the link and strength data for a 301? Question 2: How long does it take Moz and OSE to do the same? Is there something I need to do to tell them about the changes?
Link Explorer | Apr 21, 2014, 8:39 PM | GregB1230