WP Events Calendar Creates URLs Too Long in Site Crawler
-
My travel/tourism site is on WP and using an Events plugin that ads a calendar of events to many pages. The MOZ crawler is indexing almost 46K links with a URL too long, but the site only has about 3.8K pages indexed in Google. I can tell MOZ is indexing the same pages over and over again but just adding a random calendar month and year. Here are some examples.
- https://www.visitcurrituck.com/four-day-stay/?full=1&long_events=1&country[0]=US&ajaxCalendar=1&mo=10&yr=2003
- https://www.visitcurrituck.com/four-day-stay/?full=1&long_events=1&country%5B0%5D=US&ajaxCalendar=1&mo=10&yr=2034
- https://www.visitcurrituck.com/beach-houses-family-time/?full=1&long_events=1&country%5B0%5D=US&ajaxCalendar=1&mo=1&yr=1873
Any advice on how to prevent MOZ from indexing this way? I don't believe that Google is seeing this also, but maybe they are. I just know my site has over 63K issues and I'm sure at least 75% or more is because of the way they are picking up on the events calendar.
Thanks!
-
Pretty sure it would be:
- User-agent: Rogerbot
- Disallow: /*?ajaxCalendar
- Disallow: /*&ajaxCalendar
That would pretty much handle OPs concern there
-
Hi there!
Thanks so much for the great question! Sorry for the trouble here. If you're looking to keep rogerbot from crawling and reporting issues on those calendar pages all together, you could block us from those pages using a disallow directive in your robots.txt file. It may look something like this:
User-agent: Rogerbot
Disallow: /?fullor whatever tag you're wanting to block us from crawling. I have a guide here that talks more about rogerbot and this technique that may be helpful, as well. I would recommend also checking your robots file in a robots.txt checker after making changes just so that no surprises come up!
If you'd like us to look into where we're finding these links or if you have any other questions, please feel free to email us at help@moz.com with the Campaign you're working.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL column of Link Explorer reports do not have the protocol (http or https)
I exported from Link Explorer. In the URL column, there is no protocol for each URL. Isn't the protocol part of what makes a url? I'm crawling each with http:// and https:// but it feels like this info could just be in the report. Am I missing a step?
Link Explorer | | Hyper-Dog0 -
After how long Moz show the matrices of a new domain
I want to know the Matrices of this site " Clash of lights ". Its a new domain how can i find its matrices like DA PA etc in Moz. Please guide me through it.
Link Explorer | | Muhammadahamd0 -
How can I get links indexed in Just Discovered Links, or Open site Explorer? (Because it aint Bit.ly or Feedly!)
Dear Moz & Forum, I've been on a bit of a merry-go-round with one of my clients. He knows enough about SEO to ask questions, and one thing he likes is to see the links that we are building inside Open Site Explorer. Now, of course we know that it takes a while for links to show, but that was ok, because we could just plug the links that we've built into a bit.ly link and then they'd show under Just Discovered Links in Open Site Explorer, and then boom, we're happy. It stopped working some time ago, and after the support team were a little unstuck I sent a hopeful tweet to @randfish. Here is the response: https://twitter.com/randfish/status/788412272129347584 The advice to use Bit.Ly was removed swiftly after this! Oops! The current Guidance is here: https://moz.com/help/guides/research-tools/open-site-explorer/justdiscovered It says: Want to see your new link in the Just-Discovered Links tab? An easy way to start seeing your link in the Just-Discovered section is to have it shared on Twitter and available on Feedly link. Feedly stopped supporting their own link shortner a few years ago. Also, when you share a link from Feedly on twitter it is just the same as sharing a normal tweet, because it shares the standard URL for the site not a link to view on Feedly. Or, maybe these two statements are independant of one another and they should both be done separately? I've done that too! I've also added the feed to recommended feeds inside fresh web explorer and that didn't work either! So, Moz and Help team please help me! How can I get links I've built for clients into Just Discovered Links? Thank you!
Link Explorer | | stevestretton11 -
GTLDs in Open Site Explorer
Curious to know if anyone has run into a similar problem - I entered a site containing a gTLD (.dog extensions) in OSE and it gave me an error (screenshot attached). I checked another tool (ahrefs.com) and was able to extract some data. Does anyone know if these type of domains are not included in OSE and/or why there was an error? Appreciate any insight! RSA04bYsL
Link Explorer | | ATShock0 -
Difference between Link Analysis and open site explorer
Hey, when i check my links in Link Analysis in Moz Analytics is see 45 Linking Root Domains. The manual states: " If you want to look at these pages in more depth, just click the export button. This will take you to Open Site Explorer so you can dig deeper into your link profile and export the data for review later." But when i go there, the open site explorer states i only have 31 Linking Domains. What could explain the difference?
Link Explorer | | RoderickG0 -
Open Site Explorer only showing 10 internal links, and 270 external links
Hi, I run the website www.abackpackerstale.com ,and for some reasons opensite explorer is only showing 10 internal links, and a 200 and something external links. How can I fix this as I am sure it is hurting my DA authority, and overall site score. Thanks! Stephen
Link Explorer | | backpackerstephen0 -
Is there some way to tell the Moz crawler not to crawl URL's with particular dynamic tags such as "?redirect-to:http//" ?
We are encountering an issue where the crawler is finding a ton of pages from our wordpress login url that has this dynamic tag in it to kinds of different blog entries. It's madness. I can't figure out what is causing these URLs to generate to be crawled in the first place! Does this sound familiar to anyone out there, any constructive suggestions? Robots text or maybe meta robots tags that would resolve this crawl issue?
Link Explorer | | RegistrarCorp0 -
How can I use Open Site Explorer to Identify why my own URL's are appearing as inbound links
I'm a new member of Moz, and am trying to understand why I am seeing my sites URL's in my Inbound Link results in Open Site Explorer. There are quite a few, and they are all product category URL's that are generated by my stores database. This is not happening with .html URL's. Any thoughts?
Link Explorer | | Artfx0