Remove false urls
-
Hi,
we are using expressionengine as our CMS. Every url that gets tracked, which does not lead to a real page, gets directed to the home page. This gives us a lot of duplicate content. For every wrong link, we get +1 in duplicate content.
Is there a way to list the false urls, so Moz does not use them anymore in the statistics, or that Google does not crawl them anymore?
Where would be the best place to do this?
All help appreciated,
Michel -
Hi,
thank you for taking the time to respond. The issues with broken paths are fixed, but those pages seem to be still alive in Google/Moz. I guess adjusting the robots file will be the best option here.
Thanks,
Michel -
It's been a few years since I was good at EE, but this problem plagued me as well.
There's no easy workaround to my knowledge, but erhaps the most direct way is to exclude those URLs via your robots.txt file, and then use Google's URL removal tool to wipe them from the index.
Adding them to your robots.txt file should take care of your duplicate content problems in Moz, and the removing them from Google's index should help with the rest.
Another option is to redirect these through your .htaccess file via 301. I won't get into the technicalities here, but it's another route to consider.
-
Hi Michel,
I guess my first suggestion would be to fix the issue at the source - so you no longer have the broken linking in the first place.
If that is not an option and you can access the broken pages in EE, you could use the rel="canonical" to specify the preferred version of the duplicate page or use "noindex" on those pages to tell Moz and Google to not crawl those pages anymore.
If you cannot access the broken pages, you could set up 301 redirects so that the homepage content isn't being duplicated on other URLs... but that the broken links just permanently redirect to the homepage URL.
Does that make sense?
Mike
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved How do I test a single URL?
Hi,
Getting Started | | imranh70
I ran the crawler, and it found a few errors on the site. I have fixed these errors. I don't want to run the crawler against the entire site again. I just want to run the test against a single page. How can I do that?0 -
Can't track my site, keep getting "Ooops. Our crawlers are unable to access that URL"
Hello, So i keep getting this message and I went to hurl.it and I get 200 response. But it appears its not my actual homepage bc it says the body is empty and in the title it says "COMING SOON" which is not what my actual homepage says. Does anyone know what this means?? Thank you in advance! Rena
Getting Started | | Palila-Studio0 -
Https not sticking on competitor URL
One of our competitors uses Https:// on thier search results and when we add and save it, it removes the s - we know their site ranks on specific keywords, we know this from manually searching before we got moz but moz is not showing them because it is looking for http:// not https:// How can i fix this. Also - when we first set it up, we did not add the www. and our analytics are setup as such so we show no traffic on our dashboard, but it wont let me edit to change it. Thanks
Getting Started | | HansSchlamadinger
Tom1 -
Oops! This does not appear to be a valid URL. Please try again.
I tried to create a campaign for avalonu.org which is my main website. It gave me "Oops! This does not appear to be a valid URL. Please try again." this error during campaign creation please help.
Getting Started | | Fatteh0 -
How do I edit my website URL in a Campaign?
The option to change the URL is grayed out. Is there a way to do this or do I have to create a whole new campaign?
Getting Started | | mathamatix0 -
Not a valid URL?
When ever I type in my website its tells me its not a valid URL . I have bought the URL www.buykickzonline.com. This was on my 90 day trial and it is not a good way to start my trial. nlGDl3f
Getting Started | | Teddy_Corl0 -
MozBar URL Flag shows wrong country
My site has been up for about 6 months, and is visible on google (http://www.forforce.com.br/). Even when searching for my exact brand name "For Force Suplementos", using google brazil from Curitiba (the stores physical city) I show up on page 3, so I'm obviously doing something wrong. I just today started looking into SEO. And we haven't done anything to earn a penalty. (There's no links actually anywhere but our facebook page to us, which I know is a problem I need to work on). Anyway, I just signed up for a MozPro subscription, and installed the MozBar. Looking at the On-Page elements, next to the URL, I see a U.S. flag, and am wondering if that is part of my problem. The site is physically hosted in the U.S., so maybe that's where the flag comes from (Quality) hosting is too expensive in Brazil). Is this something I should be worried about? I also am using CloudFlare to help speed up the site, which I don't think has a Brazil POP. Thanks in advance for any advice! -Eric
Getting Started | | ForForce0 -
How do I interpret Duplicate Content in a Crawl Report, when it only gives me a URL? How do I know what is duplicated on that page somewhere else?
I need help interpreting the Crawl Report for Duplicate Content. It gives me the URLs of pages that have duplicate content, but how do I know what content exactly is duplicated elsewhere? And how do I figure out where it is duplicated? Also, are there Moz Analytics articles or videos teaching you how to use each component of the analytics programs? Thanks!
Getting Started | | NancyBryan0