Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Homepage canonical url with splash or not with splash? All other links are without but logo links with splash
Hello, There is so much contradicting information about the homepage canonical URL. Many websites have all the links without the trailing splash but their homepage URL still contains the splash. Now Moz is an example with this. Their urls don't have the splash, and their canonical does not have the splash. Why is it so and why so much different ways people have it?
On-Page Optimization | | advertisingcloud0 -
Should an internal link open in a new tab or in the same window?
Should an internal link open in a new tab or in the same window? Seems like this is an issue that has never had a definitive answer one way or the other. But I couldn't find any recent articles from reliable sources taking a stance and answering this question. Does anyone know if user engagement metrics (time on site, bounce rate, pages per visit) are impacted if a user clicks a link that opens in a new tab? Thanks!
On-Page Optimization | | NicheSocial0 -
Two Robots.txt files
Hi there Can somebody please help me that one of my client site have two robot.txt files (please see below). One txt file is blocked few folders and another one is blocked completely all the Search engines. Our tech team telling that due to some technical reasons they using second one which placed in inside the server and search engines unable to see this file. www.example.co.uk/robots.txt - Blocked few folderswww.example.co.uk/Robots.txt - Blocked all Search Engines I hope someone can give me the help I need in this one. Thanks in advance! Cheers,
On-Page Optimization | | TrulyTravel
Satla0 -
To use a fdqn or relative address in an internal link or not
Is it better to use my fdqn of http://1stclassdriving.co.uk/ or ~/ or ../ and then page name address for internal links or not? or does it just not matter for seo rankings?
On-Page Optimization | | brianw100 -
Google Webmaster Tools notice of detected unnatural links
Hi Has anybody else ever recieved this message from Google in Google Webmaster Tools and what action did you take to overcome the problem and get your position back - Dear site owner or webmaster of............... We've detected that some of your site's pages may be using techniques that are outside Google's Webmaster Guidelines. Specifically, look for possibly artificial or unnatural links pointing to your site that could be intended to manipulate PageRank. Examples of unnatural linking could include buying links to pass PageRank or participating in link schemes. We encourage you to make changes to your site so that it meets our quality guidelines. Once you've made these changes, please submit your site for reconsideration in Google's search results. If you find unnatural links to your site that you are unable to control or remove, please provide the details in your reconsideration request. If you have any questions about how to resolve this issue, please see our Webmaster Help Forum for support. Sincerely, Google Search Quality Team
On-Page Optimization | | AMG1000 -
Canonical links
My website is relatively new, January. We climbed steadily to 6th for our search term then overnight rocketed to 1st. This only lasted a week and have been stuck at 9th ever since. When I use the SEO Moz tools our site should theoretically be top...I only joined today btw. Anyway in Google webmaster tools I noticed it said I had duplicate title tags, when I checked to see what the pages were- it was my home page! Google also seems to have cached two versions of our homepage, the root domain and the Default.aspx page. Now I have fixed this canonical linking issue today (using canonical link tag and 301s) so time will tell but has anyone got any first hand experience of this issue? Was it a big factor? Thanks!
On-Page Optimization | | SplashBacksNI0 -
Too many links on page has a state directory how do i lessen this?
I dont mean how do i take away links on the page but i know right off i am going to have at least 50 right off the bat and with my footer and menu bar and such im up over the limit of 100. My seo guy took care of it on my one site and dont know how Site with issue is www.Preexistingconditioninsuranceplan.com/pcip-states one that was fixed that doesnt appear to be too many on seomoz was http://criticalillnesspolicies.com/insurance-coverage/united-states/ if anyone can inform or decypher the code please let me know
On-Page Optimization | | lance010 -
Does Google respect User-agent rules in robots.txt?
We want to use an inline linking tool (LinkSmart) to cross link between a few key content types on our online news site. LinkSmart uses a bot to establish the linking. The issue: There are millions of pages on our site that we don't want LinkSmart to spider and process for cross linking. LinkSmart suggested setting a noindex tag on the pages we don't want them to process, and that we target the rule to their specific user agent. I have concerns. We don't want to inadvertently block search engine access to those millions of pages. I've seen googlebot ignore nofollow rules set at the page level. Does it ever arbitrarily obey rules that it's been directed to ignore? Can you quantify the level of risk in setting user-agent-specific nofollow tags on pages we want search engines to crawl, but that we want LinkSmart to ignore?
On-Page Optimization | | lzhao0