Scraping a Password Protected Page
-
Hey Everyone,
I'm looking for some help regarding my website.
I have multiple pages on my site that are password protected (we only give the password to potential clients)....
Somehow, when doing a google search, i found a directory that has successfully scraped and pulled all of the information from the password protected pages and is displaying this info on their site...
This is very bad for us as obviously this information is critical and not intended to be shared with the rest of the world
Does anyone have any advice on how to prevent my password protected pages from being scraped/crawled?
Thanks!
-
Hi there, as Takeshi said, it is difficult to answer this without more information (especially how you are implementing password protection on these pages.) Are you able to offer any information at all, or did you figure this out? We'd love an update so we can help you resolve this.
Christy
-
How are you implementing the password protection? What website is it for? It's hard to say anything without knowing the specifics.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I prevent duplicate page title errors from being generated by my multiple shop pages?
Our e-commerce shop has numerous pages within the main shop page. Users navigate through the shop via typical pagination. So while there may be 6 pages of products it's all still under the main shop page. Moz keeps flagging my shop pages as having duplicate titles (ie shop page 2). But they're all the same page. Users aren't loading unique pages each time they go to the next page of products and they aren't pages I can edit. I'm not sure how to prevent this issue from popping up on my reports.
Technical SEO | | NiteSkirm0 -
Paginated pages are being indexed?
I have lots of paginated pages which are being indexed. Should I add the noindex tag to page 2 onwards? The pages currently have previous and next tags in place. Page one also has a self-referencing canonical.
Technical SEO | | WTH0 -
If the order of products on a page changes each time the page is loaded, does this have a negative effect on the SEO of those pages?
Hello, a client of mine has a number of category pages that each have a list of products. Each time the page is reloaded the order of those products changes. Does this have a negative effect on the pages' rankings? Thank you
Technical SEO | | Kerry_Jones2 -
Page not indexed but still has a PageRank, how?
http://www.optiproerp.com/products.aspx page is not indexed in Google but still has a PageRank of 1. How? Regards
Technical SEO | | IM_Learner0 -
How do I 301 redirect a number of pages to one page
I want to redirect all pages in /folder_A /folder_B to /folder_A/index.php. Can I just write one or two lines of code to .htaccess to do that?
Technical SEO | | Heydarian0 -
Where to put Schema On Page
What part of my page should I put Schema data? Header? Footer? Also All pages? or just home page?
Technical SEO | | bozzie3114 -
Event Landing Pages not ranking
Hi there I need to optimize the website of a club/concert venue. The site isn't bad and has authority, but the event pages don't seem to rank and I'm unsure about the reason. There is an overview page of the events: http://www.kaufleuten.ch/events/ What happens currently when clicking on a specific event (on "WEITER", top right of each event) is that users get redirected to a hashtag page by jQuery. The href of "WEITER" itself links to another landing page (which is IMO the one we should see ranking for the specific event). Here is a concrete example: Look at the event "Tanz & Konzert: Andreas Vollenweider, Seven & ROKPA-KIDS" on /events by clicking on "WEITER", you get directed to http://www.kaufleuten.ch/events/#2790/andreas-vollenweider the actual "WEITER" link in the source code though, points to the landing page http://www.kaufleuten.ch/event/andreas-vollenweider/ This seems to be done by an AJAX load: jQuery loads a DIV with the ID "ajax-content". Apparently, this is the code responsible for it: $(„.link“, click(function() {
Technical SEO | | zeepartner
el.find('.wrapper').load(target+' #ajax-content', function() {
});
return false;
}); I know the site has good authority and should rank well. however, the event landing pages never seem to appear, but only the page /events is ranking: SERP
(Strangely, when using the site command, the event page suddenly appears above: SERP. (But I have never seen this in a "normal search query", even though we are the organisers and should at least be among the top 5). Now my question: Does Google consider this AJAX load to be some sort of cloaking? (because the href in the code is different to you actually end up by clicking "WEITER"). Will the landing pages begin to rank if we disable this AJAX load? Or should we stick to hashtags and not even create landing pages? (but then, we will have no control over title tags of specific events, right?) Thanks for your help, I'm a bit lost here as my JS knowledge is meagre... Cheers,
Phil0 -
Backlinks to home page vs internal page
Hello, What is the point of getting a large amount of backlinks to internal pages of an ecommerce site? Although it would be great to make your articles (for example) strong, isn't it more important to build up the strength of the home page. All of My SEO has had a long term goal of strengthening the home page, with just enough backlinks to internal pages to have balance, which is happening naturally. The home page of our main site is what comes up on tons of our keyword searches since it is so strong. Please let me know why so much effort is put into getting backlinks to internal pages. Thank you,
Technical SEO | | BobGW0