What do you use for test rendering your dev site?
-
I'm redesigning our company ecommerce site and need to test render an infinite scroller to ensure that it is as SEO friendly as possible. My problem is that I cannot view it in Webmaster Tools since I am blocking the site from crawlers using robots.txt. I know I could simply unblock Google temporarily but I really would rather not make my dev site available to search engine crawlers.
-
Screaming Frog
-
Hi Tyler. You can use a text browser like Lynx as that's a close approximation of how Google "sees" your site. From: https://support.google.com/webmasters/answer/40349...
Use a text browser, such as Lynx, to examine your site. Most spiders see your site much as Lynx would. If features such as JavaScript, cookies, session IDs, frames, DHTML, or Macromedia Flash keep you from seeing your entire site in a text browser, then spiders may have trouble crawling it.
And a more robust list of guidelines is available here: https://support.google.com/webmasters/answer/35769.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Multiple sites using same text - how to avoid Google duplicate content penalty?
Hi Mozers, my client located in Colorado is opening a similar (but not identical) clinic in California. Will Google penalize the new California site if we use text from our website that features his Colorado office? He runs the clinic in CO and will be a partner of the clinic in CA, so the CA clinic has his "permission" to use his original text. Eventually he hopes to go national, with multiple sites utilizing essentially the same text. Will Google penalize the new CA site for plagiarism and/or duplicate content? Or is there a way to tell Google, "hey Google, this new clinic is not ripping off my text"?
Web Design | | CalamityJane770 -
Should i not use hyphens in web page titles? Google Penalty for hyphens?
all the page titles in my site have hyphens between the words like this: http://texas.com/texas-plumbers.html I have seen tests where hyphenated domain names ranked lower than non hyphenated domain names. Does this mean my pages are being penalized for hyphens or is this only in the domain that it is penalized? If I create new pages should I not use hyphens in the page titles when there are two or more words in the title? If I changed all my page titles to eliminate the hyphens, I would lose all my rankings correct? My site is 12 years old and if I changed all these titles I'm guessing that each page would be thrown in the google sandbox for several months, is this true? Thanks mozzers!
Web Design | | Ron100 -
The impact of using directories without target keyword on our Rankings
Hello all, I have a question regarding a website I am working on. I’ve read a lot of Q en A’s but couldn’t really find the best answer. For one of our new websites we are thinking about the structure of this website and the corresponding URL-structure. Basically we have a main product (and a few main keywords) which should drive the most traffic to our website, and for which we want to optimize our homepage. Besides those main keywords, we have an enormous base of long-tail keywords from which we would like to generate traffic. This means we want to create a lot of specific pages which are optimized. My main question is the following: We are thinking of two options: Option 1: www.example.com/example-keyword-one Option 2: www.example.com/directory/example-keyword-one With option 1 we will link directly from our homepage to the most important pages (which represent our most important keywords). All the pages with the long tail content will be linked from another section on our website, which is one click away from our homepage (specifically a /solutions page which is linked from the footer). All the pages with long-tail content will have this structure www.example.com/example-keyword-one so the URLs will not contain the directory /solutions With option 2 we will use more subdirectories in our URLs. Specifically, for all the long tail content we would use URLs like this: www.example.com/solutions/example-keyword-one
Web Design | | NielsB
The directories we want to use wouldn't really have added value in terms of SEO, since they don’t represent important keywords. So what is the best way to go? Option 1, straightforward, short URL’s which don’t really represent the linking structure of our website, but only contain important keywords. Or option 2, choose for more directories in our URLs which represent the linking structure of our website, but contain directories which don’t represent important keywords. Would the keyword ‘solutions’ in the directory (which doesn’t really relate to the content on the page) have a negative impact on our rankings for that URL?0 -
Is there a best practice for using a general iso code for the EAME region and APAC region or should you break it out by country?
I am creating a strategy for multiple regions and the US comes to market different than EAME (Europe, Africa, Middle East) and China. We were planning on using language and iso codes in subfolder's but the corporation only wants their content to be in German, English, and Queens English. Our current decision is to use /en-US/, /en/, /de/, /en-CN/, /zh-CN/. /en/ and /de/ will be what we use for EAME. This doesn't seem like the best idea as I think /en/ will get indexed as the US version and not the EAME version. Any suggestion or if clarification is needed is greatly appreciated.
Web Design | | GodfreyB2B0 -
Getting a highly ranked site a better result for 1 search term
I have a highly ranked website for a niche category. My site ranks higher in SEOMOZ than all of my competitors, but I can't get any higher than 4th on a page for one specific search term. What can I do to help my site increase its ranking on a specific search term?
Web Design | | tadden0 -
Using a 301 vs. 302?
I'm running into a very confusing situation - and while I think I've worked through the answer, I'm hoping someone can help provide their insight. I have a client who is in the process of rolling out a responsive site. Because we need to host both Responsive and legacy versions of product pages on the domain we are using the following URL pattern. New Responsive Product Page exampleurl.com/product (existing URL structure) Older Product Pages (redirected to) exampleurl.com/legacy/product The rollout will be approximately 2 months to complete. The question becomes - should a 302 redirect be applied from the existing URl to the /legacy/ URLs until the new designs are launched? Given that the timing will be so short this seems reasonable. Or should a 301 be applied until the new responsive designs are rolled out?
Web Design | | JonClark150 -
Development site accidentally crawled - Will this cause problems?
We are currently developing a new version of our website and to make it easy to access for all team members, we just set it up on a server accessible via a publicly accessible domain name (ie devsite.com). There has been no SEO and no links created to this site, or so I thought. Recently, I found out that Google somehow found its way to this development site and has been indexing the pages! I was a little alarmed, as there are no links to the domain and we'll soon be transitioning all the content over to our primary production domain. I immediately created a robots.txt file to disallow access to the entire development domain. My fear is that there may be some duplicate content penalty if Google sees that the content that is on our new site (once it goes live and is pushed to our REAL domain name) was previously indexed on our test domain. We're slated to launch in 2-3 weeks. Is there anything else I should do? Should I even be worried? I'm probably a bit paranoid, but given the amount of time and effort that has gone into this new site, I love any advice or thoughts. Thank You!
Web Design | | AndrewY0 -
Does using Wordpress Multisite have any negative SEO impact?
I manage multiple websites in Wordpress and the idea of managing them all under one Wordpress install is very attractive. Are there any dangers SEO-wise to doing so? I know that all of the sites would live under the same IP address, but that's not something I'm really concerned with anyway because I don't do a lot of inter-linking between the sites. Thanks for your help! -El Juano
Web Design | | JonathanFashbaugh0