What do you use for test rendering your dev site?
-
I'm redesigning our company ecommerce site and need to test render an infinite scroller to ensure that it is as SEO friendly as possible. My problem is that I cannot view it in Webmaster Tools since I am blocking the site from crawlers using robots.txt. I know I could simply unblock Google temporarily but I really would rather not make my dev site available to search engine crawlers.
-
Screaming Frog
-
Hi Tyler. You can use a text browser like Lynx as that's a close approximation of how Google "sees" your site. From: https://support.google.com/webmasters/answer/40349...
Use a text browser, such as Lynx, to examine your site. Most spiders see your site much as Lynx would. If features such as JavaScript, cookies, session IDs, frames, DHTML, or Macromedia Flash keep you from seeing your entire site in a text browser, then spiders may have trouble crawling it.
And a more robust list of guidelines is available here: https://support.google.com/webmasters/answer/35769.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site appears then disappears from Google
Hi, This is my first post to the Moz community. I hope someone can help me, as I'm at my wits end 🙂 Since mid to late April, my client's home page keeps dropping from Google rankings on and off on the desktop in incognito and non-incognito browsing. It seems to always be visible on mobile phones though. One day it's on page 4, the next day it's not anywhere to be found (even though some of her other pages rank). The site is here: evgeniaribinik.com I have done some blog writing (just text) for this client over the last few months, and recently she asked for some SEO help. After looking at her website, I noticed that she had a plugin called WordPress SEO on her site. She wasn't using this for the blogs, however. I don't think she did much of anything with SEO, but she did say that for the last few years, she was always on page 3 or 4 of Google. In mid/late April, she saw that she wasn't ranking at all for the keyword "boudoir photography nyc" anymore, despite a few years of ranking for it. I told her Yoast would be good to use. However, after she installed it, the same issues keep happening. Right now, she has Yoast and WordPress SEO plugins installed (I'm not sure if this is causing an issue as well). But I really can't figure out why she keeps going on and off page 4. She also asked me to optimize older blogs that she wrote herself for SEO. When I look at them, they don't have meta descriptions, good titles or good keywords. I realize this is hurting her, but why would her site be fine for years and all of a sudden not now? Thank you in advance for any help you can give me! Jill
Web Design | | lobeng770 -
Is there an issue if we show our old mobile site to Google & new site to users
Hi, We have our existing mobile site that contains interlinking in footer & content and new mobile site that does not have interlinking. We will show existing mobile site to google crawler & new mobile site to users. Will this be taken as black hat by Google. The mobile site & desktop site will have same url across devices & browsers. Regards
Web Design | | vivekrathore0 -
Google tag manager on blocked beta site - will it phone home to Google and cause site to get indexed?
We want to develop a beta site, in a directory with the robots.txt blocking bots. We want to include the Google Tag Manager tags and event layer tracking code on this beta site. My question is that by including the Google Tag Manager code, that phones home to Google, will it cause Google to index this beta site when we don't want it indexed?
Web Design | | CFSSEO0 -
Site as one page - SEO implications
We may be inheriting a site and will be asked to do SEO for it. We will have control over the development of the site, so this structure is what it is. My question is - how significant of an impact do you think this is going to have and can you think of any workarounds that may help? Basically, the user experience of the site will feel similar to multiple pages. However, this site will, in essence be one page and pull various content through javascript from different locations. I have not seen the site yet (and believe it is still in development), but this is how it has been explained to me. Any thoughts? My first thought was to add a blog to add page depth to the site and expand the content. Any other thoughts are welcome and appreciated. Thanks. (I know this is limited information, I'm sorry. It's just about all I have to work with right now, and I was a little concerned and was hoping for a second opinion)
Web Design | | AdamWormann0 -
SEO tricks for a one page site with commented html content
Hi, I am building a website that is very similar to madebysofa.com : means it is one page site with entire content loaded (however are commented in html) and by clicking on sections it modify the DOM to make specific section visible. It is very interesting from UX point of view but as far as I know, since this way most of my content is always commented and hidden from crawlers, I will loose points regarding SEO. Is there any workaround you can recommend or you think sites like madebysofa.com are doomed to loose SEO points by nature? Best regards,
Web Design | | Ashkan10 -
Website redesign testing environment
An external company is redesigning our website. They have a draft version active in a folder on their website to enable us to view it (example: www.externalcompanywebsite.com/ourwebsite/). There is no permission required to view the website, anyone with the url could just go there. I am not sure if they're using the robots.txt to block crawlers, or if they're blocking specific ip addresses. If the engines are allowed to crawl their website and they index pages of our draft website, does that affect our website negatively in the future once we host it on our domain?
Web Design | | jdossetti0 -
Critique a site I'm working on please
Hi guys, I have a list of things to do on this website that is longer than my arm but thought I would gather a few more thoughts on the site before moving on further with it. The site is here. It went live a couple of days ago for live testing. I have a news section coming very soon at theworkplacedepot.co.uk/news - it is in live testing now. One glaring issue for me is that some of the products simply aren't "buyable". What I mean by that is that the options are too vague and too vast - as an outsider I wouldnt have a clue what I was buying! Other issues: The banner at the bottom of the HP is a test - its design will be changing. The sitemap has some issues and will be addressed. There is no area yet to "log-in". However, critique for me pretty please! 🙂
Web Design | | MattJanaway0 -
Using "#" anchors to display different content
If I have a page that has an area on the page that acts like a widget and has three different tabs. These tabs provide 3 different types of information relevant to the page subject matter. By default when someone goes to the page one of the tabs is showing but you have to click on the others to see the info on them. Is it OK to use domain.com/topic#TAB1, domain.com/topic#TAB2, domain.com/topic#TAB3 to create shortcut links so that people can land on the page and have that predetermined tab showing. I'm wondering what search engines might think. Essentially all the content of all three tabs is there for people to see but they'd have to click to see the other tabs. I don't consider the content to be hidden. But I'd like to hear people's thoughts.
Web Design | | Business.com0