SEO Web Crawler IP addresses
-
What are the IP addresses for the SEO Web Crawler?
There is a firewall on my clients website before it goes live, I would like to crawl the site before it goes live, but need to provide the web crawlers IP addreses.
Thank you for your time
-
Hi Jason,
This is Megan from the SEOmoz Help Team. Unfortunately, since we crawl from the cloud, RogerBot's IP address is changing all of the time so we cannot specify a particular IP address. Also, we do need the site to be live before we can crawl it, but you could remove the firewall and block all other bots except rogerbot so it won't get indexed anywhere else before it's ready.
I hope this helps! Please let me know if you have any other questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Integrate social networks for SEO improvement
I used some SEO tools for my site: https://www.emojionline.org. These tools said that need to integrate social networks, such as facebook, twitter... I used addthis plugin to integrate social networks. When I test some tools didn't recognize the social integration. So I want to know how to test social integration with Moz? I used addthis plugin, so if Moz also didn't recognize, what need I do?
Moz Pro | | micheal90000 -
SEO Help in LA
Hi All- We're new to the MOZ community and plan to use the tools to help our site. We sell graphic design retail products globally. We're looking for a recommendation for a MOZ pro in Los Angeles (ideally) to help us build out our campaign(s) and get us started with the MOZ tools. Any of you in LA? Any of you have a rec for a great resource close to us? Thanks so much, Scott
Moz Pro | | freshs0 -
Local SEO does not rank well
Hello everybody, before 3 months i began seo in gialia.net.gr. Its an eshop based on symfony framework. We started with on-page seo optimization and after making links in directories, forums and article directories. After 2 months we saw rankings for niche keywords (γυαλια ηλιου, γυαλια ηλιου προσφορες) at 5th and 8th page of google.gr. Now, the rankings does down and i cannot unserstand the reason. I checked old backlinks with opensite explorer, and found some spam backlinks links (3/10). Is this the reason that seo not going well? I am very dissapointed and i would like to tell me you opinion about what is going wrong with that site. My customer owns one more domain, fotispapoulias.gr and i am thinking to move all the site that is running on gialia.net.gr to fotispapoulias.gr (301 redirect to all links). The fotispapoulias.gr ranks better in the same niche keywords without seo. Is it good idea? Will achieve better rankings or not? Thanks
Moz Pro | | simplewebsol0 -
A competitor's SEO firm is building spammy links to my website.
A competitor's SEO firm is building spammy links to my website. I know the disavow link process, but even this can be time consuming. Is there a better way to protect my website? Any and all ideas appreciated.... Are there any cool Moz tools that can help me manage this assault? Sorry, just a follow up from me... It appears that many of the spammy links (referrals) I see in Google Analytics are not present in Google WebMaster Tools. Does this mean that the spammy ones that I'm concerned about have already been discounted by Google, or are the spammy links no-follow? Any input on this would also be appreciated.
Moz Pro | | BVREID0 -
Identify business pain points using SEO
Hi, can I use SEO research techniques to identify what kind of services businesses are searching for? Is it possible to find out pain points? I have been using tool like MicroNicheFinder to do such a research but I am not getting any where. Is SEO only used to outrank my competition? Do I need to first know what business I need to be in and then use SEO techniques to research the demand? Do I make sense? You can tell that I am very new at the internet marketing thing. Please let me know. Thanks.
Moz Pro | | zsyed0 -
Why seomoz crawler does not see my snapshot?
I have a web app that uses angularJS and the content is all dynamic (SPA). I have generated snapshots for the pages and write a rule to redirect ( 301) to the snapshot in case of find escaped_fragment in the URL. E.g http://plure.com/#!/imoveis/venda/rj/rio-de-janeiro Request: http://plure.com/?escaped_fragment=/imoveis/venda/rj/rio-de-janeiro is redirected to: http://plure.com/snapshots/imoveis/venda/rj/rio-de-janeiro/ The snapshot is a headless page generated by PhantomJS. Even following the guideline ( https://developers.google.com/webmasters/ajax-crawling/docs/specification) I still can't see my page crawled and I also in SEOMoz I can only see the 1st page crawled with no dynamic content on it. Am I doing something wrong? SEOMoz was supposed to get the snapshot based on same rules of GoogleBot or SEOMoz does not get snapshots?
Moz Pro | | plure_seo0 -
Local SEO Rankings Report Tools (based on several locations)?
Hi, Just a quick question. I am looking for a Rankings Report tool that allows a user to not only provide a list of Keywords, but also a list of Locations for which to check Google UK's ranking results. In practice, I would like to be able to enter 6 names of Towns next to my keyword list and get "localised results". Many thanks in advance, Cheers Greg
Moz Pro | | GregoryTK
PS: first Q within the Q&A, so please apologise for any potential "stupid questions"1 -
Any SEO moz users notice a HUGE change in OSE (Open Site Explorer) link data numbers?
Hi All, I am having some serious concern with OSE data recently for numerous clients, one client I want to talk about today has the following data from OSE for the month of August 2011 compared with July 2011: Total links to the domain: (decrease of around 100,000+)
Moz Pro | | ColumbusAustralia
External Followed links: (decrease by around 5,000)
**Linking Root domains: (decrease of over 60) ** The crazy thing is that the domain authority has actually gone up by around 5 points for this client even though every thing has suddenly gone down? Also funny thing is we have been link building quite strong for this client over the last 12 months using only high quality sources from out niche. I am worried that their is serious issues with the data, I realise we saw some updates to OSE recently yet I am suprised it can be this drastic. Kind Regards. PSV1