Issue with a particular command of robots.txt! Help needed!
-
Hi all,I'm facing a strange issue with a website: www.foodmood.itI've written this robots.txt: www.foodmood.it/robots.txt in which I'm using all commands I've tested and used on other websites.The problem is that this website is not indexing correctly, and my fear is that I've written the robots in a wrong way.In this forum I've been taught that this command Disallow: /*? should be substituted with Disallow: /? in order to be sure the crawler is not indexing all the URLs containing a question mark.My question is: do you think this command modified in the above way can cause a problem to my website?I repeat that I've used it on several sites and this is the first time I'm having this kind of problems.Thanks in advance for your help.A.
-
I would in future tend to trust sources closer to being official rather than someone on a forum. Remove the extra * so its just Disallow: /*?
You can find more info here - https://support.google.com/webmasters/answer/156449
You can also test your robots in your webmaster tools so don't forget to give that a go if in doubt.
Hope that helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will hreflang indication help single language site?
I have read a few articles that say indicating a language per webpage with hreflang really only helps sites that use multiple languages. Although my site is only in English, I see that it is ranking for a few foreign language keywords in Google Search Console (not sure exact traffic but roughly 15% of visitors from Search are non English preferred). My thought is that indicating language will help my single-language site because it will weed out the non-english speakers who are probably bouncing from the site. Overall I am thinking it might improve the quality of my search traffic. Do you think my logic is sound, or is adding hreflang not beneficial to my site? All feedback welcomed. Thanks!
Conversion Rate Optimization | | Jonathan.Smith0 -
HELP: Analysing data to make decisions, SEO vs PPC
Hi mozers! I have recently been seeing good results in the serps lately for my main keywords in my country NZ, now I'm seeing good results in Australia for these keywords and our USA domain is not far behind and making good progress... However, our NZ results show that we may get around 1 conversion every 3 days from our organic search. I read other places that click share for ppc was much higher, but there is no way we can match our competitors budgets from 3k - 300k per month. So our option was to focus on SEO. To me, the SEO results seem quiet low, however I'm not really sure how to go about diving deep into the analytics of it all to find out where I need to improve or focus on, which keywords are bringing these conversions. Maybe I need to go for long tail keywords etc... It seems my rankings are coming from general keywords which are still highly competitive, but even so, we are not performing to what I know we should be. Our competitors are mostly paying ppc, however I was told my ROI would be better spent via SEO. Any suggestions perhaps what I might be missing or doing wrong in this case. (I have recently done a new design overhaul with a new registration process) I have 3 top level domains you can see the site here http://bit.ly/1yhz96v
Conversion Rate Optimization | | edward-may0 -
Google Experiments issues
Hey fellow Mozzers! We're in the middle of working on CRO for a client of ours and we were going to try out Google Experiments for the first time. Following the click path described on the Google help page should take me to an option that says 'Create Experiment', but this isn't coming up - do we need to create the various URLs before it will let us set up an experiment, or has the system changed now? I've found in the past Google has a habit of updating its interfaces but not the advice on using the affected tools, so wondering if I'm looking at old information? Secondly, does the Experiment show a user the same variation of the page throughout their journey? What I mean by that is: their site currently has a real mish-mash of page styles, and I think a large part of boosting their conversions is probably down to ensuring consistency across the site as it currently seems like you're bouncing around different sites. But will this issue be made worse by running Experiments - ie, will I enter the site on Variation A, click a link and be shown the next page in Variation B and so on? If so, are we better off running the experiment on one page at a time and using what we've learnt to impact the next page? Thank for any help you can give! Nick.
Conversion Rate Optimization | | themegroup1 -
Google Remarketing Conversions - Possible Issue
Hi All, When it comes to Google Adwords, when considering conversions, Google assigns each conversion back to the click that originally drove that visitor on the day they clicked. so, even if someone converts today, Google would show that conversion as occurring two weeks ago when the visitor originally clicked on a paid search ad and came to my site. fine...whatever. My question now is if we use a remarketing campaign through the Display Network, and someone converts, will that conversion be shown under their new visit or back on their original click where they did not originally convert, causing them to be included in our remarketing audience? Cant seem to find a straight answer anywhere and hoping someone out there has some guidance.
Conversion Rate Optimization | | dsinger0 -
Need advice on google adsense positioning
Hi my site www.in2town.co.uk gets a few thousand visitors a day but i am having trouble making any money from google adsense and this is because i am no good at advertising or understand it. I have tried many formats and i just cannot understand the correct way to do it and would really like some help and advice in what i am doing wrong. Would really love to see some magazines that do google adsense to see where i am going wrong. Here is one page from my site with google adsense http://www.in2town.co.uk/news/health-women/nearly-one-in-four-women-risk-skin-cancer-by-not-wearing-sun-tan-lotion and here is another example http://www.in2town.co.uk/latest-business-news/business-expert-says-franchise-opportunities-are-a-great-way-to-start-a-business If anyone can please help me understand what i am doing wrong please, then that would be great.
Conversion Rate Optimization | | ClaireH-1848860 -
Need help Tracking PPC vs Organic and respective ROI
So I have a client who uses PPC and Organic SEO to drive traffic. Once a user gets to the site, the most common action is to fill out a form. I have Auto Tagging enabled in Adwords so all PPC clicks have "gclid" in the url. I am also grabbing the Google Cookie and parsing it out to determine if the visitor is PPC or Organic. So I have an if statement in my form code to pass PPC vs organic through the form for tracking purposes: $url = 'http://' . $_SERVER['SERVER_NAME'] . $_SERVER['REQUEST_URI']; if (false !== strpos($url,'gclid')) { ?> Essentially this is saying: If gclid is in the url, or if the cookie contains PPC
Conversion Rate Optimization | | tjsherrill
set the Web field as PPC Else
Set it as Organic. this is working about 80% of the time. I am trying to raise the accuracy. any ideas where I might be missing some data? thanks0 -
TestFreaks.com Opinions Needed
I'm looking into testfreaks for a client - frankly i'm having trouble answering a few of my own questions about them, namely: Can Google spider the test freaks javascript link? Can Google then crawl the reviews in the frame generated? Will Google count that as dupe content? (the content in the frame seems to reside on the testfreaks.com domain..) Do you think this is a safe method of increasing conversions? Seems all good to me, I'm mainly concerned about dupe content (and whether reviews written within the testfreaks interface can then be spidered ... and i'm not sure it can be).
Conversion Rate Optimization | | CheapGames990 -
Can I track user journeys from a particular source?
We want to track users from a particular campaign to see what they do when they get to the site. The site generates revenue from clicks out so we want to be able to see whether visitors from a particular source click on multiple deals. i.e. enters from the source and then clicks on 3 different deals that take the user to other sites. Using GA, can I track this by implementing UTM on both the campaign links and outgoing links? Can I see exactly how many deals the average user is clicking on? As far as I know I can: Track visitors from the campaign Track overall clicks on deals Divide click outs by visitors But I can't track how many deals the visitors from campaign A clicked on... or can I?
Conversion Rate Optimization | | Red_Mud_Rookie0