Is the If-Modified-Since HTTP Header still relevant?
-
I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site. I checked and apparently our web server doesn't support this.
I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).
However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.
So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?
And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)
-
If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.
-
Hi Annie
I'm surprised there hasn't been lots of answers to your question.
Check-out this video here on SEOmoz entitled "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand. It opens with a partial answer to your first question:
"These days we use it a little less" (2 years ago) ~ basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.
In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.
**Your second question: **
- Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
- Ensuring that your site doesn't have any technical issues (say causing significant downtime).
- Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
- Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
- You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.
Hope that helps,
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Http to https redirection issue
Hi, i have a website with http but now i moved to https. when i apply 301 redirection from http to https & check in semrush it shows unable to connect with https & similar other tool shows & when i remove redirection all other tools working fine but my https version doesn't get indexed in google. can anybosy help what could be the issue?
Technical SEO | | dhananjay.kumar10 -
Http:// vs Https:// in Og:URL
Hi, Recently, we have migrated our website from http:// to https://. Now, every URL is in https:// and we have used 301 permanent redirection for redirecting OLD URL's to New Ones. We have planned to include http:// link in og:url instead of https:// due to some social share issues we are facing. My concern is, if Google finds the self http:// URL on every page of my blog, will Google gets confused with http and https:// as we are providing the old URL to Google for crawling. Please advice. Thanks
Technical SEO | | SameerBhatia0 -
HTTPS vs HTTP
A few months ago we switched all product page urls in our e-commerce site to https: Recently working on the site I was appalled at how slow our pages were loading and on investigating further with our hosting partner they advised to switch back to http instead of https for all of the site content to help page speed. We obviously still need to use https in the cart and check-out. I think that that Google will be pushing all commerce web pages to https but for now I need to improve page load speed. Will the switch back from https to http impair our keywords? https://www.silverandpewtergifts.com/
Technical SEO | | silverpewter0 -
For keyword relevance, how important is proximity to beginning of the line and/or page?
A client and I are debating between two taglines, which both contain an important keyword phrase. This tagline will exist as the H1 on the homepage, as well as in the header with the logo on internal and landing pages. The tagline structure is either: [keyword phrase] Solutions or Single Source [keyword phrase] Can anyone weigh in on which is better? Thanks much!!
Technical SEO | | imagodig0 -
Is “Bad IP Neighborhood” Theory still relevant?
Hi, I want to hear if there are any who believe that "Bad IP Neighborhood" Theory is still relevant? Find more information here: http://www.searchenginejournal.com/does-google-safebrowsing-prove-bad-ip-neighborhood-theory/10702/ Best regards, Jonathan
Technical SEO | | JoLinda911 -
Need for a modified meta-description every page for paginated content?
I'm currently working on a site, where there url structure which is something like: www.domain.com/catagory?page=4. With ~15 results per page. The pages all canonical to www.domain.com/catagory, with rel next and rel prev to www.domain.com/catagory?page=5 and www.domain.com/catagory?page=3 Webmaster tools flags these all as duplicate meta descriptions, So I wondered if there is value in appending the page number to the end of the description, (as we have with the title for the same reason) or if I am using a sub-optimal url structure. Any advice?
Technical SEO | | My-Favourite-Holiday-Cottages0 -
Why Do Transparent Networks Still Work
Hi Mozzers, My client has a major competitor that dominates several industry head terms. A check of their link profile reveals that they have 50 low DA domains that are identical to the main site, the only difference being that they all link to the main domain for these terms. They're not even attempting to disguise the network but it works. Can anyone tell me why? See: www.omega.com/vhpc/
Technical SEO | | waynekolenchuk0 -
Https-pages still in the SERP's
Hi all, my problem is the following: our CMS (self-developed) produces https-versions of our "normal" web pages, which means duplicate content. Our it-department put the <noindex,nofollow>on the https pages, that was like 6 weeks ago.</noindex,nofollow> I check the number of indexed pages once a week and still see a lot of these https pages in the Google index. I know that I may hit different data center and that these numbers aren't 100% valid, but still... sometimes the number of indexed https even moves up. Any ideas/suggestions? Wait for a longer time? Or take the time and go to Webmaster Tools to kick them out of the index? Another question: for a nice query, one https page ranks No. 1. If I kick the page out of the index, do you think that the http page replaces the No. 1 position? Or will the ranking be lost? (sends some nice traffic :-))... thanx in advance 😉
Technical SEO | | accessKellyOCG0