Site Maps
-
I have provided a site maps for google but although it craws my site www.irishnews.com at 6:45AM the details in the site map are not seen on google for a few days - any ideas how to get this feature working better would be great.
example
<url><loc>http://www.irishnews.com/news.aspx?storyId=1126126</loc>
<priority>1</priority>
<lastmod>2012-01-23</lastmod>
<changefreq>never</changefreq></url>thanks
-
Hi Liam,
There is always a natural delay between crawling and indexing, and it's rarely instantaneous. Although I can see why you'd want a news site to be getting indexed pretty quickly.
The one thing that stands out is from the example is the <changefreq>tag, which you've got set on 'never'. This is essentially for archived pages, and tells crawlers that it's low importance (even though you've given it high priority). Even if you're not intending to change the article, I'd still recommend giving it a change frequency of 1 month so you're inviting the crawlers to come and check it more often. Saying that, this doesn't mean that if you set the frequency to hourly that the crawlers would come back every hour, as they'd soon figure out that nothings changing.</changefreq>
Really I'd have your home and catergory pages on daily, the articles on monthly, and the static pages on monthly or yearly.
In terms of getting them indexed quicker after the crawl, this is just a case of establishing trust and importance from the search engines. They need to know that you have news content that requires a quicker indexed. You can gain this trust by providing regular, high quality pages. When their crawlers pick up that there are new pages going live daily you will see the index get quicker.
Depending what CMS you are using, it is also worth getting a script or plugin that pings the search engines with an updated site map every time a new page or post is added. I've done this through Wordpress and articles get indexed within 4 hours. Again though, it didn't start this way and it only got this quick once I was putting up 4 posts a week like clockwork.
Good luck.
Cheers
David
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is the best strategy to SEO Discontinued Products on Ecommerce Sites?
RebelsMarket.com is a marketplace for alternative fashion. We have hundreds of sellers who have listed thousands of products. Over 90% of the items do not generate any sales; and about 40% of the products have been on the website for over 3+ years. We want to cleanup the catalog and remove all the old listings that older than 2years that do not generate any sales. What is the best practice for removing thousands of listings an Ecommerce site? do we 404 these products and show similar items? Your help and thoughts is much appreciated.
White Hat / Black Hat SEO | | JimJ3 -
Scraped site, hijacked searches for business name.
Hello, I have a site that was scraped (possibly by a competitor's seo company), who then built links to the duplicate site. When people do a search for the name of the business the scraped site is all that comes up along with the usual third-party sites. They seem to take the site down and put it back up every couple of weeks to maintain the rankings in Google. Has anyone ever dealt with something like this? Any advice or recommendations would be appreciated. Search: LIC Dental Associates Scraped site: old-farmshow.net Legit site: licdentalassociates.com Thanks, Emery
White Hat / Black Hat SEO | | tntdental1 -
Clean-up Question after a wordpress site Hack added pages with external links from a massive link wheel?
Hey All, Thought I would throw this out to ensure I am dotting my "i's" and crossing my "t's"..... Client WordPress site was hacked injected 3-4 pages that cross linked to hundreds (affiliate junk spam link wheel). Pages were removed, 3rd party cleared all malware/viruses. Heavy duty firewall and security monitoring are in place. Hacked pages are now showing as 404. No penalties, ranking issues....If anything there was a temporary BOOST in rankings due to the large link-wheel type net that the pages were receiving....That has since leveled out rankings. I guess my question is, in your opinion is it best to let those pages 404, I am noticing a large amount of links going to them from all over the world from this large link net that was built. I find the temptation to 301 re-direct deleted pages to the homepage difficult...lol..{the temptation is REAL}. Is there anything I am missing? Any other steps that YOU would take? I am assuming letting those pages 404 would be the best bet, as in time they will roll off index.... Thank you in advance, I appreciate any feedback or opinions....
White Hat / Black Hat SEO | | Anthony_Howard0 -
Do legitimately earned links from unrelated sites help or hurt?
We have a few charity events coming up that have offered to link back to our homepage. While we do genuinely like the charities we are going to sponsor, I'm not sure how those links will look seo-wise. For example, one is for the local high school basketball team and another is for a Pediatric Care Mud Run. To a human, these links make perfect sense, but to a robot, I'm not sure if it differentiates these links from spam/some negative link. Granted, I understand that a small percentage of links probably won't do anything either way, but I'd like to ignore that for the purposes of my question. All things being equal, do links such as these help or hurt? Thanks for your time and insight, Ruben
White Hat / Black Hat SEO | | KempRugeLawGroup0 -
Failed microsites that negatively affect main site: should I just redirect them all?
While they are great domain names, I suspect my 7 microsites are considered spammy and resulted in a filter on my main e-commerce site for the important keywords we now have a filter blocking from showing up in search. Should I consider it a sunk cost and redirect them all to my main e-commerce site, or is there any reason why that would make things worse? I've fixed just about everything I can thinking of in response to Panda and Penguin, before which we were on the first page for everything. That includes adding hundreds of pages of unique and relevant content, in the form of buyers guides and on e-commerce category pages -- resolving issues of thin content. Then I hid URL parameters in Ajax, sped up the site significantly, started generating new links... nothing... I have tons of new keywords for other categories, but I still clearly have that filter on those few important head keywords. The anchor text on the microsites leading to the main site are typically not exact match, so I don't think that's the issue. It has to be that the sites themselves are considered spammy. My bosses are not going to like the idea because they paid for those awesome domains, but would the best idea be to redirect them to the e-commerce site?
White Hat / Black Hat SEO | | ElBo9130 -
Spam linking site how to report
I have a spam linking site that is generation thousans of links to my site. Even if i have a good link background, this is the only spammy i have, each week number of links comings from it increases by 500 , i know have 3000 links for that site and 1800 for other sites, but that one keeps growing What should i do, i dont want that link it is imposible to remove as webmaster does not respond
White Hat / Black Hat SEO | | maestrosonrisas0 -
HELP - Site architecture of E-Commerce Mega Menu - Linkjuice flow
Hi everyone, I hope you have a couple of mins to give me your opinion. Ecommerce site has around 2000 products, in english and spanish, and around only 70 hits per day if that. We have done a lot of optimisation on the site - Page Titles, URL's, Content, H1's, etc.... Everything on page is pretty much under control, except I am starting to realise the site architecture could be harming our SEO efforts. Once someone arrives on site they are language detected and do a 302 to either domain.com/EN or domain.com/ES depending on their preferred language. Then on the homepage, we have the big MEGA MENU - and we have
White Hat / Black Hat SEO | | bjs2010
CAT 1
SubCat 1
SubsubCat 1
SubsubCat 2
SubsubCat 3 Overall, there are 145 "categories". Plus links to some CMS pages, like Home, Delivery terms, etc... Each Main Category, contains the products of everything related to that category - so for example:
KITCHENWARE
COOKWARE BAKINWARE
SAUCEPANS BOWLS
FRYING PANS Kitchenware contains: ALL PRODUCTS OF SUBCATS BELOW, SO COOKWARE ITEMS, SAUCEPANS, FRYING PANS, BAKINGWARE, etc... plus links to those categories through breadcrumbs and a left hand nav in addition to the mega menu above. So once the bots hit the site, immediately they have this structure to deal with. Here is what stats look like:
Domain Authority: 18 www.domain.com/EN/
PA: 27
mR: 3.99
mT: 4.90 www.domain.com/EN/CAT 1
PA: 15
mR: 3.05
mT: 4.54 www.domain.com/EN/CAT 1/SUBCAT1
PA: 15
mR: 3.05
mT: 4.54 Product pages themselves - have a PA of 1 and no mR or mT. I really need some other opinions here - I am thinking of: Removing links in Nav menu so it only contains CAT1 and SUBCAT1 but DELETE SUBSUBCATS1 which represent around 80 links Remove products within the CAT1 page - eg., the CAT 1 would "tile" graphical links to subcategories, but not display products themselves. So products are only available right at the lowest part of the chain (which will be shortened) But I am willing to hear any other ideas please - maybe another alternative is to start building links to boost DA and linkjuice? Thanks all, Ben0 -
Beating the file sharing sites in SERPs - Can it be done and how?
Hi all, A new client of mine is an online music retailer (CD, vinyls, DVD etc) who is struggling against file sharing sites that are taking precedence over the client's results for searches like "tropic of cancer end of things cd" If a site a legal retailer trying to make an honest living who's then having to go up against the death knell of the music industry - torrents etc. If you think about it, with all the penalties Google is fond of dealing out, we shouldn't even be getting a whiff of file sharing sites in SERPs, right? How is it that file sharing sites are still dominating? Is it simply because of the enormous amounts of traffic they receive? Does traffic determine ranking? How can you go up against torrents and download sites in this case. You can work on the onsite stuff, get bloggers to mention the client's pages for particular album reviews, artist profiles etc, but what else could you suggest I do? Thanks,
White Hat / Black Hat SEO | | Martin_S0