Is URL appearance defined by crawling or by XML sitemap
-
I am having a problem developing a sitemap because I have long URLs that are made by zend. They go like this: http://myagingfolks.com/professionals/20661/social-workers/pennsylvania-civi-stanger
Because these URL's are long and are fed by Zend when I try to call them all up, to put on the sitemap, the system runs out of memory and crashes.
Do you know what part of a search result, in google, say, comes from the URL? Would it be fine for me to submit to google only www.myagingfolks.com/professionals/20661. Does the crawler find that the URL is indeed http://myagingfolks.com/professionals/20661/social-workers/pennsylvania-civi-stanger or does it go with just what the sitemap tells it?
-
Hi Joe,
THanks for the response. One thing: given that my URL structure gets everything beyond /professional/number/blah blah blah from Zend, does that automatically count as a 301 forward. Meaning, if I get the entire URL in the sitemap, will I still awaken the ire of the google-god?
thanks
-
Google is going to go to the pages submitted in the sitemap and see that they are serving a 301 response code, which they don't want to see in sitemaps. Either find a way to create a sitemap for the URLs you want to use (this is what I'd do) or shorten your URLs so they work with your sitemapping solution (although it is not a good idea to change URL structure because of a software limitation).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Only half of the sitemap is indexed
I have a website with high domain authority and high quality content and blog. I've resubmitted the sitemap half a dozen times. Search console getr half way through and then stops. Does anyone know any reason for this? I've seen the usual responses of 'google is not obligated to crawl you' but this site has been fully crawled in the past. It's very odd Does anyone have any ideas why it might stop half way - or does anyone know a testing tool that might illuminate the situation?
Algorithm Updates | | Andrew-SEO0 -
Sitemaps for landing pages
Good morning MOZ Community, We've been doing some re-vamping recently on our primary sitemap, and it's currently being reindexed by the search engines. We have also been developing landing pages, both for SEO and SEM. Specifically for SEO, the pages are focused on specific, long-tail search terms for a number of our niche areas of focus. Should I, or do I need to be considering a separate sitemap for these? Everything I have read about sitemaps simply indicates that if a site has over 50 thousand pages or so, then you need to split a sitemap. Do I need to worry about a sitemap for landing pages? Or simply add them to our primary sitemap? Thanks in advance for your insights and advice.
Algorithm Updates | | bwaller0 -
One of my pages doesn't appear in Google's search
Our page has been indexed (I just checked) but literally doesn't exist in the first 300 results despite having a respectable DA & PA. Is there something I can do? There's no reason why this specific page doesn't rank, as far as I can see. It's not a new page. Cheers, Rhys
Algorithm Updates | | SwanseaMedicine0 -
Sitemap Question - Should I exclude or make a separate sitemap for Old URL's
So basically, my website is very old... 1995 Old. Extremely old content still shows up when people search for things that are outdated by 10-15+ years , I decided not to drop redirects on some of the irrelevant pages. People still hit the pages, but bounce... I have about 400 pages that I don't want to delete or redirect. Many of them have old backlinks and hold some value but do interfere with my new relevant content. If I dropped these pages into a sitemap, set the priority to zero would that possibly help? No redirects, content is still valid for people looking for it, but maybe these old pages don't show up above my new content? Currently the old stuff is excluded from all sitemaps.. I don't want to make one and have it make the problem worse. Any advise is appreciated. Thx 😄
Algorithm Updates | | Southbay_Carnivorous_Plants0 -
A Serious drop in Pages crawled per day
On 21st April ,I spotted a sudden decrease in pages crawled per day.Previously it was about 5,000 bust after the drop it reached to 225.From the crawl rate never spiked. Here is my website url - http://www.wpstuffs.com/ 8fQHW2G.png
Algorithm Updates | | vividvilla0 -
Canonical URl
Hello, All the pages of my site contained canonical url it shows me in the source, but on seomoz site it shows error that some the pages not containing canonical urls, anyone will help me ??
Algorithm Updates | | KLLC0 -
Vanity URL's and http codes
We have a vanity URL that as recommended is using 301 http code, however it has been discovered the destination URL needs to be updated which creates a problem since most browsers and search engines cache 301 redirects. Is there a good way to figure out when a vanity should be a 301 vs 302/307? If all vanity URL's should use 301, what is the proper way of updating the destination URL? Is it a good rule of thumb that if the vanity URL is only going to be temporary and down the road could have a new destination URL to use 302, and all others 301? Cheers,
Algorithm Updates | | Shawn_Huber0 -
Do I nee 2 sitemaps?
Our ecommerce software produces a sitemap.html which is very large. We also use a sitemap.xml file for Google and other main search engines. Is there any point in maintaining the sitemap.html or should we hide it?
Algorithm Updates | | FFTCOUK0