Google Sitemaps Update
Well Google Sitemaps seems to work. After tweaking my config file a few times, I’m only submitting one sitemap file to Google instead of three. But more importantly, my saturation in Google jumped from an average of 20000 pages to 178000! That’s the highest page saturation I’ve ever had on Google, and more than all my top five competitors combined!
Because of the number of pages on Belchfire.net, it’s not possible to manually create a sitemap file. So one of the options in the config file is to use your web server log file to generate the sitemap file. Since I rotate my log files daily, any time I update and upload my sitemap file to Google they’re only seeing a portion of the links on the site. So I didn’t expect to see more than 1500 - 5000 pages indexed.
But coincidentally, I had just used a spidering sitemapping tool to generate a static sitemap file. This would have caused all pages on the site to be logged in the web server log file on the day my Google sitemap was updated.
So this is interesting. I suspect that the next update I send to Google will only contain the 1500 - 5000 pages hit on a particular day. I’ll monitor this and if that’s the case, an easy way to ensure Google sitemaps records all pages on one’s site may be to simply spider the site using a third-party tool they day you update your sitemap file.
Interesting… :yes: