Recently, the two heaviest hitters in the search engine world – Google and Bing – jointly announced that they would increase the file size limit for sitemaps from 10MB to 50MB. A sitemap file contains all the URLs that belong to a particular site, and the maximum number of URLs a sitemap file may contain remains capped at 50,000. While the jump from 10MB to 50MB may seem insignificant at first glance, it is actually a big change for the search engine world. Sitemaps’ Impact on SEO Sitemaps are invaluable files for search engine optimization (SEO). Essentially, a sitemap is a map of a website that shows how the website is structured, the links that comprise the whole, and how they interact. They are important for both users and search engines, and due to their nature, two main types of sitemaps exist – one for human visitors to navigate websites and another designed for search engines’ bots to crawl through for SEO purposes. Sitemaps are useful for search … [Read more...] about Sitemap Max File Size Limit Increased On Google and Bing
Fabrice Canel, the Principal Program Manager of the Bing Index Generation team, posted their Sitemaps best practices guide for large web sites.Bing says they can support up to 125 trillion links through multiple XML sitemap files. With one sitemap file, Bing allows you to list 50,000 x 50,000 links, which brings you to 2,500,000,000 links (2.5 billion). If you need more URLs, then Bing allows you to use 2 sitemap index files, which can then lead to the 125 trillion number.Bing however recommends you don't list so many URLs. Rarely will Bing index all those URLs, so just list the URLs that are important to the site.The total size of sitemap XML files can reach more than 100GB. For really large sites, Bing recommends you take things slow:To mitigate these issues, a best practice to help ensure that search engines discover all the links of your very large web site is that you manage two sets of sitemaps files: update sitemap set A on day one, update sitemap set B on day two, and … [Read more...] about Bing: You Can Have Up To 125 Trillion URLs In Sitemaps
The major search engines have announced an update to the sitemaps.org protocol which enables site owners to store their XML Sitemap files in any location — even on a different domain than the one referenced in the Sitemap. This will be a welcome change for those who manage multiple domains and would like to keep all Sitemap files in one place, as well as for those who would like to store their Sitemap in a location other than the root. The only caveat? You have to be able to edit the robots.txt file of the domain the Sitemap file references. The search engines made the announcement today on the Search Engineers Q&A panel at SMX West. Below, more about how this works and how to implement it on your site. Historically, your Sitemap file had to be stored in the same location as the URLs listed in that Sitemap. For instance: http://www.example.com/sitemap.xml could include:www.example.comwww.example.com/folder1/page1.htmlwww.example.com/folder2.page1.html But it could not … [Read more...] about You Can Now Store Your XML Sitemap Files Anywhere!
Google’srolled out a new tool at GoogleWebmaster Central, a robots.txt generator. It’s designed to allow siteowners to easily create a robots.txt file, one of the two main ways (along withthe meta robots tag)to prevent search engines from indexing content. Robots.txt generators aren’tnew. You can find many of them out there by searching. But this is the firsttime a major search engine has provided a generator tool of its own. It’s nice to see the addition. Robots.txt files aren’t complicated to create.You can write them using a text editor such as notepad with just a few simplecommands. But they can still be scary or hard for some site owners tocontemplate. To access the tool, log-in to yourGoogle Webmaster Toolsaccount, then click on the Tools menu option on the left-hand side of the screenafter you select one of your verified sites. You’ll see a "Generate robots.txt"link among the tool options. That’s what you want. By default, the tool is … [Read more...] about Google Offers Robots.txt Generator
Back in November 2009, Google News announced they were “in the midst of an exciting transition period” that included a change to the News Sitemap Protocol. News publishers have through April 2010 to modify their News Sitemap to accommodate the new format. What’s so exciting and transitional? I asked Google, thinking that they were changing the protocol to prepare for some exciting new things in Google News. I was a bit disappointed in the answer, then, when they told me the exciting transition was simply the change to the protocol itself. The changes do make things a bit easier for News publishers though in a couple of ways: You can now reference your News Sitemap in your robots.txt file or ping Google with its location, rather than submitting via Google Webmaster Tools (I would still recommend submitting via Webmaster Tools the first time for the benefit of the parsing error information) You can now combine articles of multiple types into one News Sitemap. … [Read more...] about The Latest On Google News Sitemaps