Follow or nofollow instructs Web crawlers whether or not to follow the links on a page. It is like adding an rel=”nofollow” tag to every link on a page. Nofollow evaporates PageRank, the raw search engine ranking authority passed from page to age via links. Even if you noindex a page, it is probably a bad idea to nofollow it. Let PageRank flow through to its final conclusion. Otherwise, you could be pouring perfectly good link juice down the drain. … [Read more...] about Have You Considered Privacy Issues When Using Robots.txt & The Robots Meta Tag?
Robots txt sitemap xml
Certainly the search engines need to get their act together more, however. It’s time to stop referring people to the REP site which is run by no one. It’s time to stop having a myriad of help pages scattered about within their respective sites. Yes, they should continue to have their own help pages (see Google’s webmaster help from here; Bing’s from here). But I’d like to see Google and Microsoft take the lead to also consolidate material into a common site, perhaps building off Sitemaps.org. … [Read more...] about ACAP Versus Robots.txt For Controlling Search Engines
Stephan Spencer is the creator of the 3-day immersive SEO seminar Traffic Control; an author of the O’Reilly books The Art of SEO, Google Power Search, and Social eCommerce; founder of the SEO agency Netconcepts (acquired in 2010); inventor of the SEO proxy technology GravityStream; and the host of two podcast shows The Optimized Geek and Marketing Speak. … [Read more...] about A Deeper Look At Robots.txt
Danny Sullivan moderated, with panelists Keith Hogan, Director of Program Management, Search Technology, Ask.com, Sean Suchter, Director of Yahoo Search Technology, Yahoo Search, Dan Crow, Product Manager, Google and Eytan Seidman, Senior Program Manager Lead, Live Search. The Robots.txt summit session was not on how to use the robots.txt file, rather as Danny Sullivan explained, “We’re assuming you know how to use it and are frustrated with it. This is about how you want to see it evolve.” … [Read more...] about Up Close & Personal With Robots.txt