Follow or nofollow instructs Web crawlers whether or not to follow the links on a page. It is like adding an rel=”nofollow” tag to every link on a page. Nofollow evaporates PageRank, the raw search engine ranking authority passed from page to age via links. Even if you noindex a page, it is probably a bad idea to nofollow it. Let PageRank flow through to its final conclusion. Otherwise, you could be pouring perfectly good link juice down the drain. … [Read more...] about Have You Considered Privacy Issues When Using Robots.txt & The Robots Meta Tag?
Robot txt add sitemap
Overall, there are some ideas in ACAP that would be useful for the search engines to consider. However, there are many ideas outside of ACAP that would also be useful for them to consider. There’s nothing I see within ACAP that provides some type of crucial control that if only news publishers had, all their online woes would be over. What the news publishers really want are licensing agreements, and given that Google already has several of these without using ACAP (see Josh Cohen Of Google News On Paywalls, Partnerships & Working With Publishers), I can’t see that having it somehow advances any business model changes. … [Read more...] about ACAP Versus Robots.txt For Controlling Search Engines
"Preserve," with similar time limits available for "index," would stipulate whether a copy may be stored in a search engine’s cache. … [Read more...] about ACAP Launches, Robots.txt 2.0 For Blocking Search Engines?
Danny Sullivan moderated, with panelists Keith Hogan, Director of Program Management, Search Technology, Ask.com, Sean Suchter, Director of Yahoo Search Technology, Yahoo Search, Dan Crow, Product Manager, Google and Eytan Seidman, Senior Program Manager Lead, Live Search. The Robots.txt summit session was not on how to use the robots.txt file, rather as Danny Sullivan explained, “We’re assuming you know how to use it and are frustrated with it. This is about how you want to see it evolve.” … [Read more...] about Up Close & Personal With Robots.txt
Obviously, replacing the URL above with the URL of your sitemap index file. Neither of these announcements was specific but looking at the example provided by Yahoo this does appear to support sitempas exported into an XML file, not just an HTML file. … [Read more...] about Google, Yahoo, MSN add SiteMap Auto-Discovery