Remember earlier this year Bing wanted to slow crawling the web and request webmasters to push them content via their URL submission tool. They expanded their rate limits and opened their API for it. Now, Bing posted an easy to understand how to blog post on this topic. It really is something simple for a web developer to set up for you and probably recommended you do it.Bing said "In the last few months we have seen rapid adoption of this capability with thousands of websites submitting millions of URLs and getting them indexed on Bing instantly." Forum discussion at Twitter. … [Read more...] about Bing Explains How To Submit URLs Via API
Bing submit url
Last week we reported that Bing expanded their submit URL tool from 10 URLs per day to 10,000 URLs per day, removing essentially all the quotas. The shocking part in that announcement was that they seemed to want to do away with crawling as a strategy for getting web pages into their index.Yesterday, I interviewed Christi Olson, Bing’s head of search evangelism for Search Engine Land and she confirmed, that is the direction Bing hopes to take. It is inefficient and resource intensive for both Bing and the publishers for Bing to crawl their sites. Instead, they hope that publishers begin submitting content to their search engine.One point Christi Olson made crystal clear to me is that publishers should be using this now. If you produce content, using the tool will tell Bing as a "strong signal" to crawl the page NOW. If you want to pretty much ensure faster crawling and thus indexing of your content in Bing, then use it. You can submit stuff manually in the Bing Webmaster … [Read more...] about Bing Strongly Recommends Publishers Use The Submit URL Tool
Yesterday Bing announced at SMX West that they have increased the ability to submit URLs to their search engine by 1,000X from 10 URLs per day to 10,000 URLs per day. They also said this is a fundamental shift in how search engines discover content and reduce crawling of web sites. Bing wrote "We believe that enabling this change will trigger a fundamental shift in the way that search engines, such as Bing, retrieve and are notified of new and updated content across the web. Instead of Bing monitoring often RSS and similar feeds or frequently crawling websites to check for new pages, discover content changes and/or new outbound links, websites will notify the Bing directly about relevant URLs changing on their website. This means that eventually search engines can reduce crawling frequency of sites to detect changes and refresh the indexed content." What? Did you read that? Bing removed the public URL submission tool last year and had issues with the private one so had to add rate … [Read more...] about Bing Increased URL Submission To 10,000 URLs Per Day. Bing Moving Away From Crawling?
I am hearing some reports that even the submit URL tool within Bing Webmaster Tools is not working. We know Bing killed off the anonymous submit URL tool and told us to use the one within Bing Webmaster Tools. But if you try to use it and check your server logs, you may notice that Bing is not accessing the content after you submit the request.Not many folks track Bing in such detail, but I am hearing some chatter within some of the private areas that the tool is not working. In the past, if you would submit a URL to Bing, it would send BingBot within minutes to fetch the URL you submitted. Now, BingBot might not visit that URL for days, let alone minutes.I tried it myself and I did not see BingBot access the URLs I tried to submit within a few minutes. Of course, a site like this does get a lot of traffic, even from BingBot, so it gets messy fast. But I am hearing from multiple sources that Bing is not fetching URLs submitted via the submit URL tool.In addition, some are saying … [Read more...] about Bing Not Listening To Submit URL Webmaster Tools Feature
This new method for submitting URLs to Google is limited, so you should use it when it’s important that certain pages be crawled right away. Although Google doesn’t guarantee that they’ll index every page that they crawl, this new feature does seem to at least escalate that evaluation process. To better understand how this feature works, let’s take a look at how Google crawls the web and the various ways URLs are fed into Google’s crawling and indexing system. How Google Crawls & Indexes the Web First, it’s important to know a bit about Google’s crawling and indexing pipeline. Google learns about URLs through all of the ways described below and then adds those URLs to its crawl scheduling system. It dedups the list and then rearranges the list of URLs in priority order and crawls in that order. The priority is based on all kinds of factors, including the overall value of the page, based in part on PageRank, as well as how often the content … [Read more...] about Fetch, Googlebot! Google’s New Way To Submit URLs & Updated Pages