Now when someone mentioned to John Mueller of Google that providing this API for the URL inspector tool would reduce scraping, John laughed. He said, "Hah, I doubt it." Adding also that it is not right to say we will break your TOS (terms of service) if you don't give us a feature we want, is not a good argument. … [Read more...] about Google: Providing APIs Won’t Reduce Search Results Scraping
Scraping robots txt
Like the Google Analytics solution, the log file analysis isn't foolproof. It may be possible for Googlebot to visit a page but not actually include it in its index (), but it will help us narrow down our list of possibly non-indexed web pages. … [Read more...] about How to check which URLs have been indexed without upsetting Google: A follow-up
We were very careful not to scrape or break any Terms of Service. We leveraged an unrestricted, existing Google API believing it was openly available for such use. We don't want to do any damage to anyone, though I wish our industry could be made more aware of Google's hard lines. The existing materials are often too fuzzy or too difficult to find." … [Read more...] about Google Warns Against Automated Queries, What Does This Mean for Third-Party Tools?
Thin Content The other common problem of e-commerce sites is thin content. Shallow or low-quality web pages are considered as having thin content. Such pages are deemed unnecessary and inappropriate by Google because they create a bad user experience. These pages have high bounce rates because users leave them in few seconds. For that matter, Google completely slams ‘thin sites’ on the search engines. … [Read more...] about Duplicate Content Not Good for E-Commerce SEO
Regarding first steps for those that are hit, it’s impossible to tackle Panda with band-aids. A full audit must be conducted (through the lens of Panda). Those audits typically produce a number of important action items. Panda targets low quality content, but “low quality” can mean a lot of things. It can mean thin content, duplicate content, low-quality affiliate content, scraped content, technical problems causing content quality issues, etc. An audit will surface problematic areas to address. That’s the first step (it’s a big step, but it’s critically important). … [Read more...] about Ask the Experts: What’s Changed After Panda 4.0?