Crawling Google Trivia
How many bots does Google have?
More than 25 and less than 1,000
Crawling Hosting and Servers
Should I use the Vary HTTP header on URLs that redirect based on user-agent?
Yes, it's still used by Google though maybe not other services
Crawling Website Coding
How does Google handle "not found" pages that don't return a 404?
Google tends to treat as regular 404s
Crawling Website Content
Should I block Googlebot from crawling a dynamically generated calendar?
Additional text has been truncated due to copyright reasons. Things without URLs and private things don't get truncated.