• Calculate the most remarkable formula in mathematics
  e ^ (pi * i)
• Measure very big distances in very small mesurements
  1 light year in angstroms
• Embiggen your vocabulary
  define:euphonious
• Find sequels
  site:imdb.com rocky 1..5
• Bring the Romans into the Information Age
  MMVII in binary
• Find ebooks
  ext:pdf c# guidelines
• Find reviews of the greatest films ever made
  movie:The Karate Kid
• Wish you’d bought ten years ago
  stocks:MSFT
• Get a four day forecast for your business jolly trip
  weather:Singapore
• Find ‘interesting articles’
  allinanchor: interesting article
• Perform complex baking calculations
  10 teaspoon + 5 tablespoons in cups
• Will the strange imported chocolate make you fat?
  700 kilojoules in kilocalories
• Was that ducks are hazards in the classroom?
  “Pink Floyd” no * in the classroom
• Check out criminally offensive your favourite tunes
  music:Michael Bolton
• Plan your holiday finances
  currency of Estonia in Peruvian money

Got any more?

http://www.mysite.com/gadgets/widget.aspx

Instead of this:

http://www.mysite.com/DisplayPage.aspx?productId=747393

I remembered Scott Guthrie’s ASP.NET URL rewriting article, which explains the various options, and chose the best solution for my situation which was an HttpModule with the HttpContext.RewritePath method. The implementation details are explained in Scott’s article, and the approach works really well.

A few weeks after launching the new site, I noticed Google had indexed only a few hundred pages, and the bulk of my content was still missing. As soon as I started getting diagnostic data from Google Webmaster Tools I could see the problem immediately — I had thousands of unreachable URLs, each throwing an ASP.NET error! This was weird because the pages loaded fine in Firefox and IE, and only URL rewritten pages were throwing errors.

I needed to see what was happening when Googlebot looked at the page, so I installed a Firefox plug-in called User Agent Switcher which lets Firefox appear to the web server as Googlebot by changing the user-agent string to:

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

To my surprise, and slight relief, the page threw an ASP.NET exception:

Exception message: Cannot use a leading .. to exit above the top directory.

After searching for a solution, it turns out the problem is caused by the way ASP.NET does browser detection and subsequent adaptive rendering. It doesn’t seem to recognise Googlebot’s new user-agent string, so downgrades to a buggy rendering mode that uses the Html32TextWriter instead of the usual HtmlTextWriter. There’s more information about the problem and some fixes in these articles:

ASP.NET 2.0 Mozilla Browser Detection Hole
Get GoogleBot to crash your .NET 2.0 site

The solution, as described on Brendan Kowitz’s .Net Blog, is to add a new browser definition file to your app_browsers folder. This will tell ASP.NET to use the working HtmlTextWriter (and not Html32TextWriter) when rendering for Googlebot.

This is a fairly new problem, introduced towards the end of March 2007 when Google deployed their BigDaddy infrastructure update, which included an update to the Googlebot user agent string.