The Purpose Of "robots.txt" For A Website

My blog has now move to http://www.tiglo.co.uk/. This post can now be found at http://www.tiglo.co.uk/IT-internet/The-Purpose-Of-robots-txt-For-A-Website.html.

Snippet: "What Is "robots.txt"? It is a file place in the root directory of a website that provides some instructions for a crawler/spider (Bot) about which files and/or directories are not to be indexed. It can be used to instruct a specific Bot or all Bots. Some people prefer to disallow Google's image Bot (to reduce bandwidth consumption)."