You may not want certain web pages of your internet site crawled for the reason that they may not be useful to users if located in a search engine's search engine results. Observe that if your internet site utilizes subdomains and you want to acquire particular web pages not crawled on a particular subdomain, you will have to produce a independent