Using Robots.txt to Boost Your SEO.

A highly essential, and yet widely disregarded, piece of technical SEO is the robots.txt file, which is essential in terms of dictating how your site will be indexed by search engines. This plain text document serves as a guide to search engine robot...