What is a Robots.txt file?
It is a text file that tells the search engine bots what files to index and what files to skip. It is a public file so it is not used to hide files. It is only used to tell the bots what files to skip indexing.
Filed under Robots.txt, Website SEO by


