When should you use a robots.txt file?

1. When you have multiple versions of a page to indicate the preferred version
2. When your website receives a penalty from Google
3. When you have pages that you don’t want search engines to crawl and index
4. Whenever you feel like it

Correct Answer:

• When you have pages that you don’t want search engines to crawl and index