home73k/assets/static/robots.txt

13 lines
323 B
Text
Raw Normal View History

2021-03-28 21:12:55 -04:00
# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
# User-agent: *
# Disallow: /
Disallow: /pcasts/
Disallow: /acasts/
Disallow: /swm/
Disallow: /acasts/swm/
Disallow: /mjrpt/
Disallow: /pcasts/mjrpt/