is_suspect_robotstxt function

is_suspect_robotstxt