is_valid_robotstxt function

function that checks if file is valid / parsable robots.txt file

robotstxt packageRead PDF manual
  • Maintainer: Pedro Baltazar
  • License: MIT + file LICENSE
  • Last published: 2024-08-29

Useful links