Home
Packages
Datasets
Task Views
R resources
Packages
Toggle theme
Toggle Menu
Home
Packages
robotstxt
get_robotstxts
get_robotstxts function
function to get multiple robotstxt files
Copy
robotstxt package
Read PDF manual
Maintainer: Pedro Baltazar
License: MIT + file LICENSE
Last published: 2024-08-29
Useful links
https://github.com/ropensci/robotstxt/issues
https://docs.ropensci.org/robotstxt/
https://github.com/ropensci/robotstxt