Robots exclusion standard (Q3700)
Jump to navigation
Jump to search
standard used to advise web crawlers and scrapers not to index a web page or site
| Language | Label | Description | Also known as |
|---|---|---|---|
| English | Robots exclusion standard |
standard used to advise web crawlers and scrapers not to index a web page or site |