Robots.txt Generator
Build a robots.txt file.
How to use Robots.txt Generator
- Add your content Type or paste into the box. Nothing is uploaded — this runs in your browser.
- Set the options Adjust the settings to suit what you need.
- See the result The result updates instantly as you change things.
- Copy or download Copy the result to your clipboard, or download it as a file.
robots.txt controls crawling, not indexing. A page blocked here can still appear in results if other sites link to it — use a noindex tag to keep a page out of the index.
This tool runs entirely in your browser. Nothing you enter is uploaded, stored or seen by us.
Build a robots.txt with per-crawler rules and your sitemap reference, with the common mistakes flagged before you publish it.