Book a free call

Free tool

Tell search engines where to look.

Choose what to block and add your sitemap. Copy the result into a robots.txt file at the root of your site.

Your code

User-agent: *
Disallow: /wp-admin/
Disallow: /cart
Disallow: /checkout
Allow: /wp-admin/admin-ajax.php

Sitemap: https://www.example.com/sitemap.xml

Like this tool?

I can build one for your website.

See how

FAQs

Robots.txt generator questions

Does robots.txt hide pages from Google?

No. It asks crawlers not to visit; use noindex to keep a page out of results.

Does Shopify let me edit robots.txt?

Yes, through the robots.txt.liquid template.

More free tools

WebsiteWebsite ROI calculatorSee how much more a better website could earn you each month.WebsiteColour contrast checkerCheck your text colours are readable and meet WCAG.WebsiteWord & reading time counterCount words, characters and reading time for any text.

Tell me what you need. I’ll give you an honest plan.

A free 20-minute video call with me. A written plan and a fixed quote within three working days.

Book a free call