Skip to main content
Webflow fix guide

robots.txt Blocks Important Pages on Webflow

Your robots.txt file contains Disallow rules that may prevent search engines from crawling important pages on your site.

Issue ID: CRAWL-ROBOTS-BLOCKS-001Severity: majorEffort: S

Why this matters

Your robots.txt file contains Disallow rules that may prevent search engines from crawling important pages on your site. This affects how clearly search engines understand the page and how persuasive it looks in search results.

Webflow fix path

How to fix this on Webflow

Use page settings, Designer, or project settings.

  1. 1Locate the element or setting causing the issue.
  2. 2Update it in Webflow.
  3. 3Publish and validate the output.

After you fix it

  • Use the robots.txt tester in Google Search Console to verify important URLs are allowed

See every Webflow issue on your site

This is one issue of many the full audit checks. Run a free scan to see what else might be holding your site back.

Run Free Audit