← Help Center

Fix crawler access blocks on product pages

Robots rules, firewalls, geoblocks, or bot blocking can stop Google from reading products and structured data.

Symptom

The page works for the team but fails for Googlebot because of robots, security middleware, WAF rules, geoblocks, or unstable responses.

Exact cause

Google flagged this state because storeSteady detected a data inconsistency between your Shopify store, Google Merchant Center, or storefront markup. Review the details below to understand the specific mismatch.

This page is about the “works for me” trap. A storefront can render in a browser and still be effectively blocked for Google crawlers.

Which system wins

Public, crawlable HTTP 200 pages win. Robots blocks, noindex, 4xx or 5xx responses, geo rules, and bot blocking break eligibility before feed quality matters.

Correct edit point

Fix robots.txt, WAF or CDN rules, geoblocking, rate limits, and security middleware before changing theme markup.

Manual fix steps

  1. Compare the values shown in the Shopify, Google, and Markup columns below.
  2. Identify which source has the incorrect or missing data.
  3. Update the data in the appropriate system.
  4. Trigger a sync to verify the issue is resolved.

Validation steps

  1. Load the affected product or image URL as an anonymous visitor and confirm it returns a clean HTTP 200 response.
  2. Use URL Inspection or Merchant Center diagnostics to confirm Google can fetch the page or asset.
  3. Wait for Google to recrawl and then re-check the live diagnostic state.

Expected resolution time

Resolution time depends on the specific issue. Most changes are processed by Google within 24–72 hours.

Free Scanner

Scan your store for this exact issue

Check whether your storefront is showing the same public signal problem described in this article before you connect the app. Start with storefront access blocked.

Source article: Fix crawler access blocks on product pages

Related Issues

Fix landing page errors blocking Google Shopping visibility
Broken product URLs, bad redirects, 4xx/5xx responses, or crawler blocks can stop approval.
Remove storefront password protection blocking Google
Google cannot approve or crawl product pages behind a storefront password.
Fix image link internal error in Merchant Center
Google could not process your image yet. Learn when to wait and when to replace the asset.
Fix mismatched domains in Merchant Center
Your product links do not match the domain claimed in Merchant Center.