# aicodingpricing indexability / crawler credibility evidence — 2026-05-26

Verdict: P0 BLOCKER FOUND. The site is not just “waiting for indexing”. Live HTML pages and `/sitemap.xml` are currently challenged by Cloudflare.

## Root cause
Cloudflare Bot Fight Mode / JS challenge is active. Live fetches to `/`, `/sitemap.xml`, `/ai-coding-pricing-calculator/`, and `/claude-code-vs-cursor/` return HTTP 403 with `cf-mitigated: challenge`. `robots.txt` returns 200 and allows crawling.

This makes the current indexing problem primarily a Cloudflare crawlability/discovery risk, not a local `robots.txt`, canonical, noindex, or static build issue.

## GSC URL Inspection
- Homepage `https://aicodingpricing.com/`: PASS; Submitted and indexed; crawled as MOBILE; last crawl `2026-05-24T11:47:01Z`; rich results PASS.
- Core tool page `https://aicodingpricing.com/ai-coding-pricing-calculator/`: NEUTRAL; URL is unknown to Google.
- Comparison page `https://aicodingpricing.com/claude-code-vs-cursor/`: NEUTRAL; URL is unknown to Google.
- Sitemap submitted in GSC: `https://aicodingpricing.com/sitemap.xml`, submitted/downloaded `2026-05-21T18:45:22Z`, 0 errors/warnings in API.

## Live HTTP evidence
- `/`: 403 challenge for default, Googlebot desktop, Googlebot smartphone, Bingbot, and browser UA simulations.
- `/sitemap.xml`: 403 challenge for default, Googlebot desktop, Googlebot smartphone, Bingbot, and browser UA simulations.
- `/robots.txt`: 200 for all tested UAs; content allows `/`, disallows only `/_state/`, points to sitemap.
- `/ai-coding-pricing-calculator/`: 403 challenge for all tested UAs.
- `/claude-code-vs-cursor/`: 403 challenge for all tested UAs.

Caveat: UA simulation is not the same as verified Googlebot/Bingbot IP. But Cloudflare Bot Fight Mode is confirmed enabled, and public crawl paths are challenged unless Cloudflare classifies the request as verified bot. This is enough to treat as P0 crawlability risk.

## Cloudflare read-only evidence
- Zone: `aicodingpricing.com`, active, full setup.
- Firewall rules: 0 legacy rules.
- Filters: 0.
- Page rules: 0 active.
- WAF setting: off.
- Security level: medium.
- Browser check: on.
- Bot management: `fight_mode=true`, `enable_js=true`, crawler/content/AI bots protection disabled.

## Local SEO/index policy evidence
- Local `npm run verify`: PASS; 23 routes checked, 4 redirects checked, 21 contract routes.
- Local sitemap has 22 URLs.
- Page-copy matrix indexable routes: 21.
- Route contract/page-copy mark canonical pages as indexable; redirect aliases excluded from sitemap.
- No evidence from local contract that core pages are intentionally noindex.

## P0 fix list
1. Disable Cloudflare Bot Fight Mode for this zone, or create a verified-bot / SEO-file bypass before any resubmission. Minimum: `/`, `/sitemap.xml`, `/robots.txt`, P0 indexable pages, and static assets must return 200 HTML/XML to Googlebot/Bingbot.
2. Keep `robots.txt` as-is unless changing format only; it is not the blocker.
3. After CF fix, re-run live checks from non-browser clients and Googlebot/Bingbot UA simulations; expected: no `cf-mitigated: challenge`, status 200 for HTML and sitemap.
4. Re-run GSC URL Inspection on `/ai-coding-pricing-calculator/` and `/claude-code-vs-cursor/`; request indexing only after live fetch is clean.
5. Submit/resubmit sitemap in GSC and Bing Webmaster after CF fix. Do not resubmit before the 403 challenge is gone.
6. Add an IndexNow key file and submit homepage + sitemap + P0 URLs after CF fix; this is P2 for Google but useful for Bing/Yandex/Naver/Yep discovery.

## Residual risk
- Homepage is already indexed, so `submitted=22 indexed=0` in previous synthesis may reflect sitemap-report lag or a different GSC coverage aggregate, not total site indexability. Internal P0 URLs are still unknown to Google and must be resubmitted after crawl access is clean.
- Bing URL Inspection was not available through local credentials in this run; Bingbot UA live evidence is used as alternative evidence.
- Actual verified Googlebot may be treated differently by Cloudflare than UA simulation. Still, Bot Fight Mode is active and should not be left on for an SEO-driven launch unless verified-bot bypass is proven.

## Artifact
Structured evidence JSON: `/mnt/HC_Volume_105300089/projects/aicodingpricing/reports/site-review/site-review-20260526/aicodingpricing-indexability-evidence.json`