Audit and Rebuild Website Indexing Architecture

Indexing Issue Auditor & Technical SEO Architect acts as a senior technical SEO architect, web infrastructure engineer, and site reliability auditor, performing a deep-dive scan of a website's architecture to identify, diagnose, and fix crawl health issues, indexing blocks, and structural SEO failures - not just finding issues, but designing and rebuilding the site's architecture into a system Google fully trusts.

It accepts four input types: a directory path (scanning local sitemap.xml, robots.txt, and canonical logic in templates), exported Search Console CSV reports (404s, soft 404s, redirect loops), a public domain URL for a live scan of crawl depth and response codes, or architecture drafts (proposed URL structures or internal linking maps evaluated before deployment).

The audit runs through ten mandatory phases in strict order. Phase 1 (Indexing System Health) detects 404s, "crawled but not indexed," soft 404s, and noindex tags, explaining why Google rejected indexing and classifying the cause as content, technical, or structural. Phase 2 (Crawl Architecture) analyzes crawl depth, identifies orphan pages, and maps the internal linking graph to find crawl-budget waste. Phase 3 (Sitemap Architecture Audit) validates that sitemaps contain only indexable URLs (no redirects, no 404s), segments sitemaps by type (pages/posts/products), checks canonical alignment, and for multi-region sites validates that hreflang tags have correct return links matching sitemap entries. Phase 4 (URL Architecture Design) identifies URL duplication patterns and parameter-heavy URLs, proposing a clean URL architecture model. Phase 5 (Redirect & Link Flow) identifies redirect chains and loops and maps internal link equity flow into a clean redirect flow map. Phase 6 (Content Quality Engine) detects thin pages, duplicate clusters, and auto-generated content, proposing a consolidation plan. Phase 7 (Technical Server Health) checks for 5xx errors, 403 blocks, and API failures affecting crawler stability, and for JavaScript-heavy frameworks (Next.js/Nuxt) verifies whether Googlebot sees the same content as users or whether content is hidden behind client-side hydration Google can't complete. Phase 8 (Performance & Resource Loading) audits render-blocking JS, CSS delays, and lazy-loading errors from a structural perspective. Phase 9 (Internal Linking System Design) redesigns the internal linking graph into a topical SEO silo (hub-and-spoke) model. Phase 10 (Final Rebuild Plan) produces a step-by-step cleanup order and a 30-day SEO stabilization roadmap.

Every audit must produce a Master Issue Control Table with columns for issue number, issue, affected layer (SEO/Crawl/Server/Content), affected URLs/patterns, root cause, technical fix, structural fix, priority, and status. Two worked examples: a local directory scan finding robots.txt blocking /public/static but missing a /api disallow rule, fixed by adding the missing rule and verifying the sitemap only includes app routes; and a GSC report showing 40% "crawled - currently not indexed," diagnosed as architectural duplication between parameter-based and static URLs, fixed with strict canonicalization and parameterized URL handling in robots.txt.

Best practices: always pair the fix with a structural redesign rather than just reporting the issue; never assume a cause - verify every response code and link logic; quantify the system-level impact of every architectural choice; and keep output engineering-level and actionable, no fluff. Common pitfalls flagged: treating indexing issues as content-only when they're often architectural (check server status codes and canonical logic first) and ignoring crawl depth (design a flatter hierarchy, max 3 clicks from home).

Use this skill when preparing or auditing a site for Google Search Console health, encountering "discovered but not currently indexed" or other mass indexing errors, auditing sitemaps/robots.txt/URL structures for crawl budget waste, designing a new site architecture or content silo migration, or performing a site reliability audit focused on SEO stability and redirect integrity. It cannot initiate a Google Search Console "Request Indexing" action directly (instructions only), and it can identify render-blocking assets but relies on provided text/code for deep DOM analysis rather than live rendering.

Technical SEO practitioners and web infrastructure engineers diagnosing mass indexing failures, crawl budget waste, or redirect/canonicalization problems who need a structured, engineering-level audit with both immediate fixes and a longer-term architectural rebuild plan.