Technical SEO KPIs: Crawl Health, Indexation, and Core Web Vitals

Overview

Technical SEO KPIs measure the health of the infrastructure that search engines use to discover, render, and index your content. Unlike executive KPIs that track business outcomes, technical KPIs are diagnostic: they tell you when the pipeline from request to index is broken. Without these metrics, every content and link investment sits on an unverified foundation. Google's 2025 crawling documentation confirms that sites with consistent crawl health signals see faster content discovery (Google Search Central, 2025).

Crawl Health KPIs

Crawl Efficiency Ratio

Track the ratio of crawled URLs to discovered URLs per week from Google Search Console.

Crawl Efficiency = unique_urls_crawled / unique_urls_discovered

A ratio below 60% indicates crawl budget issues. Common causes: server latency exceeding 200ms, too many low-value URLs in the sitemap, or robots.txt directives blocking important paths. The 2025 Google I/O session on crawling emphasized auditing crawl efficiency quarterly and removing URL patterns that consume budget without providing indexable content (Google I/O 2025).

Server Response Metrics

TTFB at P95: target under 800ms for crawl requests. Slow responses reduce the number of URLs Googlebot can fetch per unit time.
4xx/5xx Rate on Crawl Requests: target under 1%. A rate above 5% causes Googlebot to reduce crawl frequency.
Crawl Request Distribution: Googlebot should hit all sections proportionally. If 70% of requests hit the blog and only 5% hit product pages, the crawl is unbalanced.

import pandas as pd
from urllib.parse import urlparse

crawl_log = pd.read_csv('googlebot_crawl_log.csv')
crawl_log['path_section'] = crawl_log['request_uri'].apply(
    lambda x: urlparse(x).path.split('/')[1]
)
distribution = crawl_log['path_section'].value_counts(normalize=True) * 100

Indexation KPIs

Indexation Rate

The most important indexation metric is the percentage of discovered, valuable URLs that are indexed.

Indexation Rate = indexed_urls / (indexed_urls + excluded_urls_from_sitemap)

Pull indexed vs. excluded counts from the GSC Index Coverage report. URLs marked as "Crawled - currently not indexed" (CCNI) are the most concerning. A CCNI rate above 5% signals content quality or technical issues. John Mueller clarified in a 2025 office hours that CCNI is often caused by insufficient content depth or duplicate content signals (Mueller, 2025).

Cannibalization Signals

Track keywords where two or more pages appear in the top 20 results. Ahrefs found that 42% of sites with over 10,000 pages have keyword cannibalization affecting at least 15% of tracked keywords (Ahrefs Blog, 2025). Set a weekly alert when this number increases by more than 5% week over week.

Core Web Vitals KPIs

Track origin-level CWV from the GSC Core Web Vitals report.

CWV Pass Rate = (good_lcp + good_inp + good_cls) / total_pages * 100

Set a 90% pass rate target at the origin level. Pages failing any metric go into a remediation queue with owner and fix date. Use the CrUX API for programmatic tracking:

import requests
API_URL = "https://chromeuxreport.googleapis.com/v1/records:queryRecord"
response = requests.post(API_URL, json={"origin": "https://yoursite.com"}).json()
lcp_p75 = response['record']['metrics']['largest_contentful_paint']['percentiles']['p75']

Track mobile and desktop separately. Mobile LCP target is under 2.5 seconds. INP should be under 200ms for a "good" rating across both form factors.

Audit Checklist

[ ] Calculate crawl efficiency (crawled/discovered) weekly and trend over 90 days
[ ] Verify server response times for Googlebot TTFB stay under 800ms at P95
[ ] Monitor indexation rate monthly, with specific attention to CCNI URLs
[ ] Set up keyword cannibalization alert threshold
[ ] Track CWV pass rate at origin level and for top 200 landing pages
[ ] Implement CrUX API data pull for automated CWV reporting
[ ] Review crawl distribution by site section quarterly for balance

Closing

Technical SEO KPIs are the early warning system for your organic search program. Crawl health, indexation rates, and Core Web Vitals tell you whether the infrastructure is sound before you invest in content and links. Audit your current technical monitoring against this framework and close gaps between what you track and what you need to catch before problems compound.

References

[1] Google. "Crawling and Indexing Best Practices." Google Search Central, 2025. https://developers.google.com/search/docs/crawling-indexing

[2] Google I/O. "Crawling Best Practices (Google I/O 2025)." YouTube, 2025. https://www.youtube.com/watch?v=google-io-2025-crawling

[3] Mueller, John. "Search Central Office Hours, May 2025." Google Search Central, 2025. https://developers.google.com/search/blog/2025/05/office-hours-may

[4] Ahrefs. "Keyword Cannibalization: Data and How to Fix It." Ahrefs Blog, 2025. https://ahrefs.com/blog/keyword-cannibalization/

Ready to Build Your Dream Website?

Let's discuss your project and create something amazing together.