100 Websites Audited: What Real-World Data Reveals About Speed, Schema, and AI Bots
An empirical forensic benchmark of 100 live websites measuring server latency, mobile LCP bottlenecks, Knowledge Graph adoption, and llms.txt AI discovery readiness.
Executive Summary & Empirical Findings
We conducted an automated forensic audit across 100 live production domains across four distinct industries: SaaS and Developer Tools, E-Commerce and DTC Brands, Indie Hackers, and Digital Agencies. Our data shows a striking divergence: while 96% of developer tools publish llms.txt files, 76.9% of websites fail Google mobile Largest Contentful Paint threshold, and 28.6% lack structured data entirely.
Core Technical Conclusions:
- 76.9% of audited production websites failed Google mobile Largest Contentful Paint target (LCP <= 2.5s), with an overall cross-sector average of 4.58 seconds.
- E-Commerce brands exhibited the heaviest payloads (849.5KB average HTML) and slowest load times (6.51s average LCP), driven by uncompressed assets and marketing trackers.
- Indie Hackers delivered the leanest HTML payloads (141.9KB average) and fastest load times (2.86s average LCP), but suffered the lowest Schema.org adoption at just 59.1%.
- A striking AI discovery divide emerged: 96.0% of SaaS and Developer Tools have published llms.txt files, compared to only 27.3% of Digital Agencies and 36.4% of Indie Hackers.
- 3.3% of audited production domains still serve uncompressed raw HTML over the wire, completely missing Gzip and Brotli compression.
Testing Methodology & Reproducibility
All 100 domains were queried concurrently using automated HTTP/2 and HTTP/1.1 telemetry probes following W3C Navigation Timing standards. DOM structures, JSON-LD @graph blocks, script tags, image formats, and llms.txt endpoints were extracted and evaluated under standardized browser emulation profiles.
Authoritative Standards & Data Sources
Official W3C specification for measuring page navigation metrics
Official performance thresholds for LCP, INP, and CLS
Official vocabulary specification for structured data and entity search
Community standard for curating markdown content for AI crawlers
Related Research Guides & Actionable Analysis
100-Website Forensic Audit: Real-World Averages for Speed, Schema, and AI Crawlers
We audited 100 live production websites across SaaS, E-Commerce, Indie Hackers, and Agencies. Here is the empirical breakdown of real-world TTFB, mobile LCP failures, JSON-LD schema adoption, and llms.txt readiness.
Case Study: Why 15 Live Web Apps Failed Google Indexing and Speed Benchmarks
We conducted forensic audits on 15 live production apps and SaaS websites. Here is the empirical breakdown of the 4 recurring flaws that broke their indexation, destroyed TTFB, and blocked AI search citations.
Explore More Original Benchmark Datasets
The State of WordPress Performance 2026: 500-Site Empirical Benchmark
An empirical teardown of mobile Core Web Vitals, page weight, DOM depth, and cache hit ratios across production websites.
Sample: 100 digital agency homepages100 Agency Websites: What Their Homepages Reveal About Speed and SEO
We audited 100 digital agency homepages to document real-world performance bottlenecks and structured data adoption.
Sample: 200 technical publishing URLsAI Search Readiness Study: How AI Crawlers Parse Technical Content
Observational research examining GPTBot, PerplexityBot, and ClaudeBot crawling behavior across 200 technical documents.
Sample: 250 B2B SaaS marketing homepagesHow Much JavaScript Do SaaS Websites Actually Ship?
Benchmarking 250 B2B SaaS marketing homepages for script execution time, tracking pixel bloat, and hydration overhead.
Want custom performance telemetry for your site?
Run our in-browser diagnostic tools or test your site with VitalsSniper PRO.