We Crawled 18 AI SEO Tools: They Deleted 2,556 Pages
Discover how we crawled 18 AI SEO tools and found they deleted 2,556 pages, shedding light on the impact of programmatic SEO strategies.
programmatic seo | Updated October 2026 | 10 min read | hyperank Editorial Team
We crawled 18 AI SEO and AI visibility vendors, checked 27,004 URLs from their sitemaps and the Wayback Machine, and confirmed 2,556 pages that now return 404 or 410 errors. Most of the deletions are concentrated in a few companies: Tely accounts for 1,337 dead pages, AirOps for 808, Scrunch for 92 and SEOWriting for 64. Another 1,861 URLs redirect to a homepage. Yet the damage to rankings is small: only 6 of 1,693 competitor ranking keywords still point at a dead page, while 116 point at redirects, some to pages on unrelated topics. Here's what we found, how we measured it, and what it suggests about programmatic SEO and AI content in 2026.
Publishing at scale is easy now. Deciding which pages deserve to stay published is the hard part.
Key Findings
The headline numbers come from a crawl of 18 vendors in AI content, AI SEO and AI visibility, run in early October 2026, with ranking data from September 2026.
| Finding | Number |
|---|---|
| Vendors crawled | 18 |
| URLs checked | 27,004 |
| Confirmed dead pages (404 or 410) | 2,556 |
| URLs redirecting to a homepage | 1,861 |
| Soft 404s | 14 |
| Competitor ranking keywords analyzed | 1,693 |
| Ranking keywords pointing at a dead page | 6 |
| Ranking keywords pointing at a redirect | 116 |
- Concentrated deletions: Four companies (Tely, AirOps, Scrunch and SEOWriting) account for 2,301 of the 2,556 dead pages, about 90%.
- Clean-up mostly worked: Very few ranking keywords still lead to an error page.
- Redirects are messy: Some redirected pages still rank while sending visitors to a page about something else.
Method: Sitemaps, the Wayback Archive, Double-Checked Status Codes and Ranking Data
We combined each vendor's current sitemap with historical URLs from the Wayback Machine, so we could find pages that existed before but have since been removed. Every URL was fetched and classified by its current HTTP status, and dead URLs were re-checked slowly to rule out temporary errors.
- URL discovery: Every sitemap URL plus every archived URL for each domain.
- Classification: Live, dead (404 or 410), redirect, redirect to homepage, soft 404, or blocked.
- Double checks: Dead URLs were re-requested at a slower rate before being counted.
- Rankings: Each vendor's top 100 Google US ranking keywords came from DataForSEO, and we checked the live status of each ranking URL.
Exclusions and caveats
| Issue | How we handled it |
|---|---|
| babylovegrowth.ai returns random 404s | Its 819 404s are marked unreliable and not counted as dead |
| higoodie.com blocks crawlers (403) | All 233 URLs marked blocked or unknown |
| Reasons for deletion | Not knowable from a crawl; we report what happened, not why |
| Point in time | Statuses reflect early October 2026 and may have changed |
We also included hyperank.ai itself: all 115 of our URLs were live at the time, though we're a much smaller site, so that isn't a like-for-like comparison.
Who Deleted What
Deleted pages are heavily concentrated. Tely has the most dead pages, followed by AirOps, Scrunch and SEOWriting, and the deletions cluster in specific sections of each site rather than being spread evenly.
| Vendor | URLs checked | Dead (404/410) | Redirects | Where the dead pages are |
|---|---|---|---|---|
| Tely | 5,648 | 1,337 | 497 | /post (1,295), /case-study-new (23) |
| AirOps | 2,475 | 808 | 522 | /agencies (200), /schedule (120), /nlp-guide (104), /use-case-guides (101) |
| Scrunch | 762 | 92 | 297 | /scrunch-influencer-lists (29), /blog (19), /case-studies (7) |
| SEOWriting | 159 | 64 | 0 | /blog (64) |
| Profound | 947 | 47 | 249 | /leaderboard (22), /blog (8) |
| Outrank | 958 | 46 | 68 | /blog (43) |
| Peec AI | 254 | 42 | 46 | /careers (22), /blog (10) |
| Indexly | 692 | 34 | 190 | /blog (19), /use-cases (13) |
- Others with fewer deletions: Junia (29), RankYak (28), AthenaHQ (16), Arvow (8), Koala (3) and Otterly (2).
- Zero dead pages: Autoblogging.ai and SEObot had no confirmed dead pages, though Autoblogging.ai had 1,928 redirects, mostly from its /news section.
- Section patterns: Large deletions often affect a single templated section, such as Tely's /post pages or AirOps' /agencies pages, which is typical of programmatic SEO page sets.
Reading these numbers fairly
Deleting pages isn't automatically a mistake. Companies prune outdated content, consolidate thin pages and restructure sites for good reasons. The useful question is whether the removal was handled well, with redirects to relevant pages, or left links and rankings pointing at errors.
Redirects to Unrelated Pages That Still Rank
Of the 116 ranking keywords that now point at a redirect, 67 redirect to a page with a different slug, and several of those go to pages on a different topic entirely. Google can keep ranking the old URL for a while, but visitors land somewhere that doesn't answer their search.
| Vendor | Keyword (monthly volume, position) | Original page | Now redirects to |
|---|---|---|---|
| AirOps | chatgpt ai model (1,000, #5) | /blog/what-type-of-ai-model-does-chatgpt-use | /blog/airops-is-soc-2-type-ii-compliant |
| AirOps | make it sound more human (2,900, #42) | /blog/how-to-make-ai-writing-sound-more-human | /blog/how-to-make-content-rank-for-saas |
| AirOps | negative prompts (720, #15) | /blog/what-is-a-negative-prompt-in-ai | /blog/announcing-prompt-mining |
| AirOps | meta description character limit (480, #15) | /blog/how-many-characters-should-a-meta-description-be | /blog/answer-engine-optimization-tools |
| AirOps | advertising headline (320, #14) | /blog/how-to-write-a-google-ad-headline | /blog/cited-google-ai-overviews-framework |
- Not all redirects are bad: Many are clean moves to the same article at a new path, such as Profound moving /resources/articles/ pages to /articles/.
- The problem case: Redirecting a how-to article to an unrelated announcement or compliance post gives searchers the wrong answer.
- Likely outcome: Google tends to treat irrelevant redirects more like removed pages over time, so these rankings may fade.
For guidance on handling moves properly, see our guide to 301 redirects.
A redirect is a promise that the answer moved. When it points somewhere unrelated, the promise is broken for every visitor who follows it.
Only 6 of 1,693 Keywords Still Point at Dead Pages
Despite 2,556 dead pages, only 6 of the 1,693 competitor ranking keywords we analyzed still point at a page returning 404. That suggests most deleted pages weren't ranking, or that rankings dropped away once pages died.
| Vendor | Keyword | Monthly volume | Position |
|---|---|---|---|
| Profound | searchable | 22,200 | 9 |
| Profound | searchable ai | 1,900 | 7 |
| RankYak | hootsuite social media scheduler | 1,300 | 50 |
| RankYak | ai social media posts generator | 880 | 40 |
| RankYak | b2b marketing automation platform | 590 | 74 |
| RankYak | ai post generator | 480 | 47 |
- The biggest case: Profound's /blog/searchable-review page returned 404 while still ranking 9th for "searchable," a 22,200-search-a-month keyword.
- Overall status of ranking URLs: 1,567 were live, 116 redirected, 6 were dead and 4 returned unreliable 404s.
- Takeaway: Dead pages rarely keep ranking for long; the larger risk is the traffic and links lost before and after removal.
What It Means for AI Content and Programmatic SEO in 2026
The pattern suggests that programmatic SEO and AI-assisted publishing make it easy to create large page sets, and just as easy to end up deleting them. Large deletions concentrated in templated sections point to page sets that didn't earn their place, though we can't know each company's reasons.
- Quality over volume: Pages that don't attract traffic, links or citations become candidates for deletion; publishing fewer, stronger pages avoids that cycle.
- Plan the exit: When pruning, redirect to the most relevant page, not the homepage or an unrelated post.
- Watch your rankings: Check ranking URLs regularly so redirects and errors don't quietly waste demand.
- AI answers notice too: AI assistants that cite pages need those pages to exist; broken or irrelevant redirects can remove you from answers.
- Automation needs review: Our guides to AI blogging on auto-pilot and SEO automation cover how to automate without publishing junk.
Our own approach
hyperank publishes AI-assisted content too, so this applies to us. Our rule is that every number is sourced and every claim checked before it goes out, and we track what AI engines such as ChatGPT, Perplexity and Gemini say about brands, including whether they cite pages that still exist.
Key Takeaway: Programmatic SEO isn't the problem; unreviewed page sets and careless clean-ups are.
Download the Data
The dataset behind this study includes every URL we checked and its status, the full list of confirmed dead pages by vendor and section, and each ranking keyword with its volume, position and current page status.
| Dataset | Rows | Key fields |
|---|---|---|
| All crawled pages | 27,004 | Domain, URL, source, status, final URL, section |
| Dead pages | 4,431 | Dead, soft 404 and redirect-to-home URLs with Wayback last-seen dates |
| Competitor rankings | 1,702 | Keyword, volume, position, URL, live status |
| Ranking but broken | 122 | Keywords whose ranking page is dead or redirected |
To get the full dataset, contact the hyperank team and we'll share the files.
Conclusion
Across 18 AI SEO and AI visibility vendors, we found 2,556 confirmed dead pages and 1,861 redirects to homepages, concentrated in a handful of companies and templated sections. Only 6 of 1,693 ranking keywords still point at dead pages, but some redirects send searchers to unrelated content, which wastes demand that was hard to win.
Key Takeaways
- Scale: 27,004 URLs checked; 2,556 confirmed dead.
- Concentration: Tely (1,337), AirOps (808), Scrunch (92) and SEOWriting (64) account for about 90%.
- Rankings: Only 6 of 1,693 keywords point at dead pages; 116 point at redirects.
- Redirect quality: Some redirects send how-to traffic to unrelated posts.
- Lesson: Review programmatic SEO page sets before publishing and redirect carefully when pruning.
Want to see whether AI engines cite your pages or your competitors'? Try hyperank.
FAQ
How many pages did the 18 AI SEO tools delete?
Our crawl of 27,004 URLs across 18 AI SEO and AI visibility vendors in early October 2026 found 2,556 confirmed dead pages (404 or 410), plus 1,861 URLs redirecting to a homepage. Tely (1,337), AirOps (808), Scrunch (92) and SEOWriting (64) accounted for most of them, and only 6 of 1,693 ranking keywords still pointed at a dead page.
What is programmatic SEO?
Programmatic SEO is creating many pages from templates and data, such as location, comparison or integration pages, to target large sets of related searches.
Why do companies delete pages?
Common reasons include pruning thin or outdated content, consolidating similar pages and restructuring a site. A crawl shows what was removed, not why.
Do deleted pages lose their rankings?
Usually. In our data, only 6 of 1,693 ranking keywords still pointed at dead pages.
Are redirects to unrelated pages bad for SEO?
They give searchers the wrong answer, and Google may treat them more like removed pages over time. Redirect to the most relevant page instead.
How was the data collected?
From each vendor's sitemap and Wayback Machine archive, with every URL fetched and dead URLs re-checked, plus DataForSEO ranking data for each vendor's top 100 Google US keywords.
Methodology: hyperank crawled 18 vendor domains in early October 2026 using sitemap and Wayback Machine URLs, classified each by HTTP status and re-checked dead URLs. Ranking data is each vendor's top 100 Google US keywords from DataForSEO (September 2026). babylovegrowth.ai's 404s were excluded as unreliable and higoodie.com blocked crawling. Statuses may have changed since the crawl.
See where your brand stands
How six AI engines describe you, next to your competitors. About two minutes.