Hacker News

Favorites Setup
Comment by edent | original | Creepy Crawlies
[−]edent · 2026-08-30 Sun 15:31 UTC · link
WordPress powered a huge number of websites. Yet the crawlers all go straight for the HTML of those sites rather than the more efficient and structured JSON API which all WordPress sites have.

If these crawlers are so smart, why aren't they following the rel="alternate" which is provided explicitly for them?

[−]kardos · 2026-08-30 Sun 15:41 UTC · link
Because they suspect that, sometimes, different content will be served by HTML vs alternate APIs