How Comprable visits websites
Comprable checks whether AI assistants can read online stores. To do that it reads public pages: the home page, robots.txt, the sitemap and a sample of product pages. If you got here from your server logs, this is what we do and how to ask us to stay out.
How to recognise us
Every Comprable request carries this header:
X-Comprable-Check: https://comprable.co/bot (holacomprable@gmail.com)
Pages are requested with a browser user-agent, to see the site the way a shopper does. We also make a single request to the home page with the OAI-SearchBot user-agent and another with the PerplexityBot one, to check whether the server lets them in. Those two requests carry the Comprable header too: they are not the real bots.
What we do and what we don’t
- We only read public pages. We do not submit forms, log in, add anything to a cart or test for security flaws.
- Few requests: a normal check usually asks for about 25 addresses; an in-depth review, normally fewer than 100. On sites that have not hired us, at most two at a time.
- We keep the result of the check (what was found), not the content of the pages. We do not use your site to train AI models.
How to ask us to stay out
Add this to your robots.txt:
User-agent: Comprable Disallow: /
We honour it on every site that has not hired us: the public checker, sector studies and competitor comparisons. We also honour User-agent: * with Disallow: /, and any specific paths you exclude. If your robots.txt cannot be read because of a server error, we do not go in.
You can also write to holacomprable@gmail.com and we will exclude you by hand.
When the store itself hires us
If a site owner hires us for an audit, the weekly monitoring (Centinela) or the “AI-readable” badge, we review their site even if their robots.txt closes it to crawlers, because they asked us to. In that case, a robots.txt that blocks AI bots is often one of the findings.
Contact: holacomprable@gmail.com.