The PitchWhip reader.
If you saw PitchWhip in your logs, this page explains why.
What it visits, why it visits and how to ask it to leave your website alone.
Who we are
PitchWhip is a small UK company that makes sales tools. One of them helps food, drink and gift brands find independent shops that would suit their products.
What the reader does
It visits the public pages of a shop’s own website to see which products and brands the shop lists. It reads the same pages any visitor can open: product pages, the shop’s public product feed and its sitemap. Brands then use that to find shops that stock things like theirs.
The same reader sometimes checks a company’s public careers page or job board feed to see whether it is hiring.
What it does not do
- No logins, no accounts and nothing behind a password.
- No forms, no baskets and no checkout.
- No data about your customers or visitors. The only people details it may note are the business contacts your site publishes, such as a shop email, phone number or the owner’s name where the site gives it.
- No prices collected for resale.
- No browser, no scripts run and no images downloaded.
How often and how fast
About once a month per shop. It reads slowly: one request at a time per site, a few seconds apart on our monthly read, and a limited number of pages per visit. If your site answers “too many requests” or “unavailable”, it backs off and tries again days later.
It follows robots.txt
The reader fetches your robots.txt before anything else and obeys it. It follows the rules for PitchWhip if you have any, and otherwise your rules for all crawlers (*). Our monthly read respects a Crawl-delay too.
It identifies itself with the token PitchWhip in its user agent, which links to this page:
Mozilla/5.0 (compatible; PitchWhip/1.0; +https://pitchwhip.com/bot)
How to block it
Add this to your robots.txt to keep it off your whole site:
User-agent: PitchWhip Disallow: /
To keep it off only part of your site, list those paths instead of /. The change takes effect on its next visit.
Ask to be left out
Email hello@pitchwhip.com with your website address and we’ll stop reading it. A person reads every email.