MASHPriceBot
MASHPriceBot is an automated checker of publicly listed prices and availability on online stores.
- It reads
robots.txt first and obeys it, including Crawl-delay.
- It leaves at least 5 seconds between requests and reads one store at a time.
- It backs off when it receives a 429 or
Retry-After.
- On Shopify stores its requests are signed with Web Bot Auth; the public key is at /.well-known/http-message-signatures-directory.
To stop it visiting a site, disallow MASHPriceBot in robots.txt.