Vextorium · robots.txt & AI crawler rules
https://api.vextorium.com/v1/robots-txtOverview
Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your agent scrapes.
Protocol
x402 (POST)
Price
—
Payment Asset
USD Coin
Payment Network
eip155:8453
Category
tools/utilities
Provider
vextorium
Source
bazaar
Indexed
2026-10-03 21:23:32
Health
Status
healthy
Latency (p50)
163ms
Uptime (30d)
100.0%
Reliability Score
97/100
Last Checked
2026-10-07 13:19:52
Last Healthy
2026-10-07 13:19:52
Consecutive Failures
0
x402 Payment Validation
Payment Requirements
Valid
Asset Verified
Known USDC
Facilitator
—
Recent Health Checks
| Time | Status | HTTP | Latency | Error |
|---|---|---|---|---|
| 2026-10-07 13:19:52 | healthy | 402 | 609ms | |
| 2026-10-07 00:23:20 | healthy | 402 | 149ms | |
| 2026-10-06 08:28:47 | healthy | 402 | 163ms | |
| 2026-10-05 14:40:14 | healthy | 402 | 211ms | |
| 2026-10-05 02:34:36 | healthy | 402 | 53ms | |
| 2026-10-04 20:55:07 | healthy | 402 | 147ms |