Vextorium · robots.txt & AI crawler rules
https://api.vextorium.com/robots-txt-pagoOverview
Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your agent scrapes.
Protocol
x402 (POST)
Price
—
Payment Asset
USD Coin
Payment Network
eip155:8453
Category
tools/utilities
Provider
vextorium
Source
bazaar
Indexed
2026-10-02 16:24:14
Health
Status
healthy
Latency (p50)
153ms
Uptime (30d)
100.0%
Reliability Score
97/100
Last Checked
2026-10-08 08:42:20
Last Healthy
2026-10-08 08:42:20
Consecutive Failures
0
x402 Payment Validation
Payment Requirements
Valid
Asset Verified
Known USDC
Facilitator
—
Recent Health Checks
| Time | Status | HTTP | Latency | Error |
|---|---|---|---|---|
| 2026-10-08 08:42:20 | healthy | 402 | 92ms | |
| 2026-10-08 00:31:50 | healthy | 402 | 132ms | |
| 2026-10-07 13:46:53 | healthy | 402 | 186ms | |
| 2026-10-06 20:19:44 | healthy | 402 | 153ms | |
| 2026-10-06 10:19:42 | healthy | 402 | 208ms | |
| 2026-10-05 15:21:40 | healthy | 402 | 56ms |