Overview
robots.txt API: fetch and parse a site robots.txt with user-agent rules and sitemap links. Param: url. · Keywords: crawl rules, seo, scraping, data · Built with @piprail/sdk
Protocol
x402
Price
$0.0030
Payment Asset
USDC
Payment Network
base
Category
data
Provider
x402render
Source
self-registered
Indexed
2026-06-28 00:44:22
Health
Status
healthy
Latency (p50)
208ms
Uptime (30d)
100.0%
Reliability Score
95/100
Last Checked
2026-10-11 15:19:54
Last Healthy
2026-10-11 15:19:54
Consecutive Failures
0
x402 Payment Validation
Payment Requirements
Valid
Asset Verified
—
Facilitator
—
Recent Health Checks
| Time | Status | HTTP | Latency | Error |
|---|---|---|---|---|
| 2026-10-11 15:19:54 | healthy | 402 | 737ms | |
| 2026-10-11 02:18:30 | healthy | 402 | 135ms | |
| 2026-10-10 19:00:54 | healthy | 402 | 165ms | |
| 2026-10-10 05:27:10 | healthy | 402 | 1995ms | |
| 2026-10-09 10:40:31 | healthy | 402 | 208ms | |
| 2026-10-08 22:29:14 | healthy | 402 | 145ms |