Overview
robots.txt API: fetch and parse a site robots.txt with user-agent rules and sitemap links. Param: url. · Keywords: crawl rules, seo, scraping, data · Built with @piprail/sdk
Protocol
x402
Price
$0.0030
Payment Asset
USDC
Payment Network
base
Category
data
Provider
x402render
Source
self-registered
Indexed
2026-06-28 00:44:22
Health
Status
healthy
Latency (p50)
189ms
Uptime (30d)
100.0%
Reliability Score
100/100
Last Checked
2026-08-25 14:24:13
Last Healthy
2026-08-25 14:24:13
Consecutive Failures
0
x402 Payment Validation
Payment Requirements
Valid
Asset Verified
—
Facilitator
—
Recent Health Checks
| Time | Status | HTTP | Latency | Error |
|---|---|---|---|---|
| 2026-08-25 14:24:13 | healthy | 402 | 160ms | |
| 2026-08-25 08:21:44 | healthy | 402 | 649ms | |
| 2026-08-25 03:11:38 | healthy | 402 | 182ms | |
| 2026-08-24 19:40:38 | healthy | 402 | 204ms | |
| 2026-08-24 11:56:33 | healthy | 402 | 260ms | |
| 2026-08-24 07:57:44 | healthy | 402 | 137ms | |
| 2026-08-23 20:46:09 | healthy | 402 | 189ms | |
| 2026-08-23 19:53:19 | healthy | 402 | 183ms | |
| 2026-08-23 09:03:01 | healthy | 402 | 162ms | |
| 2026-08-23 01:47:30 | healthy | 402 | 1019ms | |
| 2026-08-22 17:07:59 | healthy | 402 | 191ms | |
| 2026-08-22 14:09:44 | healthy | 402 | 147ms |