Check if a URL is allowed to be fetched according to the website's robots.txt fi...
https://minifetch.com/api/v1/x402/preflight/url-checkOverview
Check if a URL is allowed to be fetched according to the website's robots.txt file. Use this super-cheap endpoint before using our extraction endpoints to avoid spending extra on un-fetchable URLs & get more granular info about why a URL may or may not return data
Protocol
x402
Price
$0.0010
Payment Asset
USD Coin
Payment Network
Base
Category
uncategorized
Provider
minifetch
Source
bazaar
Indexed
2026-03-02 01:01:28
Health
Status
healthy
Latency (p50)
433ms
Uptime (30d)
100.0%
Reliability Score
95/100
Last Checked
2026-05-14 02:29:28
Last Healthy
2026-05-14 02:29:28
Consecutive Failures
0
x402 Payment Validation
Payment Requirements
Valid
Asset Verified
Known USDC
Facilitator
—
Input Schema
{
"method": "GET",
"queryParams": {
"url": "https://github.com"
},
"type": "http"
}
Output Schema
{
"example": {
"results": [
{
"data": {
"allowed": true,
"crawlDelay": 1,
"message": "Access allowed by robots.txt",
"url": "https://github.com"
}
}
],
"success": true
},
"type": "json"
}
Recent Health Checks
| Time | Status | HTTP | Latency | Error |
|---|---|---|---|---|
| 2026-05-14 02:29:28 | healthy | 402 | 441ms | |
| 2026-05-13 23:05:51 | healthy | 402 | 433ms | |
| 2026-05-13 16:23:26 | healthy | 402 | 436ms | |
| 2026-05-13 14:40:29 | healthy | 402 | 857ms | |
| 2026-05-13 09:10:27 | healthy | 402 | 428ms | |
| 2026-05-13 04:40:35 | healthy | 402 | 429ms | |
| 2026-05-12 21:44:52 | healthy | 402 | 914ms | |
| 2026-05-12 20:10:11 | healthy | 402 | 428ms | |
| 2026-05-12 13:45:23 | healthy | 402 | 430ms | |
| 2026-05-12 09:47:47 | healthy | 402 | 433ms | |
| 2026-05-12 06:44:48 | healthy | 402 | 841ms | |
| 2026-05-12 03:07:08 | healthy | 402 | 428ms | |
| 2026-05-11 22:14:46 | healthy | 402 | 430ms | |
| 2026-05-11 16:57:53 | healthy | 402 | 435ms | |
| 2026-05-11 16:17:49 | healthy | 402 | 437ms | |
| 2026-05-11 11:14:25 | healthy | 402 | 428ms | |
| 2026-05-11 06:07:50 | healthy | 402 | 429ms |