← Back to directory

Check if a URL is allowed to be fetched according to the website's robots.txt fi...

https://www.minifetch.com/api/v1/x402/preflight/url-check

Overview

Check if a URL is allowed to be fetched according to the website's robots.txt file. Use this super-cheap endpoint before using our other paid endpoints to avoid spending extra on un-fetchable URLs

Protocol x402
Price $0.0010
Payment Asset USD Coin
Payment Network Base
Category uncategorized
Provider minifetch
Source bazaar
Indexed 2026-03-02 01:12:24

Health

Status healthy
Latency (p50) 438ms
Uptime (30d) 100.0%
Reliability Score 95/100
Last Checked 2026-05-05 20:55:17
Last Healthy 2026-05-05 20:55:17
Consecutive Failures 0

x402 Payment Validation

Payment Requirements Valid
Asset Verified Known USDC
Facilitator

Input Schema

{ "discoverable": true, "method": "GET", "queryParams": { "url": { "description": "The URL to check against robots.txt", "required": true, "type": "string" } }, "type": "http" }

Output Schema

{ "properties": { "queryParameters": { "type": "object" }, "results": { "type": "array" }, "success": { "type": "boolean" } }, "type": "object" }

Recent Health Checks

Time Status HTTP Latency Error
2026-05-05 20:55:17 healthy 402 438ms
2026-05-05 18:01:17 healthy 402 431ms
2026-05-05 14:49:19 healthy 402 435ms
2026-05-05 11:04:42 healthy 402 443ms
2026-05-05 09:28:47 healthy 402 443ms
2026-05-05 06:15:54 healthy 402 434ms
2026-05-05 01:46:30 healthy 402 454ms
2026-05-04 22:58:51 healthy 402 429ms
2026-05-04 19:44:22 healthy 402 437ms
2026-05-04 17:32:03 healthy 402 440ms
2026-05-04 16:04:35 healthy 402 430ms
2026-05-04 13:42:02 healthy 402 441ms
2026-05-04 10:18:47 healthy 402 437ms
2026-05-04 09:17:32 healthy 402 439ms
2026-05-04 07:31:47 healthy 402 430ms
2026-05-04 04:52:31 healthy 402 440ms
2026-05-04 03:27:43 healthy 402 439ms
2026-05-03 23:54:21 healthy 402 442ms
2026-05-03 22:36:49 healthy 402 433ms
2026-05-03 20:52:32 healthy 402 432ms