← Back to directory

Parse robots.txt files and check URL crawl permissions per user-agent.

https://orbisapi.com/proxy/robots-txt-parser-api-a263c4/openapi

Overview

Parse robots.txt files and check URL crawl permissions per user-agent.

Protocol x402
Price
Payment Asset USD Coin
Payment Network eip155:8453
Category uncategorized
Provider orbisapi
Source bazaar
Indexed 2026-04-27 08:52:48

Health

Status healthy
Latency (p50) 243ms
Uptime (30d) 100.0%
Reliability Score 92/100
Last Checked 2026-05-01 05:21:36
Last Healthy 2026-05-01 05:21:36
Consecutive Failures 0

x402 Payment Validation

Payment Requirements Valid
Asset Verified Known USDC
Facilitator

Recent Health Checks

Time Status HTTP Latency Error
2026-05-01 05:21:36 healthy 402 261ms
2026-05-01 03:02:18 healthy 402 408ms
2026-05-01 00:57:20 healthy 402 352ms
2026-04-30 22:37:09 healthy 402 310ms
2026-04-30 21:16:59 healthy 402 224ms
2026-04-30 18:38:25 healthy 402 243ms
2026-04-30 16:59:18 healthy 402 231ms
2026-04-30 14:14:12 healthy 402 210ms
2026-04-30 12:10:51 healthy 402 190ms
2026-04-30 09:48:40 healthy 402 694ms
2026-04-30 07:44:34 healthy 402 793ms
2026-04-30 06:33:09 healthy 402 158ms
2026-04-30 04:13:49 healthy 402 215ms
2026-04-30 03:24:14 healthy 402 390ms
2026-04-30 00:16:26 healthy 402 312ms
2026-04-29 22:25:06 healthy 402 220ms
2026-04-29 20:27:46 healthy 402 226ms
2026-04-29 18:33:35 healthy 402 261ms
2026-04-29 15:48:58 healthy 402 240ms
2026-04-29 14:49:04 healthy 402 221ms