Purchase: Public Robots.txt Crawler Policy — Rules, Delay & Sitemaps
https://api.the402.ai/v1/services/svc_049286bebcf3424b/purchaseOverview
Purchase: Public Robots.txt Crawler Policy — Rules, Delay & Sitemaps
Protocol
x402
Price
$0.05
Payment Asset
USD Coin
Payment Network
Base
Category
uncategorized
Provider
the402
Source
bazaar
Indexed
2026-07-18 22:14:54
Health
Status
down
Latency (p50)
240ms
Uptime (30d)
0.0%
Reliability Score
30/100
Last Checked
2026-09-05 13:54:02
Last Healthy
2026-08-02 16:20:46
Consecutive Failures
130
x402 Payment Validation
Payment Requirements
Invalid
Asset Verified
—
Facilitator
—
Input Schema
{
"body": {
"additionalProperties": false,
"properties": {
"url": {
"description": "Public HTTPS website or robots.txt URL.",
"format": "uri",
"maxLength": 2048,
"type": "string"
},
"userAgent": {
"default": "*",
"description": "Crawler user-agent whose effective rule group should be selected.",
"maxLength": 100,
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"bodyType": "json",
"discoverable": true,
"method": "POST",
"type": "http"
}
Output Schema
{
"properties": {
"job_id": {
"type": "string"
},
"status": {
"type": "string"
},
"thread_id": {
"type": "string"
}
},
"type": "object"
}
Recent Health Checks
| Time | Status | HTTP | Latency | Error |
|---|---|---|---|---|
| 2026-09-05 13:54:02 | down | 503 | 127ms | HTTP 503 |
| 2026-09-05 05:18:38 | down | 503 | 74ms | HTTP 503 |
| 2026-09-04 18:11:26 | down | 503 | 459ms | HTTP 503 |
| 2026-09-04 10:44:08 | down | 503 | 703ms | HTTP 503 |
| 2026-09-04 04:35:08 | down | 503 | 68ms | HTTP 503 |
| 2026-09-03 23:07:10 | down | 503 | 498ms | HTTP 503 |
| 2026-09-03 06:56:06 | down | 503 | 741ms | HTTP 503 |
| 2026-09-03 04:33:28 | down | 503 | 240ms | HTTP 503 |
| 2026-09-02 18:21:39 | down | 503 | 103ms | HTTP 503 |