SFGate API endpoint
Returns public SFGATE article metadata and body paragraphs from a canonical article URL. Article pages are served through proxied browser renderers; the service does not bypass authentication or access controls, and marks paywalled=true when only a teaser is public.
/sfgate/articleQuery parameter: url is a canonical HTTPS sfgate.com/<category>/article/<slug>-<id>.php URL (the category can have more than one segment, for example travel/burningman). Returns public metadata and body paragraphs. paywalled is true when the page's content tier is not free, in which case paragraphs is only the public teaser. Article pages are fetched through proxied browser renderers; when none can load the page the endpoint returns a 503 upstream error.
| Field | In the sample | Value in the sample |
|---|---|---|
| url | Has a value | sfgate.com |
| title | Has a value | 12 characters |
| description | Has a value | 138 characters |
| published_at | Has a value | 2020-10-01 |
| updated_at | Has a value | 2023-08-31 |
| section | Has a value | “Home” |
| image_url | Has a value | s.hdnux.com |
| paragraphs | Has a value | 11 entries |
| Parameter | Type | Required | Default | Description | Example |
|---|---|---|---|---|---|
| url | string | Yes | Canonical SFGate article URL | https://www.sfgate.com/bayarea/article/example-story-22400001.php | |
| x-api-key (header) | string | Yes | API key required |
curl -X GET "https://api.crawlora.net/api/v1/sfgate/article?url=https%3A%2F%2Fwww.sfgate.com%2Fbayarea%2Farticle%2Fexample-story-22400001.php" \ -H "x-api-key: $CRAWLORA_API_KEY"
{
"code": 200,
"data": {
"url": "https://www.sfgate.com/bayarea/article/example-story-22400001.php",
"title": "Example story",
"section": "Bay Area",
"authors": [
"Example Writer"
],
"paragraphs": [
"Example public paragraph."
]
}
}Request schema
No body schema
Response schema
#/definitions/publicnews.articleResponseDoc
| Field | Type | Required | Enum | Bounds | Example | Description |
|---|---|---|---|---|---|---|
| data | newsplatform.ArticleResponse | No | ||||
| data.authors | array | No | ||||
| data.description | string | No | ||||
| data.image_url | string | No | ||||
| data.paragraphs | array | No | ||||
| data.paywalled | boolean | No | Paywalled is true when the publisher serves only a teaser of this article to anonymous readers, so Paragraphs is a partial body rather than the full text. It is omitted when the full public body was returned. | |||
| data.published_at | string | No | ||||
| data.section | string | No | ||||
| data.title | string | No | ||||
| data.updated_at | string | No | ||||
| data.url | string | No |
| Status | Description | Schema |
|---|---|---|
| 400 | Bad Request | #/definitions/app.Response |
| 404 | Not Found | #/definitions/app.Response |
| 503 | Service Unavailable | #/definitions/app.Response |
Use environment variables for secrets and keep Crawlora API keys server-side.
curl -X GET "https://api.crawlora.net/api/v1/sfgate/article?url=https%3A%2F%2Fwww.sfgate.com%2Fbayarea%2Farticle%2Fexample-story-22400001.php" \
-H "x-api-key: $CRAWLORA_API_KEY"