API reference

This section lists every public Data Hub REST endpoint by resource. Each endpoint page covers method and path, auth, parameters, real response samples and field notes, possible errors, and the matching client-library method. This page only holds conventions shared by every endpoint.

Base URL

Item	Value
Base URL	`https://api-datahub.octoparse.com`
Path prefix	Every endpoint starts with `/v1`
Protocol	HTTPS; request and response bodies are JSON

The /v1 contract is additive only: new fields and endpoints may appear over time, but published fields keep their names and meaning. Ignore unknown fields when you integrate, and do not depend on field order.

Authentication

Authenticated endpoints use standard Bearer auth. Put the Data Hub API key in the Authorization header:

Authorization: Bearer <your API key>

Create an API key in the Octoparse account center. Access tokens from web login or OAuth go in the same place. Each endpoint page labels its auth class:

Class	Meaning
Anonymous	No credential required
Optionally authenticated	Works without a credential; with one you unlock logged-in filters or more content
API key required	Credential required
App author only	Credential required, and only the publisher of that app may call it; everyone else gets `404`

The Data Hub API only accepts Authorization: Bearer. It does not accept credentials in URL query parameters. This is not the same convention as the Octoparse scraper MCP in the top navigation, which uses an x-api-key header. Do not mix them. Treat an API key like an account credential: never commit it to a repo, shared config, or public screenshot.

Response shape

Successful responses wrap the payload in data. Error responses wrap the payload in error. The two never appear together.

{ "data": { "...": "..." } }

{
  "error": {
    "code": "unauthorized",
    "category": "forbidden",
    "message": "valid API key or access token required (Authorization: Bearer <credential>)",
    "retryable": false
  }
}

A few endpoints return non-JSON content (raw Markdown, CSV / JSONL text, or a zip binary). Those pages call that out explicitly.

Errors

Field	Meaning
`code`	Stable error id for branching, for example `unauthorized`, `app-not-found`, `balance-negative`
`category`	Broad class: `invalid_input`, `not_found`, `forbidden`, or `temporary`
`message`	English developer-facing text. Read it; do not branch on it
`retryable`	Whether a retry of the same request can succeed. If `true`, back off and retry. If `false`, fix the request or wait for user action
`details`	Present only on input-validation failures: an array of field-level `path` and `message` items

Each endpoint page lists codes unique to that endpoint. These appear on most endpoints:

HTTP	`code`	Meaning
401	`unauthorized`	Missing or invalid API key
404	`*-not-found`	Object does not exist or is invisible to the current credential. The two cases are intentionally not distinguished
400	`invalid-input`	Request body or parameters are invalid

Returning the same 404 for “missing” and “invisible” is intentional: private apps, other people’s runs, and other people’s datasets never leak existence through the error code.

Pagination

List endpoints use offset / limit query parameters and return a pagination object:

{
  "data": {
    "items": [ "..." ],
    "pagination": {
      "offset": 0,
      "limit": 50,
      "count": 50,
      "total": 132,
      "has_more": true
    }
  }
}

total is the filtered total. When has_more is true, add count to offset and continue. Per-endpoint limit caps are documented on each page.

Time parameters

Endpoints that take a time range (run list, billing aggregation, publisher analytics) accept ISO 8601 timestamps. Prefer an explicit offset such as 2026-09-01T00:00:00+08:00. Unix epoch seconds are not accepted.

App references

Anywhere an app is identified, both forms work:

Form	Example	Notes
Stable id	`app_a1b2c3d4e5f6`	Survives renames. Prefer this for long-lived integrations
Human-readable	`carol/reviews-query`	`<namespace>/<app_name>`. Breaks if the publisher or app name changes

Point-to-point shared apps do not appear in market search. List them with the shared-with-me filter on the search endpoint. For scheduled jobs and long-lived automation, store app_id.

Run statuses

Status	Meaning	Terminal
`PENDING`	Accepted, not yet queued	No
`QUEUED`	Waiting to run	No
`RUNNING`	In progress	No
`SUCCEEDED`	Succeeded	Yes
`PARTIALLY_SUCCEEDED`	Partially succeeded; produced records are usable	Yes
`FAILED`	Failed; no data fee	Yes
`CANCELLED`	Cancelled; produced records are kept and billed	Yes
`EXPIRED`	Timed out	Yes

Endpoint groups

Discover Data Apps

Search, detail, versions, README, specs, and translations.

Runs and results

Start, poll, list, cancel, read records, and debug traces.

Datasets

Persistent containers for run results and retention flags.

Account and billing

Lifetime spend and billing aggregates by period and dimension.

Secrets

Upstream credentials that app authors store for their own apps.

Publishing and operations

Draft → Build → Release, ops switches, sharing, translations, contract tools, and publisher analytics.

GET STARTED

QUICK START

REFERENCE

DATA HUB API

Base URL

Authentication

Response shape

Errors

Time parameters

App references

Run statuses

Endpoint groups

Discover Data Apps

Runs and results

Datasets

Account and billing

Secrets

Publishing and operations

​Base URL

​Authentication

​Response shape

​Errors

​Pagination

​Time parameters

​App references

​Run statuses

​Endpoint groups

Discover Data Apps

Runs and results

Datasets

Account and billing

Secrets

Publishing and operations

Base URL

Authentication

Response shape

Errors

Pagination

Time parameters

App references

Run statuses

Endpoint groups