Python client library

octoparse-client is the official Python client for Data Hub. It wraps only the public /v1 REST API and depends on httpx alone. Methods map one-to-one to endpoints, parameter names match REST, and return values are the raw data payload (dict / list). Current version 0.2.9, Python 3.10+.

Install

pip install octoparse-client

Initialize

from octoparse_client import Client

client = Client(api_key="<your API key>")

Parameter	Description
`api_key`	Data Hub API key. If omitted, reads `OCTOPARSE_API_KEY`. If still missing, the client is anonymous and can call only anonymous endpoints.
`base_url`	Service URL. If omitted, reads `OCTOPARSE_BASE_URL`, defaulting to `https://api-datahub.octoparse.com`. Pass this explicitly for local or staging environments.
`timeout`	Per-request HTTP timeout in seconds. Default 90.

Client supports the with statement and closes connections on exit. You can also call client.close(). The SDK does not load .env files; load them yourself if needed.

Method to endpoint map

Method	Endpoint	Notes
`meta()`	`GET /v1/meta`	Platform constants
`search(q, *, limit, offset, ...)`	`GET /v1/data-apps`	One page of cards, including `pagination`
`get_app(app_id)`	`GET /v1/data-apps/{app_id}`	Full app detail
`run(app_id, inputs, *, wait, max_records, version, build)`	`POST /v1/data-apps/{app_id}/runs`	Start a run
`call(app_id, inputs, *, max_records, timeout, raise_on_failure, ...)`	`POST` + poll	Blocking helper: wait until terminal or timeout
`get_run(run_id, *, wait)`	`GET /v1/runs/{run_id}`	Status and billing snapshot
`list_runs_page(filters)`	`GET /v1/runs`	One page of runs, including `pagination`
`list_runs(filters)`	`GET /v1/runs`	One page of runs, items only
`iterate_runs(filters)`	`GET /v1/runs`	Iterator that pages automatically
`cancel(run_id)`	`POST /v1/runs/{run_id}/cancel`	Cancel a run
`get_records(run_id, *, offset, limit, fields)`	`GET /v1/runs/{run_id}/records`	One page of result records
`iterate_records(run_id, *, batch, fields)`	`GET /v1/runs/{run_id}/records`	Iterator that pages automatically
`export_records(run_id, format)`	`GET /v1/runs/{run_id}/records`	Full export as `jsonl` / `csv` text
`list_datasets(*, offset, limit)`	`GET /v1/datasets`	Dataset list
`set_dataset_retention(dataset_id, retained)`	`PUT /v1/datasets/{dataset_id}/retention`	Set retention flag
`get_dataset_records(dataset_id, *, offset, limit, fields)`	`GET /v1/datasets/{dataset_id}/records`	One page of dataset records
`iterate_dataset_records(dataset_id, *, batch, fields)`	`GET /v1/datasets/{dataset_id}/records`	Iterator that pages automatically
`account()`	`GET /v1/account`	Account info and lifetime spend
`billing(*, group_by, created_from, created_to, tz_offset)`	`GET /v1/billing`	Billing aggregation

Run-list filters support status, data_app, triggered_by, run_kind, credential, created_from, created_to, plus offset / limit. Publishing, operations, and secrets endpoints are not wrapped; call REST directly.

Typical usage

from octoparse_client import Client, ApiError, RunFailed

with Client(api_key="<your API key>") as client:
    # Discover
    page = client.search("reviews", limit=5)
    for card in page["items"]:
        print(card["app_id"], card["namespace"], card["app_name"])
    detail = client.get_app("carol/reviews-query")

    # Run and consume. Timeouts are seconds. TimeoutError does not cancel the server-side run.
    run = client.call(
        "carol/reviews-query",
        {"product": "p-9001"},
        max_records=100,
        timeout=120,
        raise_on_failure=True,
    )
    for record in client.iterate_records(run["run_id"]):
        print(record)

    # Non-blocking: start, then poll yourself
    started = client.run("carol/reviews-query", {"product": "p-9002"}, wait=0)
    polled = client.get_run(started["run_id"], wait=60)

    # Reconcile spend
    print(client.billing(group_by="data_app", tz_offset=480))
    for r in client.iterate_runs(created_from="2026-09-01T00:00:00+08:00"):
        print(r["run_id"], r["status"], r["billing"]["total"])

    # Results are kept 90 days by default; mark datasets you need longer
    client.set_dataset_retention(run["dataset_id"], True)

Error handling

API error responses raise ApiError. Fields match the REST error object:

Attribute	Meaning
`status_code`	HTTP status
`code`	Stable error id for branching
`category`	Broad class
`message`	English description
`retryable`	Whether the same request can be retried
`details`	Field-level issues on input validation failure, otherwise `None`

from octoparse_client import ApiError

try:
    client.run("carol/reviews-query", {})
except ApiError as e:
    if e.code == "invalid-input":
        for d in e.details or []:
            print(d["path"], d["message"])
    elif e.retryable:
        # back off and retry
        ...
    else:
        raise

RunFailed is raised when call(..., raise_on_failure=True) ends in FAILED / CANCELLED / EXPIRED. Its run attribute holds the full run object.

App references

get_app(), run(), call(), and the run-list data_app filter accept both forms: two-segment <namespace>/<app_name> (human-readable, breaks after rename) and stable app_id (app_<hex>, survives rename). For long-lived integrations, store app_id. Point-to-point shared apps are absent from market search; use search("", shared_with="me").

GET STARTED

QUICK START

REFERENCE

DATA HUB API

Install

Initialize

Method to endpoint map

Typical usage

Error handling

App references

​Install

​Initialize

​Method to endpoint map

​Typical usage

​Error handling

​App references

Install

Initialize

Method to endpoint map

Typical usage

Error handling

App references