> ## Documentation Index
> Fetch the complete documentation index at: https://www.octoparse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect Data Hub MCP to your agent

> Connect the Data Hub MCP without choosing a Data App first, then let your agent search for, understand, and run the right data capability for each task.

Use this tutorial if you do not yet know which Data App to use, or if you want the agent to find a capability for each task on its own. The general connection adds the Data Hub discovery and run tools to your agent. Once connected, you describe the data you need, and the agent searches for a suitable Data App, reviews its parameters, and then confirms and runs it.

<Note>
  **The general connection does not require picking an app first.** This tutorial and the "Codex: connect a specific app" and "Claude Code: connect a specific app" tutorials are two different usage patterns. A specific-app connection suits a clear, long-term, fixed capability. The general connection suits changing needs or cases where the app has not been chosen yet.
</Note>

## Understand the two connection methods

| Method                      | Order of operations                                                                           | Best for                                                                                                                 |
| --------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| **General connection**      | Connect the Data Hub MCP → search for apps inside the agent → review details → choose and run | You do not know which app to pick; each task may need a different capability; you want the agent to help with discovery. |
| **Specific-app connection** | Pick an app in Data Hub → copy that app's install prompt → connect the agent → run directly   | You have a fixed app; you want a smaller tool scope; you run a stable, repeated business process.                        |

Both methods call the same Data Apps on Data Hub. The only difference is where you choose the app. You can keep both kinds of connection, but give them recognizable names so the agent does not pick the wrong tool.

## What you can do afterwards

After this tutorial, you can do the following inside your agent, in order:

1. Search for Data Apps by keyword, platform, or scenario.
2. Review an app's purpose, input parameters, output fields, and billing.
3. Confirm the app and parameters, then start a run.
4. Check the status of an asynchronous task and get the final result.

<Note>
  For the full parameters and handling rules of `search_data_apps`, `run_data_app`, asynchronous status checks, and the large-result `handoff`, see <a href="/docs/en/datahub/mcp-capabilities" target="_blank" rel="noopener noreferrer">Data Hub MCP capabilities</a>. This page focuses on connecting and first use.
</Note>

## Before you start

Have the following ready:

* An Octoparse account you can sign in to.
* An agent client that supports Streamable HTTP MCP, such as Claude Code, Codex, or Cursor.
* If you choose the API key method: create an Octoparse API key in advance.
* A simple data target for testing, such as "find a Data App that can enrich company information".

<Warning>
  An API key is an account credential. Never commit a real key to a code repository, shared config, public screenshot, or group chat. In OAuth mode, never copy a temporary access token from a browser session into a local config.
</Warning>

## Step 1: open the general MCP connection

<Steps>
  <Step title="Go to the Data Hub Open Platform">
    Sign in to the Octoparse website and open the Data Hub menu at the top, then click **Data Hub Open Platform**.
  </Step>

  <Step title="Open MCP connection">
    In the left navigation of the Open Platform, click **MCP connection** to open the Data Hub MCP Server page. The default toolset shown on the page can search for and run any Data App on Data Hub. You do not need to choose an app from the catalog first.
  </Step>
</Steps>

<Tip>
  The **Integration** section of any Data App also has an MCP connection link at the bottom that leads here. That link is a shortcut. It does not mean you have to choose that app first.
</Tip>

## Step 2: choose an authentication method

The general connection supports both API key and OAuth. The choice affects only authentication. It does not change how Data Apps are searched or run.

### Option 1: API key (recommended)

Best for long-term stable use, command-line clients, and automation. The configuration generated by the page contains:

```text theme={null} theme={null}
Authorization: Bearer <YOUR_API_KEY>
```

`Bearer` is simply the header format used to pass the API key. Use an Octoparse API key, not a temporary access token from a signed-in browser session.

If you do not have a key yet, create one in the <a href="https://www.octoparse.com/console/account-center/api-keys" target="_blank" rel="noopener noreferrer">Octoparse account center</a>. An API key is usually shown in full only once, at creation time. Store it in a trusted password manager.

### Option 2: OAuth sign-in

Best for interactive clients that support MCP OAuth. When you choose OAuth, the configuration contains no API key. The first time the client connects, it opens a browser where you sign in to Octoparse and confirm the authorization. The session may expire, in which case you authorize again.

## Step 3: copy the general install prompt

On the **MCP connection** page, choose your agent client and authentication method, then click **Copy install prompt**. **Copy MCP URL** only gives you the server address. Copy the complete content the page generates right now. Do not type the server address, tool scope, or headers from memory.

<Tip>
  The general server address shown on the page is based on `https://mcp-v2.octoparse.com`. The exact configuration and tool parameters may change, so always use what the Open Platform currently generates.
</Tip>

## Step 4: let the agent complete the configuration

The example below uses Claude Code. Codex and Cursor place the buttons differently, but the core steps are the same: paste the install prompt, choose the authentication method, allow changes to the current user's MCP configuration, then reload the client.

<Steps>
  <Step title="Send the install prompt to the agent">
    Start a new conversation and paste the complete prompt you copied. The agent should ask for the authentication method before writing the configuration. If it tries to write credentials into a project file, stop it and ask for the current user's local MCP configuration instead.
  </Step>

  <Step title="Confirm the authentication method">
    If you chose API key, provide the key securely when the agent asks. If you chose OAuth, provide no key and let the agent write a configuration without credentials.
  </Step>

  <Step title="Reload the client">
    After configuration, reload the MCP servers or restart the client. With OAuth, the first connection may show "Needs authentication". That is expected.
  </Step>
</Steps>

## Step 5: complete OAuth authorization (OAuth only)

If you chose API key, skip to the next step.

<Steps>
  <Step title="Trigger the connection in the client">
    In Claude Code, run `/mcp` and select the Data Hub server you just added, named `octoparse_datahub` by default. When it shows that authentication is needed, choose to authenticate. Other clients show a **Connect** or similar button in their MCP settings.
  </Step>

  <Step title="Sign in and authorize in the browser">
    The browser opens the Octoparse identity page. Check the domain and the current account, read the requested scope, and confirm. Follow the page prompt to return to the client. Some clients return automatically.
  </Step>

  <Step title="Confirm the server is enabled">
    Back in the client, confirm the "Needs authentication" state is gone and the server is enabled. If it still shows as unauthenticated, reload the client and try again.
  </Step>
</Steps>

## Step 6: search first, do not run yet

The key to the general connection is letting the agent discover Data Apps first. For the first use, explicitly ask it to search and compare only, and not to create a billable task yet. For example:

```text theme={null} theme={null}
Use Data Hub to search for Data Apps related to "company enrichment".
List the 3 most relevant results. For each, describe the purpose, required inputs, main output fields, and billing model.
Do not run anything yet. Wait for my confirmation.
```

The agent usually calls `search_data_apps` to search the catalog, then `get_data_app_details` to read the full contract of each candidate.

<Note>
  The number, names, and platform coverage of the apps an agent finds change as the Data Hub catalog changes in real time. This documentation does not provide a fixed list. Rely on what `search_data_apps` actually returns.
</Note>

## Step 7: confirm the app, then run

Choose one app from the candidates. Have the agent restate the parameters and planned actions, then run a small test:

```text theme={null} theme={null}
Choose the first Data App. First tell me its required parameters, default values, and billing model.
After I confirm, run with the smallest data volume only, and return the task status, record count, and the first 5 records.
```

<Steps>
  <Step title="Check inputs and cost">
    Confirm the required parameters, data scope, returned fields, and billing unit. If anything is unclear, have the agent call the details tool again instead of guessing parameters.
  </Step>

  <Step title="Run a small test">
    Only after confirmation, let the agent call `run_data_app`. A synchronous app returns results directly. An asynchronous app needs follow-up status checks.
  </Step>

  <Step title="Get asynchronous results">
    For asynchronous tasks, have the agent use the status and result tools to wait for completion, then return the final data. Do not treat "task submitted" as a successful extraction.
  </Step>

  <Step title="Check the results">
    Check the task status, actual record count, and key fields. A missing field in a single record may reflect a difference in the source data. If most records do not match expectations, switch apps or adjust the parameters.
  </Step>
</Steps>

## FAQ

<AccordionGroup>
  <Accordion title="Does the general connection require choosing a Data App first?">
    No. The general connection provides the search and run tools first. After connecting, the agent uses `search_data_apps` to find capabilities. Only the specific-app connection requires choosing an app in Data Hub first.
  </Accordion>

  <Accordion title="Should I choose API key or OAuth?">
    Prefer API key for long-term use, command-line clients, and automation. Choose OAuth when you want to sign in through the browser and your client explicitly supports MCP OAuth. OAuth sessions may expire and need re-authorization.
  </Accordion>

  <Accordion title="Configuration succeeded, but there is no search_data_apps tool">
    Confirm you used the general configuration from the Open Platform **MCP connection** page, not the restricted configuration of a specific app. Copy the current prompt again and reload the client.
  </Accordion>

  <Accordion title="OAuth keeps showing that authentication is needed">
    Trigger the connection in the client's MCP settings, complete the browser sign-in and authorization, then return to the client. Check whether the browser blocked the redirect, and confirm that the authorization page belongs to the official Octoparse identity service.
  </Accordion>

  <Accordion title="The agent found an app and ran it immediately">
    Write "search and compare only, do not run yet" explicitly in the prompt. When cost or large data volumes are involved, ask the agent to wait for confirmation before calling `run_data_app`.
  </Accordion>

  <Accordion title="The run returned only a task ID and no data">
    The app is probably asynchronous. Have the agent keep checking the task status and fetch the results when it completes. Do not resubmit the same task.
  </Accordion>
</AccordionGroup>

## Checklist

* You copied the general install prompt from the **MCP connection** page of the Data Hub Open Platform.
* You chose either API key or OAuth and did not mix the two credentials.
* The agent loaded the Data Hub general tools and can use `search_data_apps`.
* You searched and reviewed app details before confirming a run.
* You completed one real call with the smallest data volume and checked the final result.

## Already know which app to use?

If your business uses a fixed app long term, you can narrow the connection scope:

<CardGroup cols={2}>
  <Card title="Codex: connect a specific app" href="/docs/en/datahub/quick-start/agent-connection/codex">
    Pick a specific app first, then add it to Codex as a fixed tool.
  </Card>

  <Card title="Claude Code: connect a specific app" href="/docs/en/datahub/quick-start/agent-connection/claude-code">
    Pick a specific app first, then add it to Claude Code as a fixed MCP server.
  </Card>
</CardGroup>

<Note>
  The actual number of apps, their names, publishers, and prices returned by a search change as the Data Hub catalog changes. Rely on what the tool returns in each call.
</Note>