> ## Documentation Index
> Fetch the complete documentation index at: https://www.octoparse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# How cloud tasks are split into subtasks

> Learn which Octoparse task structures can be split into subtasks, how cloud nodes run them, and how to improve parallel data scraping.

Octoparse can divide an eligible cloud task into smaller **subtasks**. Each subtask handles part of the input and uses one cloud node while it is running. Several subtasks can run at the same time, so a suitable task can finish sooner than one sequential run.

Splitting is based on the task's workflow and input structure. It is not a promise that every task will use every node in your account.

Octoparse splits the first eligible list it encounters in the workflow. A later list is not split again as a separate layer, so nesting several list loops does not multiply the number of subtasks indefinitely.

## Which tasks can be split

The most common splittable loop modes are:

| Loop mode         | How Octoparse divides the work                  | Typical use                                                     |
| ----------------- | ----------------------------------------------- | --------------------------------------------------------------- |
| **List of URLs**  | Assigns different URLs to different subtasks.   | Scrape many product, article, or detail-page URLs.              |
| **Text List**     | Divides the values in the list among subtasks.  | Search several keywords, locations, or IDs.                     |
| **Fixed List**    | Assigns fixed loop items to different subtasks. | Scrape repeated elements with known, independent positions.     |
| **Variable List** | Splits the list into three subtasks.            | Process a dynamically discovered list with limited parallelism. |

Single-element loops normally cannot be divided into independent work:

* **Single element:** the loop contains one element to process.

Variable List uses a fixed three-subtask split rather than scaling the number of subtasks with the list size. If you need more parallelism and the item structure is stable, consider converting the workflow to a Fixed List or a URL List.

If a task has no splittable loop, it runs as one execution unit even when the account has several cloud nodes.

## How splitting and node allocation work

The process is:

1. Octoparse identifies an eligible loop and its input items.
2. It creates subtasks for portions of those items.
3. Each available cloud node takes one subtask at a time.
4. When a node finishes, another waiting subtask can use it.
5. The run finishes after all subtasks complete or stop.

The number of subtasks can be greater than the number of available nodes. For example, a list with 100 URLs may produce many subtasks while a six-node account runs only up to six of them concurrently. The remaining subtasks wait for a node to become available.

### Subtask limits and input limits

Current 9.x configurations use different subtask limits for different task types:

| Task type            | Maximum subtasks per split round | Maximum subtasks for the task |
| -------------------- | -------------------------------: | ----------------------------: |
| OTD task             |                              200 |                         1,000 |
| Python template task |                              800 |                         4,000 |

For OTD tasks, the input method also affects the split limit. **Enter manually** supports up to 200 subtasks. **Import from file**, **Batch generate**, and **Import from task** can use up to 1,000 subtasks. Enter-manually, file-import, and batch-generated URL input is limited to 1 million URLs; Import from task can exceed that input size but remains limited to 1,000 subtasks.

More nodes can increase parallel capacity, but they do not automatically make every task faster. The actual result also depends on the number of input items, task concurrency, page loading time, waits, login state, target-site limits, CAPTCHA, and queue availability.

## What splitting changes for a run

Splitting affects both speed and troubleshooting:

* **Speed:** independent subtasks can process different inputs at the same time.
* **Capacity:** each active subtask consumes one cloud node, leaving fewer nodes for other tasks.
* **Progress:** subtasks can start and finish at different times.
* **Recovery:** one stopped subtask may affect only part of the input, but its records still need to be checked before restarting it.
* **Output:** duplicate handling and incomplete subtasks can make the final saved count lower than the number of records attempted.

Splitting does not remove dependencies inside a subtask. If each page must be opened after the previous page, or all items require one shared login session, parallel execution may provide little benefit or may make the target site less stable.

## How to make a task more suitable for splitting

Use independent inputs whenever possible:

1. Build a URL List, Text List, or Fixed List loop with clear items.
2. Keep each item self-contained so a subtask can process it without another subtask's state.
3. For a multi-page workflow, consider scraping detail-page URLs first, then using those URLs in a separate detail task.
4. If the three-subtask Variable List limit is the bottleneck, convert it to a Fixed List or URL List only when the item positions or URLs are known and stable.
5. Test the task with a small input list before increasing the volume or parallelism.

Do not add artificial splitting to a workflow that depends on strict ordering, a shared session, or a target site that limits concurrent requests.

## Check how a run was split

After starting the task in the cloud:

1. Open the cloud run window from the Task List.
2. Open **Subtask Status** to see the subtasks, their status, timing, and data extracted.
3. Use the event log to check how many subtasks were created and whether they were assigned gradually.
4. Compare subtask output and elapsed time to find slow or stopped portions.

The number of subtasks is not the same as the number of active nodes. Use [Cloud resources and queues](/docs/en/platform/cloud-resources-and-queues) to understand node capacity and queueing, and [Cloud run status](/docs/en/platform/cloud-run-status) to interpret the run-level result.

## When splitting does not improve the result

Review the task instead of simply adding more nodes when:

* Most subtasks spend time waiting for the same page or API response.
* The target site starts returning CAPTCHA, rate-limit, or blocked responses.
* Login cookies are not valid in parallel sessions.
* Subtasks stop after producing no data.
* The task has too few independent input items to keep nodes busy.
* The run is queued because other tasks already consume the account's nodes.

Run a small cloud test, compare the Data List with a known-good result, and check stopped subtask logs and screenshots before scaling the workload.

## Related pages

<Columns cols={2}>
  <Card title="Cloud resources and queues" href="/docs/en/platform/cloud-resources-and-queues">
    Understand node consumption, queueing, priority, and throughput.
  </Card>

  <Card title="Cloud run status" href="/docs/en/platform/cloud-run-status">
    Interpret run and subtask progress after a cloud run starts.
  </Card>

  <Card title="Standard vs boost mode" href="/docs/en/platform/standard-vs-boost">
    Compare the available execution modes for cloud and local runs.
  </Card>

  <Card title="Monitor cloud runs" href="/docs/en/platform/monitor-cloud-runs">
    Inspect subtask details, data volume, logs, and screenshots.
  </Card>
</Columns>
