Schedule task runs

Cloud schedules start an Octoparse task automatically at a time or interval you choose. They are useful for recurring data scraping such as price monitoring, inventory checks, directory refreshes, and scheduled reports. A schedule controls when Octoparse requests a run. Cloud nodes, task splitting, and account limits control when that run can actually start. If no capacity is available, the run waits in the cloud queue.

​

Before you create a schedule

Scheduled runs, in the cloud or on your computer, need Standard or above. Make sure that:

• The task completes a representative test and returns the expected fields and records, including pagination, detail-page navigation, and waits.
• If the task requires login or cookies, verify that the credentials and session remain valid in the cloud environment.
• The interval is long enough for one run to finish under normal conditions.
• Any required export or downstream workflow is configured separately.

For unattended runs, use Cloud Extraction. Use a local schedule only when the task depends on your device, local network, or local IP.

​

Schedule one task

1. Open the task in the Task List.
2. In the Next Run column, select Not Set for the cloud schedule.
3. Choose the run frequency and start time.
4. Save the schedule settings, then select Schedule ON.
5. Confirm the next execution time shown in Next Run.

The schedule starts applying immediately after you turn it on. To stop future automatic runs, open the schedule and select Schedule OFF. Turning a schedule off does not stop a cloud run that is already in progress; stop that run separately from its cloud run controls. To schedule a task built with Cloud Agent, select Schedule Runs in the task’s menu.

​

Choose a frequency and time zone

Octoparse supports one-time and recurring schedules. Depending on the interface and account, recurring choices include minute, hourly, daily, weekly, and monthly intervals. Choose the least frequent interval that keeps the data fresh enough for your use case. Octoparse records the time zone associated with your login and converts the schedule to UTC for cloud execution. If you sign in from another time zone, the scheduled time may be converted and displayed in the current time zone. Always check Next Run after signing in or changing the schedule, especially when a team works across regions or when daylight-saving changes apply. The time recorded by a task is separate from the schedule time. For example, a field that extracts the current date and time during a cloud run uses the cloud environment’s UTC time. Convert that field in the task if your downstream data needs another time zone.

​

Schedule a task group or several tasks

You can apply a schedule to multiple tasks in either of these ways:

• Task group: Switch the Task List to Task Group view, select the group, and use the schedule control for the group.
• Batch selection: Select several tasks in the Task List, then choose Schedule Run and configure the shared schedule.

Group and batch schedules reduce repeated setup, but they do not reserve a separate pool of cloud nodes. Each task still consumes the nodes required by its active subtasks, and the account’s concurrency limits still apply.

​

Reuse saved schedule settings

When you configure a schedule, save it with a recognizable name. You can then apply the saved settings to other tasks instead of entering the frequency and time again. Reuse is helpful for a data scraping program with a common refresh policy. Review the target task before applying a saved setting: the same interval may be appropriate for a lightweight directory task but too aggressive for a long, rate-limited workflow.

​

Read Next Run and schedule status

Use the Task List to check whether a schedule is active and when Octoparse expects to trigger it: Next Run confirms the schedule projection, not successful data scraping. Verify the cloud run status and output after the run starts.

​

Overlapping schedules and queues

If a task’s previous scheduled run is still active when the next interval arrives, Octoparse may skip that overlapping occurrence. It does not keep adding identical runs for the same task indefinitely. This is different from a normal resource queue:

• Overlap: the previous run has not finished, so the next scheduled occurrence may be skipped.
• Queue: the run was accepted, but cloud nodes or the account’s parallel-task limit are occupied. It waits and starts when capacity is released.

When several queued tasks compete for capacity, Octoparse uses their task priorities to determine which work is considered first. Priority changes queue order; it does not add nodes. A splittable task can also occupy several nodes, causing other scheduled tasks to wait even when only one task is running. If a refresh must finish sooner, review the interval, reduce competing schedules, or give the task an appropriate higher priority. Do not schedule every task at the shortest interval by default: frequent runs can create queues, increase duplicate checks, and raise the risk of site blocking.

​

Validate and handle failures

Treat the first scheduled runs as a validation period:

1. Confirm that the cloud run starts at the expected time.
2. Check the record count and key fields against a local sample.
3. Review pagination, login, waits, and detail-page steps in the run details.
4. Check event logs, subtask status, and available error screenshots when a run is incomplete.
5. Check whether the task was queued, skipped because of overlap, or stopped after a task error.
6. Fix the task or schedule, then run a manual cloud test before relying on the next recurrence.

A schedule only repeats the configured workflow; it does not repair selectors, renew an expired login, or make an unsplittable task parallel. If a task repeatedly fails, pause the schedule while you diagnose it so failed runs do not continue to consume capacity.

​

Schedule a local task

Cloud scheduling fits most recurring work because it doesn’t depend on your computer. Schedule a local run instead when Cloud Extraction doesn’t work for the site, or when the site needs your local IP address for sign-in or access. Your computer and the Octoparse app must be on when the run is due. Like cloud schedules, local schedules need Standard or above. For a local schedule, set Maximum runtime to stop a task that remains active longer than expected. Enter the limit in whole minutes; enter 0 when the run should have no time limit. The timer starts with the scheduled local run. When the limit is reached, Octoparse stops the task whether it is running or paused. This prevents a stalled or unusually slow task from occupying the local computer indefinitely and makes it easier to leave enough time before the next scheduled run.

​

Recommended operating pattern

• Build and test locally.
• Run the task manually in the cloud once.
• Start with a conservative frequency.
• Enable the schedule and verify Next Run.
• Monitor the first few runs and compare output.
• Adjust the interval, priority, or task split after observing real runtime and queue behavior.

For cloud capacity and queue behavior, see Cloud resources and queues. For current interface steps, see Schedule tasks to run and Lesson 6: Schedule regular runs in the Octoparse Help Center.