Skip to main content
Schedules let Octoparse run tasks automatically at defined times or intervals. They are useful for recurring data scraping workflows such as price monitoring, lead list updates, inventory checks, or routine reporting. Octoparse supports scheduled runs in both cloud and local. Cloud scheduling is recommended for most workflows because it runs without depending on your local computer. Local scheduling is available for cases where Cloud Extraction does not work or where the target website requires a local IP address for login or access.

​
When to use schedules

Use schedules when:
  • The same data needs to be scraped repeatedly
  • You need daily, weekly, or hourly updates
  • The task should run outside working hours
  • Data needs to be ready before a report or downstream workflow
  • You want to reduce manual task starts

​
Scheduling workflow

1

Build and test the task

Confirm the task extracts the correct fields and handles pagination or detail pages correctly.
2

Choose the run environment

Use Cloud Extraction when the task should run without your computer. Use Local Extraction when it depends on your device, local network, local IP, or an active local session.
3

Set the schedule

Choose when and how often the task should run.
4

Configure export

Set up manual or automatic export depending on where the data should go.
5

Monitor results

Check run history, logs, and exported records after scheduled runs. For cloud-specific time zones, queues, overlaps, and failures, see Schedule cloud runs.

​
What to check before scheduling

Before enabling a schedule, check:

​
Limit the runtime of a scheduled local task

For a local schedule, set Maximum runtime to stop a task that remains active longer than expected. Enter the limit in whole minutes; enter 0 when the run should have no time limit. The timer starts with the scheduled local run. When the limit is reached, Octoparse stops the task whether it is running or paused. This prevents a stalled or unusually slow task from occupying the local computer indefinitely and makes it easier to leave enough time before the next scheduled run.
Maximum runtime applies only to scheduled local extraction. It does not limit a manually started local run or a cloud run, and it does not guarantee that the task will finish collecting all intended data before the limit.
  • Test the task manually before scheduling it.
  • Start with a conservative frequency.
  • Monitor the first few scheduled runs.
  • Use logs to diagnose failures.
  • Review exported records for missing fields or duplicates.
  • Adjust frequency if the website updates less often than expected.
Avoid scheduling tasks more frequently than needed. Excessive runs can create duplicate data, increase usage, and raise the chance of website blocking.